Asynchronous PHP with wget

Written by Ryan on September 23rd, 2009

I have a site where there’s a cache of a bunch of tracks (music site) and there are instances when a user does something and I need to rebuild that cache. The rebuilding takes a good five or ten seconds and will only become slower as we get more tracks so having the user wait while I do that is no longer feasible. I have a script that does a periodic rebuild (every fifteen min) but I’ve found that users don’t understand that their changes were accepted when they don’t show up for several minutes. It has to rebuild on demand but asynchronously. That way you get their changes applied pretty quickly but don’t have to wait on it.

I tried googling for asynchronous php solutions but couldn’t get anything to actually work. Brent accepted this answer but I had no luck with that. It would make the call but it wouldn’t do it asynchronously. I always had to wait for the thing to finish.

Some people suggested using the command line to run php and make the call that way but I couldn’t do that because of some architectural crap. I came up with this beauty though:
exec('wget -O /dev/null -o /dev/null -qb --no-cache --post-data=foo=bar http://theurl.com/whatever.php');
and it works perfectly. You can call a script on any server (not just your own like with php) and it runs in the background and discards the output quietly. Just one line and forget it.

Naturally, this only works on Linux servers with wget installed. Sorry windows folks

 

jQuery document.body is null error

Written by Ryan on September 23rd, 2009

I had a little “fun” with jQuery where it was giving me a “document.body is null” error…or sometimes it wouldn’t say anything but would just quietly fail. After some investigating, I made an interesting discovery about jQuery’s data() function.

I was chaining calls to data, like
myObject.data('foo', foo).data('bar', bar);

which generally works fine but apparently if foo is undefined, an error is thrown and the whole thing dies. If bar is undefined, it’s no problem, but something about trying to call data() with an undefined value breaks jQuery’s ability to make any further chained calls to that element.

The easiest solution I found is just dropping in a little
if (!foo) foo = null;
myObject.data('foo', foo).data('bar', bar);

And then everything works!

 

WP Post Thumbnail IE Error

Written by Ryan on September 14th, 2009

I just finished rebuilding Fighters.com and used WordPress for the whole thing which was a new experience for me. I hadn’t done any real work customizing before and it’s a really interesting design. The documentation isn’t quite what I’d like it to be but I was able to figure out everything eventually.

One particularly vexing issue was with the plug-in WP Post Thumbnail which gives you a ridiculously convenient way to add thumbnails to each post (Note that I intentionally linked to the older version because I like the interface a lot better). When you add/edit a post, you can upload an image right there and crop/scale it to different, configurable dimensions before adding it to the post. It was perfect for what I needed it for except the admin didn’t work in IE.

When IE tries to load a post edit page with WP Post Thumbnail enabled, it gives the very helpful:

Webpage error details
User Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET CLR 1.1.4322)
Timestamp: Mon, 14 Sep 2009 23:23:31 UTC
Message: Invalid argument.
Line: 12
Char: 12949
Code: 0
URI: http://www.fighters.com/wp-admin/load-scripts.php?c=1&load=jquery,utils,editor,quicktags&ver=33f9a1c72519a04d2fd7dd5a26c9ec8d

If you track down character 12,949 on line 12, it’s jQuery code that I believe is several functions inside an attr call. However what the exact problem is at that point eludes me. It’s trying to set variable[zIndex] = null; and I fail to see an invalid argument in that.

Now IE8 actually has a pretty nice js debugger but apparently it can’t stack trace code that was dynamically added to the page using jQuery.load, which of course this was, so I couldn’t see what has actually making the attr call. All I could figure out to do was put alert() messages all over all the dynamically loaded files and I eventually tracked it the offending call to this line in plugins/wp-post-thumbnail/wppt_repository.php:

jQuery('#bigimg').imgAreaSelect({hide:true});
I soon discovered that any call to imgAreaSelect threw the same error.

Looking for the source to that function, I found it within the imgAreaSelect Library which came with WP Post Thumbnail. The version that came with WP Post Thumbnail is v0.5.1 but if you go the imgAreaSelect site, you can get v0.9.1 (as of this writing).

Download the new imgAreaSelect code, unzip it, drop the .min.js file in your plugins/wp-post-thumbnail/js folder, add the css of your choice to plugins/wp-post-thumbnail/css/wppt-admin.css (if you choose the animated one, copy the images into your css folder too), and lastly, update plugins/wp-post-thumbnail/wppt.php’s wppt_js() function to have the correct path to your new imgAreaSelect js file. Presto! WP Post Thumbnail will be fully functional in IE.

At least that fixed it for me 😛

 

Javascript Copy to Clipboard Follow-Up

Written by Ryan on July 21st, 2009

While testing a site using the previously recommended ZeroClipboard in IE6, I noticed it was throwing occassional errors I believe are related to IE saying Flash objects are available before they actually are, or before they’ve registered their ExternalInterface functions with Javascript. Anyway, I tried changing the ZeroClipboard js file to make sure all Flash functions were registered before trying to use them which got rid of the errors, but IE6 still didn’t actually copy anything to the clipboard when hitting the button.

Rather than try to fix that, I took advantage of IE’s built in clipboard manipulation object, window.clipboardData and ended up with this:

if ($.browser.msie) {
  $('#clickme').click(function() { 
    if (window.clipboardData.setData('Text', $(this).data('copyText'))) {
      postCopyFunction(); // only happens if user allows copy
    }
  }).data('copyText', 'text to copy');
} else {
  var clip = new ZeroClipboard.Client();
  clip.setHandCursor(true);
  clip.addEventListener('complete', postCopyFunction);
  clip.setText(url);
  clip.glue('clickme');
}

Works about the same except in IE you get a confirmation about whether you want to allow access to the clipboard, which is acceptable. Arguably works better in IE now that it does in other browsers because you’re not loading a Flash widget.

Just a heads up if you want to use ZeroClipboard for a project.

 

Javascript Copy to Clipboard

Written by Ryan on July 17th, 2009

I was working a project recently that required being able to copy something to the clipboard using javascript in a browser compatible way. After much searching and trying different solutions, I found one called Zero Clipboard that works perfectly.

http://code.google.com/p/zeroclipboard/

Since there is no universal way to change the clipboard in javascript, the commonly accepted solution is to use a Flash movie to do the actual copying. You used to be able to make that completely transparent the to the user and just use Flash in the background but in Flash 10, Adobe changed Flash security so clipboard altering events had to be initiated by actually clicking somewhere in the Flash movie.

http://www.adobe.com/devnet/flashplayer/articles/fplayer10_uia_requirements.html

Obnoxiously, they changed file uploading in the same way, but I digress.

Zero Clipboard addresses all these issues by letting javascript create a transparent Flash movie, exactly covering any element in the DOM, and then letting that movie intercept button presses and initiate a copy command. The user can see you’ve used Flash by right clicking on the button/movie beyond that, it’s invisible to the user.

If you’re ever trying to do copy to clipboard in javascript, I highly recommend it. Very easy, very functional.

 

Sanitize your inputs

Written by Ryan on July 17th, 2009

I was hired by a friend of a friend a couple weeks ago to put their new site up that they’d had someone else build for them. I got the impression they used to have someone who could do these sort of release in house but he’d met with an untimely death (or maybe firing, though that’s less dramatic). Regardless, they wanted me to put their new site up for them, which I was happy to do. However when the client put the new version up in a sub-directory of their current siteso I could get access to it, they noticed it wasn’t working like it was supposed to.

At this point they felt like they had a working new site but something was going wrong with the deployment (even though they had actually done the deploying) and it was my job to figure out what was going on. Ok, no problem. I go poking around.

There were a couple issues. First of all, the developer had only used WARNING: SQL INJECTION POSSIBILITY DETECTED

In a flurry of righteous indignation, I email the client and let them know about the problem. I recommend they either confront the developer or, since he obviously doesn’t know what he’s doing, just pay me a little more to give everything a once over. I say, “There’s no sanitation of inputs before stuff is thrown into database queries, which, depending on the sensitivity of the information in your database, could be a big problem. It potentially exposes any information in your database to someone’s prying eyes.”

After a phone call wherein I explain how SQL injection works to the client, they agree to let me fix the problem. A little /^[0-9]+$/ later, we’re back in business. However the client also emailed the original developer and asked if there was any merit to my claims, to which he responds, “while there IS merit to it; it’s never been a problem in any of the stuff i’ve built structured similarly – this includes sites getting hundreds of thousands of visitors a day (for like microsoft, activision, disney/abc, etc). and there’s nothing in the database not viewable on the site itself, so there’s really no real reason to worry about it.”

Now, though a part of me dies inside to say it, let me say his argument isn’t totally without merit. If you only do select statements and if there’s absolutely no sensitive information in the database and if you’re not connecting to the database as a sufficiently privileged user and if you’re using the mysql extension instead of mysqli, I’m not aware of anything particularly malicious you could do with sql injection. But that’s a lot of if statements and we’re talking about something that’s not hard to fix. Just stop being so damn lazy! And of course, just because I’m not aware of any exploits doesn’t mean they’re not out there. One tenet of good programming is to always write your code as if someone smarter than you will try to break it.

Don’t leave holes just because you don’t know how to exploit them. Fill them in and guarantee your security.

mysql_query(“SELECT * FROM table_name WHERE table_id=$_GET[table_id] ORDER BY order ASC”);

– not hard to exploit

if (preg_match(‘/^[0-9]+$/’, $_GET[‘table_id’])) {

mysql_query(“SELECT * FROM table_name WHERE table_id=$_GET[table_id] ORDER BY order ASC”);

}

very hard to exploit

See how easy that was? I usually just make a function isInt so I don’t have preg_match’s all over the place

function isInt($test) {

return preg_match(‘/^[0-9]+$/’, $test);

}

But I needed to prove to that developer that his code was sloppy rather than just taking a “moral” stance so I got to thinking. Did any of those conditions I gave a minute ago jump out at you? “If you’re not connecting to the database as a sufficiently privileged user” but in the code he sent over, the developer was connecting as root so if he was doing that with the demo version of the site he set up on his own server, he should have all the permissions I’d need to make my case. Hmmmm…..

I’m sure there are multiple ways to go about this but I used mysql’s LOAD_FILE command, because I thought that made a pretty strong point.

So, how to exploit SELECT * FROM table_name WHERE table_id=$_GET[table_id] ORDER BY order ASC

Well, a good friend of mine suggested a UNION and that was key (I’d never done this before and never have to use unions). By using a union, you can stick a completely separate query’s result rows onto the initial query’s rows returned and get whatever data you want while still keeping the first query’s column names. The last part there is key because he does a mysql_fetch_assoc and only uses a couple columns.

So just as an experiment, I tried sending in table_id=2 UNION SELECT UNIX_TIMESTAMP() – – which php will then insert as

SELECT * FROM table_name WHERE table_id=2 UNION SELECT UNIX_TIMESTAMP() – – ORDER BY order ASC

The – – is crucial because it tells the server to treat anything afterwards as a comment, basically letting you throw away anything after what you’re inserting (and it’s two separate – marks next to each other but I can’t type that, apparently). You also need the 2 to satisfy there being something for table_id to equal but after that, it’s fair game.

Note, when I say I’m “sending in” table_id=2 UNION SELECT UNIX_TIMESTAMP() – -, what I mean is that I’m going to the site’s url with that parameter. Though this isn’t the site’s actual url, it’s like doing this:

http://baddeveloper.com/myclient/actions.php?table_id=2 UNION SELECT UNIX_TIMESTAMP() – –

Anyway, of course, that query doesn’t work as an experienced unioner could’ve told me because unions require both select statements to have an equal number of columns. And I could tell the sql was invalid because I stopped getting any output from php because it was generating an error in the mysql_query. So I started throwing in commas until BAM, I started getting output again. I ended up with table_id=2 UNION SELECT UNIX_TIMESTAMP(), 2, 2, 2 —

Now at this point I could tell the query was valid because the script was showing results again but it wasn’t showing my timestamp because the script only showed certain columns’ results and apparently the first one wasn’t one of them. I ended up with the timestamp in the third spot and the final query before I got really dirty was table_id=2 UNION SELECT 2, 2, UNIX_TIMESTAMP(), 2 – -. Now the script would dump out my timestamp but just being able to do that wouldn’t prove this was a real, dangerous exploit.

mysql’s LOAD_FILE is a nasty command. It lets any user with sufficient database privileges load the full data from a file on the system into a query result. For example, SELECT LOAD_FILE(‘/etc/whatever’) will return the contents of that file in the result. I assume there are restrictions about which files you can access based on the user mysql is running as but that didn’t prove to be an issue. Once I changed my query to table_id=2 UNION SELECT 2, 2, LOAD_FILE(‘/etc/passwd’), 2 – -, I was getting a list of all the users on the system and their home directories.

At this point, I could’ve gotten pretty malicious. I just used sql injection to find a huge hole in this guy’s system and could’ve stolen massive amounts of data, but maliciousness wasn’t really the point. I emailed him with a link to the exploit and admonished him with, “That’s why you always sanitize your inputs. I fixed it on our end but you should probably do it on yours too. Even if you’re just doing SELECT’s, there are vulnerabilities when you don’t validate.” And knowing is half the battle.

He has yet to respond or fix the hole in his own site. Nevertheless, it was a glorious victory and I think I provided a lot of value to my client by catching a gaping issue in their site and hopefully disuading them from using an inexperienced developer in the future.

Life Achievement Earned: Actually Hacked Something

Validate your inputs. Even if you don’t think you have to.

 

echo ‘Hello World!’;

Written by Ryan on June 15th, 2009

Just testing out posting. I plan to use this site for development, really. I traditionally just run apache on my home machine or use the client’s server but while working on a project for gametrailers.com, I needed to be able to work on my own database and fucking mysql won’t work – keeps throwing the helpfully named “Error 0”. Awesome.

After much googling and trying a bunch of “solutions”, I decided I was wasting time and just knocked this server into shape instead. Now we’re in business! We’ll see how much I post here.