Just to re-iterate Justin's warnings: while the thought of making everything non-blocking is exciting, this probably should only be used if you're expecting (and can't do anything about it) very slow queries.
Really? I've never scaled up a consumer web app this way, but is it really considered sound to insert blocking database calls in async web apps? This sounds wrong to me.
It works if the service runs a sufficient number of application instances behind a load balancer. In one of the videos about Tornado, a person from FriendFeed stated that FriendFeed runs multiple instances of the application with blocking calls to MySQL. There were many complaints about FriendFeed, but being slow was not one of them. (See slide 21 here http://bret.appspot.com/entry/tornado-tech-talk).
At scales where it matters, you're correct that it's really wrong.
The rational hope is that by the time you get to that scale you have people/code that know what they're doing and can fix it. Alas, there's many cases where that's not been true.
Why wouldn't you use it for normal operation? It would not make the user's experience better, but it would certainly increase the capacity of your service.
