I wasn't concerned for this exact reason. I have a secure desktop password and take adequate precautions to ensure security.

The only real risk is with work computers, and anyone who doesn't have their desktop go to screensaver with password unlock after 5-15 minutes activity is just an idiot.

When I was in the bank getting my mortgage, I noticed their computers stay on for ~2 minutes before going back to a password screen for very obvious reasons.