I think you're right, but for other reasons.

First, most people don't like or can move far away.

Second, if your habits revolve around more abstract stuff than "seeing the office entrance where you smoke every day", you can end up changing some concrete circumstances and end up with the same abstract ones.