I just love clicking on something in an infinite scroll and then getting to press "back" later and waiting for 20 pages of results to stream back in to sift through.
I think infinite scrolling works best for uses cases where you don't typically open links after you've scrolled down. In Facebook's case, most of the interaction is ajax (Like, Commenting, etc.) so you don't lose the position.
For me Facebook works pretty well. I like to keep quickly scrolling down (using trackpad inertial scrolling) until I've skimmed over everything new in the feed. Forcing me to move and pinpoint the cursor to click a "Next page" link would certainly add friction to that.
Google's search results, OTOH, are a completely different use case, where you want to have full control to the paginated browsing, constantly navigating back and forth.
Perhaps this involves a distinction between "lean back" and "lean forward" use cases. Leaning back, it's nice to just keep on scrolling with simple finger swipes.
This has been my main complaint with "neverending Reddit" from the Reddit Enhancement Suite.
Sometimes when you click back, the "Page" of results you clicked from is omitted from the scroll. (result numbers jump from 50 -> 101 for example).
What's even more frustrating than losing your place is your place not actually existing anymore when you click back. I've grown to look at the result number before I click, so that I can try and find it quicker when I return.