I don't own or use a Mac anymore, but when I did in grad school a few years ago (using Tiger) I always got gcc binaries from here: http://hpc.sourceforge.net/

I never installed XCode.

I very very far from being an expert on this, so forgive me if I'm mistaken, but is there a reason these do not suffice?