You should never listen to anyone advising to use only bare-bones console vim without the mouse or cursor keys, it's simply masochistic and doesn't contribute to your efficiency learning vim at all. Only if you expect to ever be using unix vim remotely through a terminal you should be aware of hjkl for navigation and visual mode for block selections. Any other constraints you impose on yourself are philoshical, if all you want is vim on OSX/Windows/desktop Linux.
That said, after you get comfortable using insert mode with just the most basic command mode stuff, I would highly recommend incrementally expanding your use of command mode. Learn about yank registers, movement units (words, sentences, brackets, quoted text), how to repeat commands, how to record and play macro's, split buffers and how to move around buffers and windows, etc. You could start by adding one command or feature each week and go from there.
If you never invest time expanding your vim knowledge and habits, there really isn't much of a reason to use vim at all. The whole point of vim is that it has so many crazy productivity-enhancing features. If you don't want to spend time to learn any of them then just don't bother, but realize that no 'easy to use', fully GUI-driven editor could ever come close to the editing efficiency you can theoretically get using a command-mode editor such as vim. There really is truth in what people say, once you master the most useful aspects of vim, you can never go back.
Some of us do use bare-bones console vim because we're using it remotely through a terminal. Or because, god help us, Solaris still doesn't install vim by default and we're stuck using plain old vi (at least on a customer's server, where I can't just go and install vim).
That's why I learned vi back in 2004, and while I still get tripped up when backspace doesn't work in insert mode because of some terminal setting, using hjkl has become second nature. There's a lot to be said for absolutely having to learn something.
Once I learned enough to be proficient doing simple editing, I switched to using gvim on my Windows machines, and then learned more as I found myself wondering if there was a better way to do something. Now I do virtually all of my coding in vim. The only downside of this I've found is that I'll occasionally C-w when typing in a web form and close my tab instead of deleting the last word.
I never got interested enough in emacs to learn it, but if I find myself in a situation where it's what I need to accomplish something I'll learn it. Need is a powerful motivator.
EDIT: I should also note that learning vi[m] on a computer with a Sun layout keyboard makes the idea of switching into command mode a lot more palatable. Esc is right there next to the 1, rather than somewhere in the next state.
Yes, if you're using vim through a terminal, or have to use vi on systems like Solaris, it's most definitely useful to know bare-bones vi. If you're on such a system, it's often the only thing you have besides 'ed'. Been there, done that (I found HP/UX possibly even worse than Solaris in terms of usability, unbelievable as it may sound ;-). For the vast majority of newcomers to vim, none of this matters though. They will be using gvim, MacVim or win32 vim exclusively, and probably wouldn't be very comfortable working on a Solaris or HP/UX box remotely anyway. SSH-ing between Linux or OS X boxes, or using a decent Windows terminal client, there really isn't any problem if you use the arrow keys, and if all you do is local editing on a desktop system, using the mouse to select text is a good way to start before learning all the different ways to cut/copy/paste using command mode.
Sometimes you're in a situation where your arrow keys don't match up with your terminal environment variable, and when you go to use them, it spews control characters all over your screen. I remember we were using Reflections as a terminal emulator, talking to hp-ux servers. I was helping out someone using vi. I had to keep telling him to keep his hands off the arrows keys because whenever he used them, it had that problem. I guess the short answer is: sometimes the arrow keys don't work. In some situations, the arrow keys LOOK like they're working, but VI doesn't know that you moved the cursor, and things get all screwed up. So I just don't trust the arrow keys at all. VIM lets you move around in INSERT mode, which is not something I'm used to either, because you can't do that in VI. Trying to use the arrow keys in insert mode in VI is just another mistake I see people make. Just ends up inserting control characters into the file. From experience, I just don't use arrow keys in VI/VIM.
That said, after you get comfortable using insert mode with just the most basic command mode stuff, I would highly recommend incrementally expanding your use of command mode. Learn about yank registers, movement units (words, sentences, brackets, quoted text), how to repeat commands, how to record and play macro's, split buffers and how to move around buffers and windows, etc. You could start by adding one command or feature each week and go from there.
If you never invest time expanding your vim knowledge and habits, there really isn't much of a reason to use vim at all. The whole point of vim is that it has so many crazy productivity-enhancing features. If you don't want to spend time to learn any of them then just don't bother, but realize that no 'easy to use', fully GUI-driven editor could ever come close to the editing efficiency you can theoretically get using a command-mode editor such as vim. There really is truth in what people say, once you master the most useful aspects of vim, you can never go back.