You could end up biasing your community towards the mundane (the digg/fark model) if you aren't careful, but what I'm thinking about is something that instead of rewarding people who are "good voters" (for lack of a better term) instead seeks to reward those community members who consistently support high quality contributions.
I define "karma" as a means of valuing your contributions to the community, and it seems that this ought not be dominated simply by what you contribute in a literal sense, but also in what you contribute by encouraging others by voting up their valuable contributions. If our definitions differ, I can see that being the cause of the disagreement.