Modern heat pumps can get a coefficient of performance of around 3, meaning that for every watt of electricity going in, you get 3 watts coming out inside your home (the other two watts are being removed from the air outside). For everything that isn't a heat pump, energy in = energy released in your home (coefficient of performance=1). So in the end, you get like a 33% rebate on cpu usage via heating cost savings(very rough number, since the actual coefficient of performance varies wildly on outside air temp)...non-negligible, but it doesn't make it free unfortunately.
In thermodynamic terms, an air conditioner is a heat pump hooked up the other way, though in practical terms I realize a heat pump is a device that is used for heating.
While not as mathematically elegant a comparison, you could certainly run the numbers for gas or any other heating method and find a similar cost comparision. They key point is that a computer heats the way a space heater heats, and heating your home with space heaters is expensive even in spite of the 100% efficiency at turning electricity into heat.
In the province of New Brunswick, Canada, many many homes have been (and still are) using electric baseboard heating even in midst of winter, often with less than ideal insulation in the house. Upgrading to a ductless/mini-split heat pump is seen as a pretty good option to reduce the electrical grid's load from heating, so much so that rebates have been offered by the government-owned electrical utility for people to upgrade.
