Your analogy doesn't work because "mathematician" is a research position. You should compare mathematicians to computer scientists, not programmers.

You could make the same analogy and prove any side of the debate depending on the job you choose:

If you do plumbing, does that make you a plumber?

If you do painting, does that make you a painter?

If you do physics, does that make you a physicist?

If you drive vehicles, does that make you a driver?

Really, the debate here is that some people consider the word programmer to mean "computer scientist" or "computer expert", while others consider it to mean "person who programs". It's a debate on the meaning of a word and this kind of debate cannot be settled, because the word has multiple meanings. Everyone's right in their own way.