Perhaps historically there was evolutionary selection pressure towards higher sperm counts, and something about modern (as in Holocene) human behavior has eliminated this selection pressure, and thus sperm counts are now varying randomly.
Probably, right. The cave men or before that probably only had one or a few chances to mate while in our society we live in couples and can copulate many times per month, year in and year out, and thus a lower sperm count is countered by an increase in the number of tries.
I also find it hard to believe that sperm count should be such an important factor, there is still such a huge amount or sperms that have a chance to reach the egg.
But we haven’t got any observations of a steady baseline prior to the last century. There could have been declines (or rises!) during the previous centuries and millennia.
The chart from the meta-analysis does actually look very random, and the OP article discusses several confounding factors which might make us doubt the evidence. It’s easy to find patterns in noisy data.