Well, for one thing, it can give your balls cancer (among other cancers).
In one town where I hunt, the drinking water had levels 11 times the "safe" threshold. Literally thousands of private wells are contaminated. (We had ours tested, and we're in the clear.)
Eating one serving of fish from the river is calculated to be the same as drinking the water for a month.
All of this has come to light in the last 5 or 7 years, so I don't know if there's been any good science on wildlife populations. Generally, I think that's a lower priority than the human effects of cancer juice in the drinking water.