So many things can fuck with your estrogen levels. None in which would really have a dramatic effect on their own, but together.. possibly.
Plastics are a key one. Studies they have done have showed frogs exposed to water which has lots of plastic trash have a high incidence to testicles not dropping.. experiments with rats/mice have shown the same. This is one of the reasons they suggest pregnant women dont use plastic bottles/containers for food.
Even dairy products have been found to contain estrogen. So yeah.. it's sort of everywhere.
Perhaps you have a habit of microwaving everything in cheap plastic and you already have a weird thyroid issue making you even more susceptible to it.
But if your testicles have already dropped and you've gone through puberty, I wouldn't be too worried. If you start growing tits, you can always just get fat, no one will know the difference.