I've had this, and this is what works for me:
1) Let hem soak for a few hours in the washer with HOT water.
2) Don't use fabric softener.
3) Wash your gym clothes separate from other clothes.
4) I use Tide or Kirkland wash soap, but I add a little extra baking soda, and sometimes vinegar.
5) RE-rinse the clothes after washing.
6) Dry thoroughly.
You don't have to do that every time, but once every few loads is adequate for most...but of course tis will vary. I like the idea of adding a small amount of bleach, I'm always afraid I'm gonna fukkk up my colored clothes though, lol. Also, this also leads a way into buying white clothes so bleach can be used.