That's not really how medications work in the body.
For example, under certain doses, acetaminophen is perfectly safe. Over certain dosages, it's highly liver toxic. As an example, look at testosterone. It's not like everyone who uses small amounts of T has gynecomastia. If you start injecting 1g of test per week, you may very well start developing gyno absent the use of an AI. It's not like at TRT doses I have a little bit of gyno simply because I use the same drug that at high doses give people gyno. See what I'm saying? Not all effects are dose-dependent. Some are more binary in that they occur above certain thresholds and not below. If you were saying "we simply don't know" then I might be inclined to agree. But your blanket statement of "will add up to impaired abilities" is just not backed up by anything other than conjecture and a poor understanding of how substances work in the body. I'm not trying to disparage you, but it just doesn't make sense to think of medication where all side effects occur on a sliding scale. Yet that's what you are saying.