You should be using more sites and yes, scar tissue build-up will eventually become an issue. Scar tissue buildup will probably become an issue on any given site granted a long enough timeline.
If you are using a 23g and shooting delts over and over, it will probably be sooner, rather than later. The most annoying thing about scar tissue build-up is that, when you try injecting, the plunger just won't move anymore. It just doesn't go. So eventually, you will have to find another site. Glute and ventroglute work great, and for those .5cc shots, quads would almost certainly not be a problem, either. There may be other issues with scar tissue that I am not aware of, but this one is certainly the most immediate one.
Also, shooting into your anterior deltoid is asking for trouble. Use the lateral or posterior for injections, but I would not consider lateral and then posterior as two separate sites. You are just better off overall rotating more sites. Why aren't you?