In my experience alcohol is not good if you have joint problems and lower back pain. When drinking regularly I definitely notice a lot more pain, stiffness, clicks, and pops in my joints along with moderate lower back pain.
Ive had many friends take time off from gym longer than you on trt and with diets in check and just labor work and they still lost alot of size i dont believe it genetics can only help so much.