I have the luxury of having a gym at my job for our 24 hour shifts, but most of the big dudes I knew growing up in hardcore gyms were blue collar dudes working construction and similar jobs. There was a dude, “Concrete Mike” who obviously layed concrete for 12 hours a day and then crushed it in the gym. He was the biggest and strongest dude I knew who never competed. Trained like a maniac. Another huge 270lb BBer I knew was a roofer.
Don’t use it as an excuse, figure out whether you’re better off training in the AM or PM and attack your workouts
On a side note, I felt my absolute worst while training when I worked a desk job, sitting at a computer 12 hours a day. Tight hips, back problems, and a different type of fatigue, not to mention corporate bullshit stress. I feel much better as a blue collar worker