Rock,
I, obviously, don't know all the details; but I can say from personal experience, there's usually some sort of radar that goes off in your head, but we as men, tend to ignore or discount it, simply because of our ego. The caveat is, I am saying this, assuming that the man involved isn't an overly jealous or possessive person. I was never the jealous type, nor had I ever been when it came to my ex, but when I felt something was wrong, I was dead on. I even told her a few months before she got caught (by the other guy's wife) that I knew something was going on. And I have been able to tell if something was fishy in each and every relationship I have been in, before and since. From the bit that I have seen, you don't appear to be possessive or jealous and the fact that you were worried that you may have been being insecure shows that you do have the ability to question and think things through. Like I said, I don't know the details, and I know this is an extremely emotional period, but try to find some time and a quiet spot and just think and be honest with yourself and you'll figure out what's best for you. But I will say, if you two aren't happy and aren't both getting what you want from the relationship, and BOTH of you aren't willing to work on it, you're spinning your wheels. I know from experience. It's even tougher when there are kids involved, but kids are a hell of a lot brighter and than we give them credit for and are usually a lot more emotionally intelligent than adults, so DON'T use them as a reason to stay in a bad, unsalvageable relationship. All you'll be doing is teaching them to accept an unfulfilling relationship as the norm and that, in my mind is tantamount to child abuse. My son, who was only 6 at the time, told me he was glad Mommy and I weren't together anymore because he could tell we weren't happy and we were always fighting.
I forgot to add - Happy wife = happy life, but you can't MAKE her happy. That's something she has to learn and achieve on her own. All you can do is help to KEEP her happy. In the end, everybody's happiness is dependent upon themselves, first and foremost.