I think DH needs to stop worrying about rocking the boat and defend his wife. Confront her over it, do it nicely at first "Hi, how have you been? I wanted to ask you something, you are not going to believe what I heard that you've been doing to me! I just had to call and ask you myself because I'm sure you wouldn't do something like and risk the kind of lawsuit you know I would pursue against something like that." See where it goes from there. Be tough and get your DH motivated too, he shouldn't worry about rocking the boat, he should worry about your honor and his too since it's his kids you guys are neglecting allegedly.
Edited for spelling, I'm tired on drugs,, I fell out by the coop in a dirt bath hole and landed on my rodeo knees and my wrist that has had 3 reconstructions.. I'm in pain,,