I guess what I'm asking is do any of you think it's a good idea to stick around a relationship in this type of situation.
Absolutely not. It's detrimental to your self-esteem. Tell him to give you a call when he sees enough other people, and cut off all contact.