My husband and I disagree......Someone please help!?

Here's the thing.....my hsband has a building outside so he can smoke and watch his sports, lpay poker and all that with his friends, but there are also females that come around and sit out there ALL the time with him and it's getting on my nerves. Plus this one in particular has always wanted to be with him. I have known my husband for 17 years. I don't trust this girl at all and I'm ready to leave him beacuse he has no respect for me to tell her she can't be comig around and sitting out there with him or for him to just come inside when she's around. Someone please tell me if I am in the wrong for not wanting my husband to spend more time with a skank than he does with his own wife and children. He only sees it as me being jealous ( and yes I am a jealous person and can't help that but he's given me reason in the past), not him spending more time out than he does in and having more conversations with some female that he knows has wanted him in the past than he does with his own wife!