4 years in and i'm not "wife material"

My boyfriend 30, and I, 25 have been dating for a little over 4 years. We've been thru a lot together trying to make things work between the two of us. There was (i guess you can call it) infidelity at the start because he wasn't clear on his intentions. He never asked me to be his girlfriend and half the time I felt like he was avoiding me, so I was still talking to other people, which he felt betrayed over. I realized how much that affected him, though and I corrected my behavior after voicing our thoughts. But it seems like that's what the entire relationship revolved around for so long. Me doing everything in my power to convince him that I'm loyal, so much so, that I don't even feel like myself anymore. Im actually positive the stress of it killed my libido, because we havent had much sex in 2 years. I only leave the house for work, he knows all my whereabouts, who I'm talking to, everything. My breaking point was when he would hover behind me reading my video game chat. I had a conversation with him about it. Yeah, people can say some pretty weird things to women playing videogames. But thats where I come into control, that I dont care if they call me "cute" or whatever. Because I'm not entertaining it and thats not what i play the game for. And if things stay uncomfortable and we cant communicate with eachother, than its not going to work, which he agreed with. But The entire time, he was looking at me with the most DISGUSTED face. I honestly couldnt believe my eyes, no one has ever looked at me like that. Then, he tells me "see, thats just not WIFE MATERIAL." honestly, it left me speechless. He's a great guy. He's helped me mature and learn how to do things for myself. We live together and I respect him and our home and take care of myself physically and financially, so hearing him say that has made me almost 100% turned off to the relationship. All I can see now, is how much it isn't meant to be rather than wanting to make things work. Am i wrong for this?