Am I wrong

I've been married for about six years during that time I've been the one who has supported us the most financially. I've always had a steady job ever since I could work up until recently (my place of employment fired me during the pandemic) when we first got married he worked a regular job but about a year into our marriage he quit and started doing odd jobs just enough to cover his half of rent and to pay for whatever he is collecting at the time. Then the major responsibility of the finances got put on me (90% of the groceries, a pack of cigarettes a day each, going out, sometimes even his half of bills, ect). Over the last five years of this happening I have voiced how I'm not alright with how things are going. He will get better about it for a month or so and then go right back to it. He has spent money of mine that I've set aside time and time again without so much as asking me. Sometimes he won't even tell me after he does it. Just waits until I find out on my own. When I get upset about it he immediately tells me I'm wrong that since we are married what's mine is his and his is mine. We have never had a joint bank account because he is so bad about spending. It's gotten so bad that I can't even leave money in my bank account without him spending it. When I try to talk to him about it some how he ends up making me feel bad about not wanting to keep supporting us both.