Have to see my cheating ex everyday and it's wearing me down

My ex and I broke up back in January, long long story, but basically he wanted to be polyamorous after 3 years together and just getting a house together. He was very upset when I said no and was at a girl's place within 2 hours of us taking a break before we even discussed boundaries, and admitted to having emotionally cheated on me for at least a month.

It was quite possibly the worst couple weeks of my life, my mental health plummeted I couldn't eat I couldn't sleep I couldn't go a day without a breakdown. When I finally got my new place I started feeling better, and started feeling even better once I finally got all my stuff moved out of the house. But I also was completely miserable at my job at the time, and had been wanting to leave for awhile. My ex works in the same industry at a location that I hadn't worked yet, I applied thinking I could handle it, and have been there for 2 months.

Honestly it is infinitely better, I'm a lot happier there than I was at my last job but I'm still struggling with dealing with my ex. I figured we would just ignore eachother but he surprised me by actually talking to me. He asked me for a ride home one night and I told him no, we've had conversations about how our lives are going and I've been so conflicted about how to feel. He's still so easy to talk to and I miss having someone to be close to, and I cant help remembering we had and lost. But this past week a bunch of coworkers went out together, and I hadn't known my ex was invited so when I saw him I was unprepared and I just left and completely broke down.

Our mutual friends are more his friends than mine, I won't be able to hang out with coworkers without him being there most likely. I'm losing the first large friend group I have and I feel alone. I want to stop hurting and to move on but I have no idea how. I dont know how to act around him, I don't want to be friends but he seems to want to bury the hatchet. He doesn't seem to have any ill feelings towards me, but has apparently not taken any responsibility for the breakup which only makes me feel worse.