Advice needed

I'm going to sound like I'm such a mug for this!

My partner of three years was acting strange, and secretive last year, you know the stuff, constantly on his phone but moving so I can't see etc. He left his emails open on his laptop one day, so I did a bit of snooping, and found he'd basically signed up for every free sex/dating website going. I confronted him, he said he was curious, it was like porn for him but he'd not done anything.

I kept a tab on him, and over the next few weeks did more snooping (I know I'm wrong for this), and found he'd been messaging women on Facebook, and sending and receiving nudes on WhatsApp. I told him I knew, and that I was not putting up with it, last chance blah blah blah. I sent him a message one night when working a night shift, saying I wanted to work on our relationship, and he said the expected 'he's so sorry what he'd done'. I (bad again I know) had taken an old phone of his, that was logged in to messenger still, and the same time he's sending me this he's sexting some nasty little side bit.

Fast forward to now, he's admitted to sleeping with this nasty side bit, a year ago, just before he moved in with me. He's in fairness not put a foot wrong since. But my heads all over? I know I should have split up with him, but I feel heartbroken at the thought of losing him? We have a lovely life together, but I just have days where I feel so unhappy and tell myself to get rid of him?