Bringing up the past

Please give me some advice. 2013 me and my guy friend let a friend of his stay at our place for a while. But me and my guy friend aren't in a relationship. However, one day I walk around in my t-shirt and underwear in front of the guy. Nothing happen he didn't look at me or try to do anything funny. My guy friend feels some type of way about it but never brings it up until 2017. 4 years later with accusations of me messing with the guy. He says I snuck ouf of the bedroom some nights and I remember none of this. Now here it is 2021 and he is still saying something happened between us on 2013. The guy has been in jail for 4-5 years and he never asked him. I know I never did anything and I just don't know what to do at this point. Do you think he is right?