I'm not sure if I want this anymore

Hi, I'm 21 years old cis-female and I'm currently in my first ever relationship/situationship

I have no prior experience to dating or anything that it may include

So I meet this guy fairly recently (few months back) trough mutual friends and he seemed nice enough, and tho we don't have much in common we have enough that I liked him well enough

But I certainly don't love him. Nor am I sure if I like like him enough or just like him

We ended up going on few dates, and soon enough we ended up kissing

We were in my place when it happened and it was my first kiss, and while it was nice it's not anything special

And I'm not looking for The One or something like that, nor am I "saving" myself for marriage but I don't feel like I want s3x, or at least not now

And it has less to do with him and more with my self image, but in short I'm not comfortable with it

But he might want it

And maybe I gave him the impression that I would be open to it or something I'm not sure and now I don't know how to back down

When we were kissing he told me he wants, but we were short on time before he had to leave so I basically told him next time

As in SOME OTHER TIME

But he might have taken it seriously

And now he mentioned hanging out at his place, U don't have good enough reson to refuse but I'm anxious what might happen

I just don't know what to do

I'm not sure I even want to be with him, I'm not in a great mental headspce rn but I feel like it wouldn't be fair to him to just drop it, to break up, or to even tell him no

What do I do? I don't want to go to his place, but I don't want to tell him no outright

Help please