Struggling with Self-worth

I'm 36 and single, and I've never been in a relationship or even dated much. I've had an isolated life, mostly from either not having a job or working the night shift, but I also have social anxiety. I'm trying to come to terms with the fact that I'll never really matter to anyone unless there's a sexual relationship.

I've always known I wouldn't be as important to someone as much as their spouse or partner, but I didn't think it extended to less serious relationships. My otherwise close friends stop caring about my feelings as soon as they get an opportunity for sex. I thought that was just something younger people did. It feels like everything I reassured myself for the past 20 years about not needing to be in a relationship for validation has been a lie. I want to start dating, but I can't feel confident knowing my value as a person is at stake.