Not every guy you meet is going to see you the way you want him to see you. (And you won’t see every guy you meet as “the one” — even when they feel that you are!). That’s just life. And although you’ve had some good times and some intimacies with this guy, the incompatibilities that you see as surmountable, he doesn’t. He sounds like he’s not ready for the relationship you want with him, even though he does have feelings for you.
It’s always hard to consider the fact that just because you love someone, and sometimes they love you back at the same time — but that that isn’t enough to make things work. You can have different goals for a relationship, for lifestyles, or even just incompatibilities that keep you apart. It sounds like there is enough here to keep you in the game, but not enough to win it. So if you’re looking for a long-term, committed relationship, I think you’ll agree that this isn’t it — based on your past together.