How did you get around to having sex? Did you have a date at least? Or did you just get right into it?
I hate to say it, but I think he maybe was just being polite about the lunch. I’d just continue to be cool/playful/flirty with him, but don’t ask him about lunch and don’t ask to see him again. If he’s interested, he would’ve shown up or at least let you know out of respect that he wasn’t going to be able to make it. Something *could* have happened that prevented him from being there, but otherwise, assume he’s not interested. If he is, he’ll come to your apartment asking for something when he’s ready. In the meantime, move on. 😉