Your ex-boyfriend is having trouble being broken up from you. He’s really confused, and he’s trying to process what it feels like to be with you versus what it feels like to not be with you. This is his internal process, so it’s not anything you did or didn’t do, and it’s not anything you can change. This is his situation to deal with.
That said, you get to choose whether or not you want to be part of a relationship with him — whether it’s friends or lovers — considering he’s currently all over the map, emotionally. If you’re looking for a reunion — or a quick reunion — you’re no doubt going to be frustrated. And you are! So if you can readjust your expectations and enjoy being with him without looking forward to being together as boyfriend or girlfriend again, or being okay if you don’t end up being boyfriend and girlfriend again, you’re going to alleviate that frustration you’re feeling now.
If you can’t change your expectations and when you’re with him all you want is to be boyfriend and girlfriend again, you may want to reconsider spending so much time with him until he straightens out his feelings and is clear on what he wants from you.