If you do not want to be the other woman, don’t act like one. As harsh as that sounds, it’s really that simple.
My suggestion is that you tell this man not to call you again unless/until he is ready to make you The Woman in his life.
I recommend that you explain (to him) that you do not like being “the other woman,” any more than you like sneaking around, or being with someone who is cheating on someone else.
As much as it is going to hurt you — stop 100% all contact with him — right now. It would be far worse to wake up 2 years from now in the same situation, having wasted the best years of your life, waiting….
If he truly loves you and wants to be with you — your lack of contact will force him to make the changes he needs to make. If he doesn’t, he never would have.
On a very frank separate note, 9 times out of 10, someone who cheats with you will cheat on you. More, I would never recommend having an affair and breaking up a family, any more than I’d recommend believing in someone who is willing to breakup their family as a result of a 3 month affair…. Just something to think about.