You handled yourself very well. And while you may feel uncomfortable, you’re doing the right thing by back away from the situation. If there is a way to stay friends with your buddy, by seeing him away from his girlfriend, that would be great. And you definitely shouldn’t see her alone, or even in a group, if you beg out gracefully. Eventually, this problem will blow over, but it’s not your problem to own. It’s theirs.
And while you’re backing away, you should also branch out and make other friends besides them. If you have more social and emotional support, you won’t feel trapped because this one guy friend is your sole support.