Yes, you should end your relationship and accept that you loved the wrong person. If you go back to him, you will always be reminded of how he cheated on you, and it will likely lead to frequent arguments between you two.
It’s better to cut ties now, especially since he has already shown that he is pulling away and not making any effort. Don’t waste your time on the wrong person and just wait, someone right for you will also come along, someone who truly deserves you.