Relationship advice please help?

I been in a relationship for about 2 years, and although we had a great time together most of the time there was really bad arguments. Physically, from what I know, none of us cheated on each other. But he had an ex who lived in another state he'd frequently contact, phone calls, emails, he sent her pictures too. Of course when I found out, I went ballistic and he always apologized. We did have good times too. I went on vacations with this boy... we literally spent almost every day together that it felt as if we were married at one point. His family adores me, his cousins, brothers, sisters. Everything... so I was like essentially part of the family. Anyway, he broke up with me, and while we were separated for a few months this boy started talking to me. He was a sweetheart, he had goals, a job, motivation, but when my ex found out he immediately wanted me back. So I go back out with this guy... the guy I'm mentioning, he doesn't work doesn't have any goals, no education.. he's 3 years older than me and it's my now my first year in college. I know love is suppose to be blind and you aren't suppose to judge someone based on edu or work, but I'm concerned with my future too you know.... I broke up with him recently and this is the worst **** I have done all year. I feel miserable. I miss him so much but I don't know if I should go back. When I think about it.. we argued too much, he hit before on two occasions.. I don't know. I hate being indecisive