Am I hopelessly broken hearted?

When I was 17 years old, I met a guy named..let's call him Jay. Jay and I connected instantly. Within three months we got our first apartment together and were practically inseperable. I ended up going away for college and cried the entire trip and first week I was there. I ended up driving three hours one way EVERY weekend just to be near him for a day. But we were young, and I was insecure and thought he was cheating on me so we broke up. After a month of avoiding him calling and calling and calling (I was away at school) the calls suddenly stopped. It scared me to death, so I called him to see if he was okay or had gotten hurt. He was fine physically, but so depressed that he was moving across country (Literally east coast to west coast) to live with his dad. It felt like someone had stabbed me. It took me less than two heart beats to decide that I was going with him to wherever he was. So I packed my bags and we moved. Life was great. We were always smiling and laughing, we did so many amazing things...adventures I guess I consider them...and then, when I was 21 I found out I was pregnant. Everything went down hill from there as my attention turned away from him onto our child that I was carrying. And he became abusive. I tolerated the unusual negative behavior until our son was 4 months old, and I finally couldn't take it anymore, so i decided I was moving back to the east coast. There was never a discussion, if I went, he was going to go. So we moved, and life settled down. He actually really began to enjoy being a parent and I thought the worst was behind us...until we got pregnant AGAIN, less than 6 months after having our first son. The pattern of abusive behavior began again, and reckless behavior on his part...but not until I started to show that I was carrying a child. By the time our second son was two months old I called the police and had him arrested for one of the many abusive inciddents. And I knew that I had made the right choice. I was a single mother of two infants for almost a year, when I met my current husband. He had 6 children from a previous marriage so he enjoyed the babies. It took almost too much time and energy to blend our families, or even to form a deeply rooted bond between the two of us (even though he is an amazing man) and I just wrote it off to being so hurt by Jay. It's been over three years since I've seen Jay's face, heard his voice, even asked about him, but when I close my eyes at night it's his face that I dream about. I know that I should be scared of him, but honestly, after the anger wore off...the sense of abandonment, there's just been an enormous feeling of loss. When I allow myself to think about him it feels like someone is stabbing me right in the center of my stomach and I almost have to bend over from the pain. I just don't know where I go from here..I've moved on. I'm married to a wonderful, respectable man who brings out the best in me and loves and accepts my children. But I can't help feeling like I've lost everything...Am I absolutly hopeless? Are these feelings due to the abuse in the relationship? Or did we really have something more that we just weren't ready for?