Should I break up with him?

I've dated my boyfriend for the 4 months as of yesterday. We were best friends for over a year before we dated. All summer when I was in Italy he'd send me an email about how he missed me and what he was doing back home. He asked me out officially on September 20th. He spelled it out in tennis balls (I'm training to go pro in tennis for when I'm older) and he tried writing it in my yard but my grandpa chased me away so he put the tennis balls into the holes of the tennis court fence at school after my practice. He was so charming and sweet up until now.. he seems like he's stopped trying. Before we were dating he was constantly trying to win my affection, but now that he has it he thinks he can't lose it. I'm the only girl who's celebrated christmas with him and his family and the only girl who's met his dad. He's never told me he loves me.. I've always just assumed it by his actions. I started loving him about 2 months into being his official girlfriend. Now, 2 months later I want to love him but a part of me is coonfused. We almost never see eachother. My dad is crazy strict and freaks out when he finds out about times when we were alone together. But I feel like we're not going anywhere with this relationship. We haven't kissed in a month because we never see eachother. I only see him 3 times a day in the halls: once after 2nd hour, once after 5th hour, and then at the end of the day when he walks me out to my older sister's car. I know I'll miss him the second after I break up with him if I do break up with him because I will have screwed up everything we have. I want to tell him that I want him to act the way that he used to but what if we decide to break up while we're talking about that? I don't want this to end but is it just time?