He touched my boobs while I was asleep. Should I let it go?

My bet guy friend is 15 and I am 14, we have been friends for 7 years, but that Is all we will ever be. But we are really close. He had me over at his house when his parents weren't home (I never thought anything of it because we do it all the time). But their air conditioning was broken, it was really hot. He wasn't wearing a shirt, and I was all like "That's not fair! You can be shirtless and I cant?!". Then he just said "Why not? Remember that time when I was 9 and I saw you naked? There is nothing I haven't already sean!". So I thought it was ok, and I took my shirt off and stayed in my bra. But he kept starring at my breasts in an awkward way, so I put my shirt back on because it was weird. We started watching a movie in his room, and I fell asleep. I was on his bed and he was laying down by me. I was like half asleep when I felt his hand squeeze my breast. So I got up and started yelling at him asking him why he touched me without my consent. He kept apologizing, and I grabbed my things and went home and told him never to talk to me again. But he keeps trying to talk to me, he says he is sorry and that when he did it he "Forgot it was me". We have been friends for so long, and I DID take my shirt off (but it was Innocent). I should have thought that through, and he does seam sorry. But Im not sure about this, he did touch me without m consent and it was wrong. But should I let it go?