Could I have been wrong in this situation?

I was in (not still sure if I still am) a relationship with this guy for 9 months. Last night we were both at a party with our friends and earlier he had told me that he was going to leave with me, I thought, Okay good, no problem, more time alone with him for me. But then we get into this silly argument because I was being distant. I'm not going to lie, I was, and partially because this girl (who he had once left me for to try "getting with her") was there and they were both talking. I don't have a problem with her, she's really nice, but I still get insecure around her. So to avoid jealousy, I kinda stuck around my friends, and he called this as "being distant". We got into a fight and ended up not leaving the party together, WHATEVER. Next morning, I got over it and went to his house (completely unexpected) to apologize. He accepts my apology and I feel better again, so we're talking a bit outside of his house when my ex boyfriend, who happens to be his best friend, walks out. Apparently he slept over, so I'm oh snap okay gotta get out of here fast. I go inside to go to the bathroom and BAM! the girl I'm totally insecure of is wearing my boyfriend's shirt and apparently she slept over too! So I hug her and then quickly walk to the bathroom, and there I am crying like an effing hyena. He walks in and we're in there for at least an hour and I think I broke up with him (not sure). He clearly sees that he did nothing wrong (and maybe he didn't) but the fact that she slept over his house (even if my ex was there too) when he knows about my insecurities towards her, wearing his shirt and then didn't tell me about it as if he were hoping I would never find out... That's what really pissed me off. So the moral of the question is, was it wrong of me to get upset over that?