Am I being used by a guy and am I just letting him because I like him?

Ok so I had a huge crush on my best friend's brother for a long time. We all moved in together and things kind of went a little past a crush on both sides and we started having fun together. We became inseparable, both in the bedroom and out. But he never asked me out. All his friends say its obvious he likes me so they encourage me to ask him out. When I do he say's he's not attracted to me (but we were sleeping together for a long time) at all. He said it was weird because we lived together. So we cooled it for a while, but I am so attracted to him that I can't stay away from him. We always give each other back rubs and fall asleep on each other and end up kind of crushing on each other again. One thing leads to another and we end up sleeping together again. He is so loving when we do, he's very attentive and he always makes sure I am completely satisfied. On top of that all he wants to do is cuddle and kiss me. Sometimes that's all he wants to do. He is terribly hot to top things off and I can't resist him. Usually I initiate things, but recently he has been starting things and wanting me more than I want him. Then when I start bringing it up and talking about my feelings for him, he turns cold. So i've learned to keep my mouth shut because that's the only way I keep getting his affection. What do I do? I think I love this guy even though he doesn't love me back. I know he likes me alot because even though he hasn't made me his exclusive girlfrend or anything, he wont date anyone and he comes home to me every night. WHAT the heck is going on?