I appreciate all advice and suggestions; so, please read my problematic situation and voice your opinion!

My situation:
I moved to a new area and started grad school. Four months before moving here, I met a fellow grad student—let’s call him Ryan. Ryan and I kept in touch; he even sent me hand written letters every week. Then I came to school. We went out to eat a couple times, out for drinks, watched movies, etc. I helped him out when he got sick. I even got to meet his family. And yes, we did “mess around.” At the beginning he was an above-and-beyond gentleman, which was a sign that he was too good to be true.

During all of this, he referred to me as his friend and later told me that he could not have a “girlfriend” until the end of the semester and recruiting season. (We go to a top 5 school and life is challenging without considering personal issues.) But he told me that he did not even kiss or want to kiss another girl on campus, even though he knew that I had done so with other guys. Still, when he said this, I knew he was not into me and thus I needed to back away from this misguided “friendship.” However, he kept treating me better than a friend and one night I did get really drunk and we ended up having sex. Since then, I think the only reason he calls me is because he is horny. He has called past 3 in the morning a couple times. Once he called in the middle of the afternoon asking if we should get together and make each other tired before taking a nap. Lastly, he told me that he had another friend from home (a girl). She has come to visit and stay over at his place. (I would never intervene with this relationship; however, if I was this girl, I would want to know what this guy does behind her back.)

I have told him that I although it would be fun to just have a “sex buddy” and sometimes I wish I could be a person that likes that, I am not and do not want to be. I have turned him down on his offers. His response—he keeps calling (and expecting things) or says “You cannot redraw a line that you drew once.” I am not going to fold into such outrageous behavior and comments. However, I do have to see this guy every day; our friends are mutual friends; and I do want to feel better about myself. It makes me sick that I allowed myself to get used like this, create these types of expectations, not get the response I wanted, and not be able to completely move on.

How can I change these things?