Depends what you mean by being hard being with the ones you love.

If your on about staying in a relationship, then I think (and Im not speeking from experience here) is because for the relationship to work properly you have to let your guard down, which makes you vulnerable. So the slighest thing can have a big impact on you.
And with your guard down you are more prone to being hurt and then break up.

__________________