Sure it is.

Maybe what you meant to say was "self-esteem is not solely dependent upon confidence in your appearance", because confidence in one's appearance can (and usually does) STRONGLY influence his or her self-esteem. That being said, this is only one factor that comprises self-esteem, along with many others, including how you "value yourself as a person on the inside."