On general principle, Private schooling of the type that you get from a school like Harrow or Eaton is harsh and kind at the same time. It can be both vastly superior to state funded schooling in some areas and lacking in some other areas.
As to the away from home aspect, think back to your own childhood. How often do you remember day to day experiences with you parents? You remember friends or specific activities, special occasions with your parents not the day to day things. Those things can happen still when you come home on the weekend or on holidays.
We can say what if until the cows come home but the personality that people grew to like was formed from that Harrow education. So you can disparage it if you like but you should thank it for the man that Benedict has become.
Three words: Boarding School Syndrome. Yes it affected him, he is an empath which made him even more vulnerable to bullying etc. as a youth. You can’t get away from that as readily when you are boarding. On the otherhand his academic formation was more robust than it would have been at a private school.