You've hit on one of the most interesting and difficult aspects of self-defence, which is the degree to which the personal characteristics of the accused can be attributed to the reasonable person when we're looking at the reasonableness of the accused's actions.
Many of the accused's personal characteristics can be attributed to the reasonable person, but there's a limit. If we attribute every characteristic of our accused to the reasonable person, we've lost the benefit of the reasonable person test because we've made the reasonable person our accused.
The courts deal with where that line is on a continual basis. Certainly if a person had experienced violent attacks throughout their childhood, or if they had been victimized many times and that circumstance affected the way they approached situations or the way they perceived situations, to a degree those would be factors that could be attributed to the reasonable person so that we could better judge that person's actions and perceptions and be fair to them.