To deal with the stark reality of having hit or hurt a woman or child to deal with the initial responsibility you have not to do that and the knowledge you did do it can be incredibly hard.
I've had fans come and knock on my door. I'm usually polite but I'm usually very direct and say 'It's not cool that you come here uninvited.'