There seems to be a lot of "I hate..." so I'm going to focus specifically on the "When and why, exactly..?"
1972 (I think). My father had died young of a sudden heart attack the year before. Mum had taken me to London where we were staying with my sister. She's promised to take me to Spurs but as it turned out they weren't at home so she asked me if I wanted to go to Chelsea instead. I was a wide-eyed 11-year old boy from rural Norfolk, so although it wasn't the team I supported, I was happy to go to a big match in the capital.
I had a white and navy blue bar scarf - it had been my sister's school scarf but was obviously perfect as a Spurs scarf. I asked mum if I I should take it to the Chelsea game. Neither of us knew anything about these things but she reckoned that as Chelsea wore blue it would be OK.
We had seats in the recently-opened "posh" stand, As we sat down all the old geezers around us - blokes in their 50s and 60s as I remember - started hurling abuse at me and mum for wearing a Spurs scarf, really disgusting stuff for a young boy and his mum to be hearing. She tried to stick up for us but told me to take the scarf off and she put it in her handbag.
Half way through the first half, the referee keeled over away from the play. Cue great merriment, which only increased when a stretcher was brought on. "He's dead, he's dead" they cheered. How they all laughed. I was stunned, god knows what mum was going through, strangely we never ever mentioned it, either after the match or later in life. She must have been destroyed but she kept it together admirably.
Hated them with a special passion ever since.