Isn't there truth in your last statement though? It is a view expressed a lot on this forum - that tourists (by definition unlikely to be English) are an increasing proportion of those who attend.Whilst I dont think the match going spurs fans are racist there is a feeling of underlying uneasiness/tension/suspicion of anyone that does not look fully English.
People assume you have never been to a game before or know nothing about football or not a real fan. The amount of times Ive been asked if its my first time etc...
I think its got to do with fans feeling like they are being priced out and replaced by tourists willing to pay silly prices.
I think a lot of fans would object to that. Not on "racist" grounds, as they wouldn't object to what they regard as "real" fans attending based on their ethnicity, but because it makes the matchday experience less authentic and enjoyable.