Sir Bobby Charlton came to the back of the coach looking puzzled. It was 2000 and we were working together as part of the unsuccessful bid to help England win the 2006 World Cup.
"Sepp Blatter just told me that he played football against me several times," said Sir Bobby, one of the greatest footballers in the world. "I have no recollection of that at all. And from what I've been told, he didn't even play football."
My first impression of Blatter was therefore not that positive; he appeared to have foolishly invented a story to make himself look big. Nothing has ever changed my opinion.
His comment on Wednesday that there is no racism in football infuriated me. There is.
The situation is much better than it was when I was a kid when it was persistent, nasty and overt, but racism has not gone away - as the recent examples in the Premier League demonstrate.
What annoyed me even more was his assertion that a racially abused player should shake hands with the player who has abused him at the end of the game.
Shake hands? The only part of my hand I would be using would be my fist in the tunnel after the match.
I don't know one black player who could comprehend what Blatter is trying to say. I personally find it very offensive. Does he really think that it's OK for a player to accept abuse because of the colour of his skin and then almost laugh it off? Just put it down to the heat of the moment?
Does he want black players to say: "I know you didn't mean to call me that name, but don't worry about it - you can call me that name again next time we play."?
Maybe he wants players subjected to racist chants to go and applaud those "fans" at the end of the games too.
Blatter is clueless and his comments undermine all the good work that the game's authorities have done to try to eradicate racism.
The man is a dinosaur. He's 75, an age when most people are in retirement. His opinions were formed in a bygone era and his attitude is reflected in the tiny fines which Fifa gives to countries whose fans have been found guilty of being racist during a game. It's as if Fifa don't think it's an important issue when it is, almost like they want it brushed under the carpet.
Blatter's comments have shown him up for what he is, an out-of-touch buffoon who is not fit to lead the most important football organisation in the world. The man is a disgrace.
As the leader of Fifa, he should have a responsible voice, not inflame situations. Rather than backtrack and try to worm himself out of comments he made not once but twice, he should resign from his position. To say them once might be an accident; repeating these sentiments shows that this is actually what he believes.
Racism on the field is a complex and serious issue. It's different from a few idiots with no brains in the stands having a go.
No player wants to receive racist abuse and no player wants to be labelled as a racist either. So any allegations are usually denied straight away, just as Luis Suarez denied racially abusing Patrice Evra before he was charged with the offence by the English Football Association this week.
I was playing for Manchester United at Highbury in 1997 when it kicked off between Peter Schmeichel, the United goalkeeper, and Ian Wright, the Arsenal forward, as the players left the pitch. It was an ugly end to an angry game between the best two teams in England which had seen Wright jump two-footed at Schmeichel.
I thought that was that, as we began to calm down in the dressing room, but then allegations from Wright appeared in the media than Schmeichel had racially abused him.
I thought long and hard about that one. I don't know of any black player who would produce what some offensively call "the race card". That is complaining about racial abuse for sympathy or to damn another individual. It's just not the done thing.
But I knew Schmeichel very well. I liked him and trusted him. He was not a racist in my eyes and I had never heard anything from him to suggest that he would make a racist comment. I'll never know the truth of that one and since no evidence was forthcoming, we have to assume that Schmeichel was innocent.
That was then and this is now. Like many people, I don't trust or like Blatter or his cronies in Fifa. He should do the honourable thing and resign. Then he can go back to dreaming about a top-class football career he never had.
Andy Cole's column is written with the assistance of European football correspondent Andy Mitten