The issue basically comes down to this question: Are fascists entitled to free speech?
As Timmy says: the answer is "yes". To deny them free speech would be... well... fascist.
Moreover, why should fascists be given free speech when they would, if given half a chance, deny free speech to others?
Because we are not fascists.