The SCOTUS interpreted free speech in Hustler vs Fallwell as protecting even the loathsome, as you said.
However, the authors of the Constitution never once said free speech is without consequence. They left it to the states to determine whether any laws were broken regarding hateful speech. And also, the court of public opinion.
Federalist Papers.