My prejudice against prejudicial people? It's amazing how many people think that is a good argument. Typically prejudiced people.

It a common enough movie-cliche too. I.e., gambler just lost the car saying, "Don't judge me, don't judge me!"