I think you can call someone saying “it’s unfair to judge people by race” a racist when they’re using that line to applaud the removal of protections against institutional racism. We can argue the merits of AA as a form of protection, but it was protection nonetheless. To say that it was unfair is to entirely ignore the unfairness which necessitated its existence.
I agree on the last point, but there isn’t a class based system in place, nor is there a plan to implement one (that I can find).
That, I shall continue to argue, makes this very not good.