Even cats, who have very little sexual dimorphism, have the immediately identifiable trait that only females can be 3 colors.
There's NOTHING like that in humans. Even facial hair is not that obvious or reliable of an indication. Almost every trait is probabilistic.
We aren't, a solid 8 out of 10 gendered traits are social.