I've heard this sentiment from a few African friends and online too, it's a real frustration for some immigrants. As a Nigerian who's lived in the US, I get why it stings when someone who shares your skin color makes you feel "less than" over how you talk, especially when you've been speaking English your whole life (often more formally than everyday AAVE).
But honestly, it's not always "mostly" African Americans doing this. From what I've seen and read, accent bias hits from all sides, some white folks slow-talk or act confused on purpose, while others genuinely try to understand. The difference might be that when it's another Black person judging your accent, it hurts deeper because you expect solidarity, not gatekeeping English "correctness."
A lot of it comes from different histories: AAVE evolved as its own valid dialect under centuries of oppression, so some Black Americans feel protective when they hear "proper" British-influenced African English and assume it's putting on airs or looking down. Meanwhile, many Africans arrive thinking standard English = prestige, not realizing the cultural weight here.
End of the day, language is just a tool, nobody should feel inferior for their accent. We all got enough to deal with without turning on each other.