Nope, saying "men deserve to be recognized as women except in certain instances" is no different from saying "white people deserve to be recognized as black people except in certain instances."
No one else has the right to choose which demographics they are in. It's not "equal."
There can be no “moderation” on trans rights — either they have equal rights or they don’t! To say “they deserve to live according to their gender identity except for certain instances” is like saying “black people deserve to do everything white people do except eat at certain restaurants or date white women.”