> I have never heard another trans person complain about me not fitting into specific gender roles

That is a straw-man of my position. A large part of the trans community believes hormone blockers should be liberally administered to teenagers and children, for example. That is by definition an attempt to steer healthy biological development into pre-established expectations of body conformity. The entire "transwomen are women" mantra is a vehement call for binarity: transwomen can't simply be transwomen, i.e. free people living their lives how they see fit, they must be externally recognized and validated as this social defined thing called a woman, which they were born into.

So while, of course, a community member would never comment or object to another's degree of conformity, popular variants of trangederism exhibit a strong conceptual agreement that gender objectively exists, as opposed to being an entirely made up thing used to pigeonhole people into social roles according to their biology.

This is in perfect opposition to traditional feminism, which seeks to dismantle oppressive gendered institutions. And in fact, in perfect opposition with the liberal tradition that pursues individual sovereignty and de-marginalization; just imagine using the same discourse of "being born in the wrong body" and "getting the treatment they need" for any other social class that is discriminated against for minor biological variance, such as skin color, height or weight.