Automorphism (${\mathrm{Aut}}_{\mathcal{C}}(X)$)

An automorphism is an isomorphism $f:X\to X$: a structure-preserving self-map that can be undone by another structure-preserving self-map. Equivalently, it is an invertible endomorphism. Its inverse satisfieswhere $1_X$ is the identity morphism. Both maps must be morphisms in the chosen category. For ordinary groups, rings, and vector spaces over a fixed field, a bijective morphism automatically has a compatible inverse. For topological spaces, an automorphism is a homeomorphism from the space to itself: a continuous bijection needs a continuous inverse as well. See Riehl, Category Theory in Context, Definition 1.1.10 and Example 1.1.11.

For example, $T(x,y)=(x+y,y)$ is an automorphism of the vector space ${\mathbb{R}}^2$ over the real numbers, with inverse $T^{-1}(u,v)=(u-v,v)$. Both are linear maps. The linear endomorphism $P(x,y)=(x,0)$ from endomorphism preserves addition and scalar multiplication too, but discards a coordinate and cannot be undone. Preserving the structure and preserving enough information for reversal are separate requirements.

The automorphisms of a fixed object in a locally small category form a group under composition in a category: the identity morphism is the identity element, and the inverse element of a composite reverses the order of its factors. This makes reversible internal changes calculable. For a group, the resulting automorphism group records which relabelings preserve all products; the examples in group automorphism show how restrictive that condition can be.

A group automorphism is a bijective group homomorphism $\alpha :G\to G$. It preserves the group operation, the identity element, and inverse elements; its inverse is another group homomorphism. Thus it is a group isomorphism with identical source and target. These maps form the automorphism group $\mathrm{Aut}(G)$ under function composition.^[ref]

For the additive finite cyclic group $C_6$, take $f_5(k)=5k(\mathrm{mod}6)$. Its output list on $0,1,2,3,4,5$ is $0,5,4,3,2,1$, a permutation of all six elements. It preserves addition because $5(a+b)=5a+5b(\mathrm{mod}6)$, and applying it twice gives $25k=k(\mathrm{mod}6)$. Hence it is a group automorphism and its own inverse function. Compare the collapsing $f_2$ in the figure under endomorphism. The general criterion under endomorphism of a finite cyclic group says exactly when such a map is reversible: its multiplier must be coprime to the size of the finite cyclic group.

A permutation need not be a group automorphism, even if it fixes the identity element. Let $\pi$ exchange $1$ and $2$ in $C_6$ and fix $0,3,4,5$. It is a bijection, and $\pi (0)=0$, butThus it fails to preserve addition and is not even a group homomorphism. Reversibility alone does not preserve the group operation.

A group automorphism carries each cyclic subgroup onto the cyclic subgroup generated by the image of its defining element, preserving the order of a group element. In particular, it carries a generator of a group to another generator of a group. For $C_6$, the existing calculation of generators of a finite cyclic group leaves only the images $1$ and $5$ for $1$, so its automorphism group consists exactly of the identity function and $f_5$. This turns the problem of compatible relabeling into a much smaller choice than enumerating arbitrary permutations.

A subgroup is characteristic when every automorphism of the ambient group maps it to itself. A uniquely determined subgroup of a given order or structural type is characteristic.

A nontrivial group with no proper nontrivial characteristic subgroup. A minimal nontrivial normal subgroup of a finite group has this property: every characteristic subgroup of it is normal in the ambient group.