 Solution

$A$ is not an orthogonal matrix. Its first column has squared Euclidean normso $A^{T}A\ne I$. The reason this is compatible with ii is that $1,x,x^2$ is not an orthonormal basis for the given inner product. Its Gram matrix isThe correct condition on this matrix representation of a linear map iswhich follows either by direct multiplication or by the inner product preservation just proved. This is the rule for an orthogonal transformation in a nonorthonormal basis.