Use orthogonality to the image of a matrix. For , the condition is , since must vanish for every nonzero input. Direct multiplication givesEach coefficient must vanish; choosing the three coordinate basis vectors as inputs already proves necessity. Solve , to obtainThese values make every output have zero dot product with , including any zero output. Here orthogonality is understood in this algebraic sense; no direction is assigned to the zero vector.
A directional derivative measures change per unit distance, so first normalize the direction: . The gradient isTaking the dot product with the unit vector,Using the unnormalized vector would instead give , the change per unit parameter along ; it is not the rate per unit distance requested here.
Choose two perpendicular unit directions in the eigenspace:Their pairwise dot products vanish and their Euclidean norms are one. They form an orthonormal basis of eigenvectors, with eigenvalues , as guaranteed for a real symmetric matrix by the spectral theorem for real symmetric matrices.
Use outward normal vectors to the two faces. The cross productspoint away from the opposite vertices, since and . Their Euclidean norms are and , and their dot product is . The interior dihedral angle of a convex polyhedron is supplementary to the outward-normal angle, soThis also equals the cosine of the acute angle between the unoriented face planes. The outward-normal angle itself has cosine ; keeping that distinction avoids a sign ambiguity.
If either vector is zero, both sides vanish. Otherwise let be the angle between them. The magnitudes of the cross product and dot product are described bySquaring and using proves Lagrange's identity:
The two derivatives are and . At the intersection their slopes are and , so their direction vectors and have zero dot product. Thus the tangent lines are perpendicular and their angle is
The symmetry between the first and third coordinates makes an eigenvector, with eigenvalue . In the remaining invariant plane, write a vector as . Its image is , so the reduced eigenvalue equation has determinantThe corresponding directions have for and for . Normalizing givesTheir squared unnormalized norms are . The pairwise unnormalized dot products are , , and , so these unit eigenvectors are mutually orthogonal. The eigenvalues sum to and have product zero, agreeing with part (a).
Choose any nonzero auxiliary vector not collinear with . Then form a basis, so writeTaking the dot product with gives . This separates the two cases requested in the hint.
If , choose freely and solve for :If , the coefficient of is fixed and the other two are free:These formulas describe the whole plane with normal , including . Completeness follows from the basis expansion, not only from checking that the displayed vectors satisfy the equation. The parametrization of a plane with an auxiliary vector also gives a single formula valid in both cases.
A real square orthogonal matrix satisfiesequivalently . Its columns, and also its rows, form an orthonormal basis. Consequently it preserves dot products and Euclidean norms.
The gradient is , so its value at the point is . Since the specified direction is already a unit vector, the directional derivative is its dot product with that gradient:
The component of the cross product is . For the result to lie in the - plane, it must vanish, giving . The dot product condition is then , so
Set . Taking the dot product of the vector equation with givesHere and , so . Substitution then givesGeometrically the possible vector must lie on the line , and the scalar constraint picks one point. Equivalently the rank-one matrix update is invertible because . This is the nonsingular case of solvability of an affine rank-one vector equation.
The pairwise dot products of the normalized vectors areTheir squared norms are respectively , , and , all one. Hence they form an orthonormal basis. This also follows from the spectral theorem for real symmetric matrices, because is symmetric and its three eigenvalues are distinct.
The homogeneous equation is the dot product condition . A suitable vector field is thereforeWhere nonzero, it is tangent to the level curves of and perpendicular to its gradient. At the origin this vector vanishes, so no direction is defined there.
A row's squared Euclidean norm is , and the dot product of the two rows is . Thus the row Gram matrix isFor the column Gram matrix, the first and third squared column norms are , the middle one is , and the respective cross-products are . HenceThe two products are square matrices of different sizes; their unequal dimensions are consistent with being rectangular.
The matrix is symmetric under interchange of the first and third coordinates. For an antisymmetric vector , direct multiplication gives . For a symmetric vector , the linear transformation reduces toThe reduced characteristic polynomial is . At , its equations give ; at , they give . Normalizing the three eigenvectors givesTheir norms are one. Their unnormalized pairwise dot products are , and , proving mutual orthogonality. The eigenvalues sum to six and have product zero, agreeing with the matrix trace and determinant from part (a).
The three eigenvectors form an orthonormal basis, so an arbitrary vector has expansion . Multiplication by removes the zero-eigenvalue component:Its dot product with is zero for every . Thus one convenient answer is the nonzero vectorEquivalently, , so . Any nonzero scalar multiple of this vector in the null space also works.