Use orthogonality to the image of a matrix. For , the condition is , since must vanish for every nonzero input. Direct multiplication gives
Each coefficient must vanish; choosing the three coordinate basis vectors as inputs already proves necessity. Solve , to obtain
These values make every output have zero dot product with , including any zero output. Here orthogonality is understood in this algebraic sense; no direction is assigned to the zero vector.
A directional derivative measures change per unit distance, so first normalize the direction: . The gradient is
Taking the dot product with the unit vector,
Using the unnormalized vector would instead give , the change per unit parameter along ; it is not the rate per unit distance requested here.
Choose two perpendicular unit directions in the eigenspace:
Their pairwise dot products vanish and their Euclidean norms are one. They form an orthonormal basis of eigenvectors, with eigenvalues , as guaranteed for a real symmetric matrix by the spectral theorem for real symmetric matrices.
Use outward normal vectors to the two faces. The cross products
point away from the opposite vertices, since and . Their Euclidean norms are and , and their dot product is . The interior dihedral angle of a convex polyhedron is supplementary to the outward-normal angle, so
This also equals the cosine of the acute angle between the unoriented face planes. The outward-normal angle itself has cosine ; keeping that distinction avoids a sign ambiguity.
If either vector is zero, both sides vanish. Otherwise let be the angle between them. The magnitudes of the cross product and dot product are described by
Squaring and using proves Lagrange's identity:
The two derivatives are and . At the intersection their slopes are and , so their direction vectors and have zero dot product. Thus the tangent lines are perpendicular and their angle is
The two dot products are and . Therefore
The symmetry between the first and third coordinates makes an eigenvector, with eigenvalue . In the remaining invariant plane, write a vector as . Its image is , so the reduced eigenvalue equation has determinant
The corresponding directions have for and for . Normalizing gives
Their squared unnormalized norms are . The pairwise unnormalized dot products are , , and , so these unit eigenvectors are mutually orthogonal. The eigenvalues sum to and have product zero, agreeing with part (a).
Choose any nonzero auxiliary vector not collinear with . Then form a basis, so write
Taking the dot product with gives . This separates the two cases requested in the hint.
If , choose freely and solve for :
If , the coefficient of is fixed and the other two are free:
These formulas describe the whole plane with normal , including . Completeness follows from the basis expansion, not only from checking that the displayed vectors satisfy the equation. The parametrization of a plane with an auxiliary vector also gives a single formula valid in both cases.
The gradient is , so its value at the point is . Since the specified direction is already a unit vector, the directional derivative is its dot product with that gradient:
The component of the cross product is . For the result to lie in the - plane, it must vanish, giving . The dot product condition is then , so
Set . Taking the dot product of the vector equation with gives
Here and , so . Substitution then gives
Geometrically the possible vector must lie on the line , and the scalar constraint picks one point. Equivalently the rank-one matrix update is invertible because . This is the nonsingular case of solvability of an affine rank-one vector equation.
The pairwise dot products of the normalized vectors are
Their squared norms are respectively , , and , all one. Hence they form an orthonormal basis. This also follows from the spectral theorem for real symmetric matrices, because is symmetric and its three eigenvalues are distinct.
The homogeneous equation is the dot product condition . A suitable vector field is therefore
Where nonzero, it is tangent to the level curves of and perpendicular to its gradient. At the origin this vector vanishes, so no direction is defined there.
A row's squared Euclidean norm is , and the dot product of the two rows is . Thus the row Gram matrix is
For the column Gram matrix, the first and third squared column norms are , the middle one is , and the respective cross-products are . Hence
The two products are square matrices of different sizes; their unequal dimensions are consistent with being rectangular.
The matrix is symmetric under interchange of the first and third coordinates. For an antisymmetric vector , direct multiplication gives . For a symmetric vector , the linear transformation reduces to
The reduced characteristic polynomial is . At , its equations give ; at , they give . Normalizing the three eigenvectors gives
Their norms are one. Their unnormalized pairwise dot products are , and , proving mutual orthogonality. The eigenvalues sum to six and have product zero, agreeing with the matrix trace and determinant from part (a).
The three eigenvectors form an orthonormal basis, so an arbitrary vector has expansion . Multiplication by removes the zero-eigenvalue component:
Its dot product with is zero for every . Thus one convenient answer is the nonzero vector
Equivalently, , so . Any nonzero scalar multiple of this vector in the null space also works.