Solution (source code)

= Solution

Close the cone with the horizontal disk $D$ at height $h$, and let $V$ be the enclosed solid. The <divergence> is
$$
\nabla\cdot\mathbf G=3x^2+2y+3y^2+1=3r^2+2y+1.
$$
The term $2y$ integrates to zero by reflection symmetry of each circular cross-section. Using the <divergence theorem> and <cylindrical coordinates>, the closed-surface <flux> is
$$
\int_V\nabla\cdot\mathbf G\,dV
=\int_0^h\int_0^{2\pi}\int_0^{\sqrt2z}(3r^2+1)r\,dr\,d\varphi\,dz
=\int_0^h(6\pi z^4+2\pi z^2)\,dz
=\frac{6\pi h^5}{5}+\frac{2\pi h^3}{3}.
$$
The outward normal on $D$ is $+\widehat{\mathbf z}$, so its <flux> is its area times $G_z=h$:
$$
\int_D\mathbf G\cdot\mathbf n\,dS=h\,\pi(\sqrt2h)^2=2\pi h^3.
$$
Subtracting this cap contribution gives the required lateral <flux>:
$$
\boxed{\int_A\mathbf G\cdot d\mathbf A=\frac{6\pi h^5}{5}-\frac{4\pi h^3}{3}.}
$$
This is <conical flux by closing with a cap>. The normal convention matters: on the lateral cone the outward normal points radially outwards and partly downwards, so the $G_z$ contribution to the lateral <flux> is negative. To justify applying the theorem at the vertex, truncate a small tip and take its size to zero; the extra <flux> vanishes because $\mathbf G$ is regular there.