 Solution

Uniform area density in the unit disk is $1/\pi$. The polar coordinates Jacobian gives the joint densityThus the coordinates are independent and the uniform random point in a disk has

For independent points $A,B$, the area of triangle $OAB$ isNowIndependence therefore givesConditioned on $A,B$, the probability that $C$ lies inside $OAB$ is its area divided by the disk area $\pi$. Hence

Four points in general position fail to form a convex quadrilateral exactly when one lies inside the triangle formed by the other three. Each of $A,B,C$ lies inside the triangle formed by $O$ and the other two with probability $4/(9{\pi }^2)$, and these three events are disjoint. The remaining possibility is that $O$ lies inside $ABC$. By origin in a triangle of three radial random points, this has probability $1/4$. Therefore