Now let's try and use the trained ONNX file for inference on some manually drawn images on GIMP:
Note that the images must be drawn with white on black. If you use black on white, it the accuracy becomes terrible. This is a good very example of brittleness in AI systems!
Figure 1. Number 9 drawn with mouse on GIMP by Ciro Santilli (2023)
We can try the code adapted from thenewstack.io/tutorial-using-a-pre-trained-onnx-model-for-inferencing/ at python/onnx_cheat/infer_mnist.py:
cd python/onnx_cheat
./infer_mnist.py lenet.onnx infer_mnist_9.png
and it works pretty well! The protram outputs:
9
as desired.
We can also try with images directly from Extract MNIST images.
for f in /home/ciro/git/mnist_png/out/testing/1/*.png; do echo $f; infer.py $f ; done
and the accuracy is great as expected.