AI Pytorch MNIST model

Mon 15 April 2024 AI ai / pytorch / mnist

Overview

• installation: see https://arrizza.com/setup-common

Summary

This project creates an AI model from MNIST sample data. This data is set of images with human drawn digits from 0 to 9. The AI model then has to classify each one correctly. This project uses two types of models: Linear and Convolution.

Kudos for this project to my son Thomas. He's studying AI in his Master's and understands Linear vs Convolution models and the remaining aspects needed to make these models work correctly. (I just played with the python code :)

To run

You must first train the model (defaults to "mnist")

./doit --train
# takes 1m 15s to run

Note that training also invokes a run of the model:

running 500 images
<snip>
accuracy:  97.2% passed total: 500 errors:15

Shows that 500 images were tested with 15 errors, an accuracy of 97%.

Each error is reported. For example, image #2 was predicted/classified as 4 but was actually 2.

ERR    0] image:   2 False pred: 4 act: 2
ERR    1] image:  31 False pred: 8 act: 3
ERR    2] image:  53 False pred: 3 act: 5
ERR    3] image:  58 False pred: 8 act: 2
ERR    4] image:  73 False pred: 5 act: 8
ERR    5] image: 125 False pred: 3 act: 2
ERR    6] image: 192 False pred: 8 act: 1
ERR    7] image: 246 False pred: 0 act: 2
ERR    8] image: 249 False pred: 5 act: 8
ERR    9] image: 280 False pred: 6 act: 4
ERR   10] image: 352 False pred: 9 act: 4
ERR   11] image: 373 False pred: 6 act: 1
ERR   12] image: 428 False pred: 7 act: 3
ERR   13] image: 446 False pred: 8 act: 9
ERR   14] image: 448 False pred: 3 act: 8

These errors (up to 20) are displayed in a dialog box, see https://arrizza.com/img/ai-mnist-model-failures.png

To just run the linear model

# defaults to 500 images
./doit

You can explicitly run the MNIST linear model or the Convolution model:

# default is linear MNIST model
./doit --train # train the model
./doit         # load the local model in model_mnist.pt and run sample data

./doit --model mnist --train # train the linear model
./doit --model mnist         # load the local model in model_mnist.pt and run sample data

./doit --model convo --train # train the convolution model
./doit --model convo         # load the local model in model_convo.pt and run sample data

You and adjust the number of images tested:

# do max number of images
./doit --model mnist --num 10000
./doit --model convo --num 10000

If you prefer, or for timing purposes, you can prevent the showing of the error images:

# don't show the error images
./doit --model convo --num 100 --no-show

You can run your own images. The bad.png file contains a sample. The image must match the format used in the MNIST images:

• .png format
• 28 x 28 pixels
• black and white

# do a local png file
./doit --model convo bad.png

Interesting ones to try:

• your own handwriting for digits 0 - 9
• all black pixels
• all white pixels
• an "X" i.e. lines but not a digit

MNIST model

The MNIST model is linear and has good accuracy. Running all 10000 images shows 96.5% accuracy:

./doit --num 10000
<skip>
accuracy:  96.5% passed total:10000 errors:355

Typically, the errors have extra tails or swirls, or a gap in digit's lines that confuse the model.
Some of the failures are difficult even for a human (me) to figure out.

Convo model

The convo model takes longer to train and to run, but is more accurate.

accuracy:  99.0% passed total:10000 errors:97