Please follow and like us:

The first time you learn to train Neural Network, you try to train a Model for Image recognition using MNIST format.

And later, you want to use your images to train your model, and how can you do it?

With Python script, it’s very easy!

Step 1: Create folder tree:

Step 2: Create an rezise script, rescaling all of the png’s you placed in the folders the MNIST standard 28×28 pixel size:


#!/bin/bash
#simple script for resizing images in all class directories
#also reformats everything from whatever to png
if [ `ls test-images/*/*.jpg 2> /dev/null | wc -l ` -gt 0 ]; then
echo hi
for file in test-images/*/*.jpg; do
convert "$file" -resize 28x28\! "${file%.*}.png"
file "$file" #uncomment for testing
rm "$file"
done
fi
if [ `ls test-images/*/*.png 2> /dev/null | wc -l ` -gt 0 ]; then
echo hi
for file in test-images/*/*.png; do
convert "$file" -resize 28x28\! "${file%.*}.png"
file "$file" #uncomment for testing
done
fi
if [ `ls training-images/*/*.jpg 2> /dev/null | wc -l ` -gt 0 ]; then
echo hi
for file in training-images/*/*.jpg; do
convert "$file" -resize 28x28\! "${file%.*}.png"
file "$file" #uncomment for testing
rm "$file"
done
fi
if [ `ls training-images/*/*.png 2> /dev/null | wc -l ` -gt 0 ]; then
echo hi
for file in training-images/*/*.png; do
convert "$file" -resize 28x28\! "${file%.*}.png"
file "$file" #uncomment for testing
done
fi

Step 4: Create python script to convert:


import os
from PIL import Image
from array import *
from random import shuffle

# Load from and save to
Names = [['./training-images','train'], ['./test-images','test']]

for name in Names:
	
	data_image = array('B')
	data_label = array('B')

	FileList = []
	for dirname in os.listdir(name[0])[1:]: # [1:] Excludes .DS_Store from Mac OS
		path = os.path.join(name[0],dirname)
		for filename in os.listdir(path):
			if filename.endswith(".png"):
				FileList.append(os.path.join(name[0],dirname,filename))

	shuffle(FileList) # Usefull for further segmenting the validation set

	for filename in FileList:

		label = int(filename.split('/')[2])

		Im = Image.open(filename)

		pixel = Im.load()

		width, height = Im.size

		for x in range(0,width):
			for y in range(0,height):
				data_image.append(pixel[y,x])

		data_label.append(label) # labels start (one unsigned byte each)

	hexval = "{0:#0{1}x}".format(len(FileList),6) # number of files in HEX

	# header for label array

	header = array('B')
	header.extend([0,0,8,1,0,0])
	header.append(int('0x'+hexval[2:][:2],16))
	header.append(int('0x'+hexval[2:][2:],16))
	
	data_label = header + data_label

	# additional header for images array
	
	if max([width,height]) <= 256:
		header.extend([0,0,0,width,0,0,0,height])
	else:
		raise ValueError('Image exceeds maximum size: 256x256 pixels');

	header[3] = 3 # Changing MSB for image data (0x00000803)
	
	data_image = header + data_image

	output_file = open(name[1]+'-images-idx3-ubyte', 'wb')
	data_image.tofile(output_file)
	output_file.close()

	output_file = open(name[1]+'-labels-idx1-ubyte', 'wb')
	data_label.tofile(output_file)
	output_file.close()

# gzip resulting files

for name in Names:
	os.system('gzip '+name[1]+'-images-idx3-ubyte')
os.system('gzip '+name[1]+'-labels-idx1-ubyte')

 
Step 5: Running script with command:

python convert-images-to-mnist-format.py

thank for Gregory S. Kielian

Please follow and like us:

Leave a comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.