ML-in-cpp

This is my toy cpp project implementing machine learning such as forward and backward propagation.
It contains input, target batch files to train AND, OR and XOR operations.

Installation

1. Download or clone it

git clone https://github.com/kcoms555/ML-in-cpp

2. Go to the ML-in-cpp directory

cd ML-in-cpp

3. compile source files

make  

or

g++ -o bin/runner source/runner.cpp  
g++ -o bin/trainer source/trainer.cpp  

Example

Train XOR data set

(https://en.wikipedia.org/wiki/Exclusive_or)
1. Write a input batch for XOR
Open 'data/XOR.input' and write them

2 1 4 <-- row, column, count. It means there are four (row) by (column) matrices 
0 0 <-- It will create a 2 by 1 matrix ((0), (0)). It always takes columns first.
0 1
1 0
1 1 

2. Write a target batch for XOR
Open 'data/XOR.target' and write them

1 1 4
0
1
1
0

3. Set configuration file
Open 'bin/conf' and set input as ../data/XOR.input and set target as ../data/XOR.input.
Set weights, biases, activation functions, learning rate and repetition as you want.

input = ../data/XOR.input
target = ../data/XOR.target
w = 1 2 7  <- sets 2 by 7 weight matrix in first layer
w = 2 7 1
b = 1 7 1 <- sets 7 by 1 bias matrix in first layer
b = 2 1 1
func = 1 tanh <- sets activate function of first layer as 'Hyperbolic tangent'
func = 2 x3
lr = 0.01 <- sets learning rate as 0.01
repeat = 10000 <- sets training repetition
ls = 2 <- sets the layer size. In case of single layer, set its value as 1.
print_csv = false
show_cost = true
load = true
save = true

The neural network will be built as shown in the picture below.

4. train and run
Go to ./bin and execute trainer and runner

Performance benchmarking

• Tensorflow 1.11.0 with Raspberry Pi 3 B+, running the sample code functionally identical to the XOR learning above : 46.753s
• ML-in-cpp with Raspberry Pi 3 B+, running the XOR learning above : 6.040s

A configuration file of ML-in-cpp and the python(tensorflow) test code are in the test/

Because of random initialization, they are not printing same costs. If you want to see that they print the same costs, initialize the weights with same values. You can do it by editting MATRIX::RAND in source/IO.cpp to the specified value and random_normal() in test/XOR.py to fill() with the specified value you want.

ML-in-cpp is much faster than tensorflow 1.11.0 in that case by 7.74 times