GraphAttack is a simple computational graph library for machine learning in pure Python (you can think of simple tensorflow). The main point is to combine mathematical operation together to form a workflow of choice in the form of a graph. The graph then takes care of pushing the data forward and propagating the gradients backwards.
I have aimed for the library to be simple and transparent so that it would be easy to understand and modify to fit individual needs. The performance was not the primary objective as there are plenty of fast alternatives; the aim was smaller, educational models.
All feedback and ideas for improvement welcome.
- To be found in tutorial.ipynb, covers basic usage and a simple linear regression model
Setting up and running / training a dense neural network model
Setting up and running / training a Convolution neural network model
Setting up and running / training a Recurrent neural network
Matrix operations
- Dot product
- Hadamard product
- element-wise division
- addition
Regularization
- Dropout
- Batch Normalisation
Activations
- ReLU
- Sigmoid
- Tanh
- Softmax
Convolution
- 2d convolution
- Max Pooling
Cost operations
- Quadratic cost
- Cross-entropy for softmax activation
Optimisation
- Adam SGD minimizer, support for batches and continuous series
Misc
- Reshape / flatten
- Slice
- Sum all elements/axis
- Sum all elements squared
- element-wise exponent
Limited GPU support can be made avaliable by modifying the file: graphAttack/gaUtilities/graphAttackFunctions.py
Commenting out lines 1-2 and uncommenting lines 4-22 will send the dot product computations to your GPU
- pyCUDA for GPU computations
- scikit-cuda for easy access to cuBLAS
http://www.deeplearningbook.org/ section 6.5.1 for more information on computational graphs and the rest of the book for more details about ML/deep learning.
- Python 3.5 or above
- numpy - linear algebra for Python
- scipy - Scientific Python library, here used for utilities
MIT