Simple and flexible code for Reservoir Computing architectures like Echo State Networks (ESN).
from reservoirpy.nodes import Reservoir, Ridge, Input
data = Input(input_dim=1)
reservoir = Reservoir(100, lr=0.3, sr=1.1)
readout = Ridge(ridge=1e-6)
esn = data >> reservoir >> readout
forecast = esn.fit(X, y).run(timeseries)ReservoirPy is a simple user-friendly library based on Python scientific modules. It provides a flexible interface to implement efficient Reservoir Computing (RC) architectures with a particular focus on Echo State Networks (ESN). Advanced features of ReservoirPy allow to improve computation time efficiency on a simple laptop compared to basic Python implementation, with datasets of any size.
Some of its features are: offline and online training, parallel implementation, sparse matrix computation, fast spectral initialization, advanced learning rules (e.g. Intrinsic Plasticity) etc. It also makes possible to easily create complex architectures with multiple reservoirs (e.g. deep reservoirs), readouts, and complex feedback loops. Moreover, graphical tools are included to easily explore hyperparameters with the help of the hyperopt library. Finally, it includes several tutorials exploring exotic architectures and examples of scientific papers reproduction.
This library works for Python 3.8 and higher.
Follow @reservoirpy updates and new releases on Twitter.
See the official ReservoirPy's documentation to learn more about the main features of ReservoirPy, its API and the installation process. Or you can access directly the User Guide with tutorials.
pip install reservoirpy(See below for more advanced installation options)
Step 1: Load the dataset
ReservoirPy comes with some handy data generator able to create synthetic timeseries for well-known tasks such as Mackey-Glass timeseries forecasting.
from reservoirpy.datasets import mackey_glass
X = mackey_glass(n_timesteps=2000)Step 2: Create an Echo State Network...
...or any kind of model you wish to use to solve your task. In this simple use case, we will try out Echo State Networks (ESNs), one of the most minimal architecture of Reservoir Computing machines.
An ESN is made of a reservoir, a random recurrent network used to encode our inputs in a high-dimensional (non-linear) space, and a readout, a simple feed-forward layer of neurons in charge with reading-out the desired output from the activations of the reservoir.
from reservoirpy.nodes import Reservoir, Ridge
reservoir = Reservoir(units=100, lr=0.3, sr=1.25)
readout = Ridge(output_dim=1, ridge=1e-5)We here obtain a reservoir with 100 neurons, a spectral radius of 1.25 and a leak rate of 0.3 (you can learn more about these hyperparameters going through the tutorial Understand and optimize hyperparameters). Here, our readout layer is just a single unit, that we will receive connections from (all units of) the reservoir. Note that only the readout layer connections are trained. This is one of the cornerstone of all Reservoir Computing techniques. In our case, we will train these connections using linear regression, with a regularization coefficient of 10-5.
Now, let's connect everything using the >> operator.
esn = reservoir >> readoutThat's it! Next step: fit the readout weights to perform the task we want. We will train the ESN to make one-step-ahead forecasts of our timeseries.
Step 3: Fit and run the ESN
We train our ESN on the first 500 timesteps of the timeseries, with 100 steps used to warm up the reservoir states.
esn.fit(X[:500], X[1:501], warmup=100)Our ESN is now trained and ready to use. Let's run it on the remainder of the timeseries:
predictions = esn.run(X[501:-1])As a shortcut, both operations can be performed in just one line!
predictions = esn.fit(X[:500], X[1:501]).run(X[501:-1])Let's now evaluate its performances.
Step 4: Evaluate the ESN
from reservoirpy.observables import rmse, rsquare
print("RMSE:", rmse(X[502:], predictions), "R^2 score:", rsquare(X[502:], predictions))Run and analyse this simple file (in the "tutorials/Simple Examples with Mackey-Glass" folder) to see a complete example of timeseries prediction with ESNs:
-
simple_example_MackeyGlass.py (using the ESN class)
python simple_example_MackeyGlass.py
If you have some issues testing some examples, have a look at the extended packages requirements in readthedocs.
To install it, use one of the following command:
pip install reservoirpyor
pip install reservoirpy==0.3.5If you want to run the Python Notebooks of the tutorials folder, install the packages in requirements file (warning: this may downgrade the version of hyperopt installed):
pip install -r tutorials/requirements.txtIf you want to use the previous version 0.2.4, you can install ReservoirPy using:
pip install reservoirpy==0.2.4If you want to enable the hyper package and its hyperparameter optimization helpers using
hyperopt, use:
pip install reservoirpy[hyper]Go to the tutorial folder for tutorials in Jupyter Notebooks.
Go to the examples folder for examples and papers with codes, also in Jupyter Notebooks.
Tutorial for ReservoirPy (v0.2) can be found in this Paper (Trouvain et al. 2020).
A quick tutorial on how to explore hyperparameters with ReservoirPy and Hyperopt can be found in this paper (Trouvain et al. 2020).
Take a look at our advices and our method to explore hyperparameters for reservoirs in our recent paper: (Hinaut et al 2021) HTML HAL
Turorial and Jupyter Notebook for hyper-parameter exploration
More info on hyperopt: Official website
If you want your paper to appear here, please contact us (see contact link below).
- Chaix-Eichel et al. (2022) From implicit learning to explicit representations. arXiv preprint arXiv:2204.02484. arXiv PDF
- Trouvain & Hinaut (2021) Canary Song Decoder: Transduction and Implicit Segmentation with ESNs and LTSMs. ICANN 2021 HTML HAL PDF
- Pagliarini et al. (2021) Canary Vocal Sensorimotor Model with RNN Decoder and Low-dimensional GAN Generator. ICDL 2021. HTML
- Pagliarini et al. (2021) What does the Canary Say? Low-Dimensional GAN Applied to Birdsong. HAL preprint. HAL PDF
- Which Hype for My New Task? Hints and Random Search for Echo State Networks Hyperparameters. ICANN 2021 HTML HAL PDF
If you have a question regarding the library, please open an Issue. If you have more general question or feedback you can contact us on twitter or by email to xavier dot hinaut the-famous-home-symbol inria dot fr.
Trouvain, N., Pedrelli, L., Dinh, T. T., Hinaut, X. (2020) Reservoirpy: an efficient and user-friendly library to design echo state networks. In International Conference on Artificial Neural Networks (pp. 494-505). Springer, Cham. HTML HAL PDF
If you're using ReservoirPy in your work, please cite our package using the following bibtex entry:
@incollection{Trouvain2020,
doi = {10.1007/978-3-030-61616-8_40},
url = {https://doi.org/10.1007/978-3-030-61616-8_40},
year = {2020},
publisher = {Springer International Publishing},
pages = {494--505},
author = {Nathan Trouvain and Luca Pedrelli and Thanh Trung Dinh and Xavier Hinaut},
title = {{ReservoirPy}: An Efficient and User-Friendly Library to Design Echo State Networks},
booktitle = {Artificial Neural Networks and Machine Learning {\textendash} {ICANN} 2020}
}
This package is developped and supported by Inria at Bordeaux, France in Mnemosyne group. Inria is a French Research Institute in Digital Sciences (Computer Science, Mathematics, Robotics, ...).