Skip to content

Nets module

Jump to bottom
hcruiz edited this page Oct 12, 2018 · 7 revisions

In this page, we describe the Nets module containing following scripts for the neural network (NN):

  1. DataHandler: load and pre-process data for training/using the NN
  2. staNNet: a class with the basic structure to define, train and load a standard NN
  3. predNNet: a child of staNNet with extra methods to train the control voltages by gradient descent (GD)

Notes:

1.This page must be seen as work in progress. If something is unclear or you have comments/feedback, please send an email to me ([email protected]). 2. The Nets module is based on PyTorch. If you do not have experience with this package but want to learn more about the inner workings of Nets, please go to this tutorial. However, unless you want to work under the hood, you will not need to deal with PyTorch.

DataHandler

DataHandler contains functions to pre-process and load the data needed for the neural net training:

  1. DataLoader(data_dir, file_name, **kwargs)
  2. GetData(dir_file, syst = 'cuda')
  3. PrepData(main_dir, list_dirs, threshold = 1000)

More information about these functions can be found in their docstrings. Here we give a more general description.

DataLoader

The DataLoader loads and defines data structures suitable for use with the NN. It returns two objects,
data, noise = [(x_train,y_train),(x_val,y_val)], baseline_var
The first is a list of two tuples, each containing training and validation sets with inputs x and targets y. The x's and y's are torch variables that must be passed to the NN to define it (see below). The second output baseline_var is a (very!) rough estimate of the noise in the system by computing the variance of the device output conditioned on the inputs and averaging over all of them. It is meant as a guiding measure that is probably underestimating the noise.

NOTE: there is a kwarg test_set = False that controls the partition of the data. If True, then 10% of the data (default) will be partitioned for a test set and will be saved as test_set_from_trainbatch.npz. You can select the fraction of the data being saved for testing. By default, the inputs of the task are also saved separately as X_inputs.npy. All files are saved by default in the data_dir.

GetData

If you want to load data that you can use with the NN, for instance if you already trained it and want to asses its generalization performance, then GetData is the function you need. It is basically a wraper to convert numpy arrays into torch variables with the correct data type for the NN. The function assumes that the data loaded is a npz file with keys 'inputs' and 'outputs'.

PrepData

This is the first function you want to apply to a newly acquired data batch. The measurements must be in an training_NN_data.npz file with key 'data'. If the second argument is a list of directories with data, it goes through them and combines all sets into a single one. The first dimension being the number of samples, the second the number of electrodes for the first indices and the rest must be a time-series for that given voltage configuration. A mean and variance over these last indices is calculated. The data is cropped with a threshold (default value at 1000!) and scaled to volts or the desired units. The control voltages for the remaining data is shifted and scaled to lie in [0,1]. Remember that the first indices represent the inputs defined by the task. We need to separate between inputs and control voltages for two reasons. First, for compatibility with the Genetic Algorithm (GA) and second because of the curse of dimensionality we cannot sample the electrodes defined as inputs, so their values must be for know task dependent. PrepData does not return any value but instead it saves the processed data to a directory called data4nn under the subdirectory created using the data at runtime. The name of the data file is 'data_for_training'. The format is a npz file with keys: inputs, cv_trafo, outputs, var_output, list_dirs. Please refer to the docstring for further info.

staNNet

This is the base class containing the NN. When you initialize it with a data set, it constructs a network with the number of inputs given by the data. If you pass a string with the path to a saved NN as argument, it loads the NN. With the train_nn method you can train the NN and with the save_model method you can save it to a given path. The train_nn method must get learning_rate, nr_epochs as arguments. Finally, with the output method you evaluate the inputs, i.e. you call a forward pass for every sample in your inputs array.

predNNet

The predNNet is a child of staNNet so it has the same methods as above. In addition, with the method predict you can estimate the CVs needed for a given functionality. They are obtained by gradient descent on the CVs, which are not regarded as inputs to the NN anymore, but rather as biases of a layer on top of the input layer in the trained NN. Hence, this mode of the NN only accepts the inputs of the task for which you want to train the device. This all happens under the hood, so you only need to provide to the predict method the arguments data, learning_rate, nr_epochs, batch_size. The data has the same structure as if you would train the NN, see above. The predict method returns the control voltages and the validation error.

Clone this wiki locally