MultiStepBootstrappingInRL

Here, we compare Q(\sigma) learning presented by Sutton and Barto in [1] to Tree-Backup, n-step Expected Sarsa, and n-step Sarsa.

The main notebook file with our analysis is: VarianceAnalysisMultistepBootstrapping.ipynb

All experiments are included here that generated the graphs in the notebook.

Authors:

Peter Henderson Wei-Di Chang

Name		Name	Last commit message	Last commit date
Latest commit History 45 Commits
images		images
.gitignore		.gitignore
README.md		README.md
VarianceAnalysisMultistepBootstrapping.ipynb		VarianceAnalysisMultistepBootstrapping.ipynb
average_reward.png		average_reward.png
double_expected_sarsa.py		double_expected_sarsa.py
double_sarsa.py		double_sarsa.py
expected_sarsa.py		expected_sarsa.py
mdp_matrix.py		mdp_matrix.py
n_step_expected_sarsa.py		n_step_expected_sarsa.py
n_step_sarsa.py		n_step_sarsa.py
n_step_tree_backup.py		n_step_tree_backup.py
q_sigma.py		q_sigma.py
q_sigma_with_varianced_sigma.py		q_sigma_with_varianced_sigma.py
q_variance.png		q_variance.png
sarsa.py		sarsa.py
test_n_step.py		test_n_step.py
variance_reward.png		variance_reward.png
variance_vs_n_steps_experiment_deterministic_gridworld.py		variance_vs_n_steps_experiment_deterministic_gridworld.py
variance_vs_n_steps_experiment_stochastic_gridworld.py		variance_vs_n_steps_experiment_stochastic_gridworld.py
varying_alpha_epsilon_experiment.py		varying_alpha_epsilon_experiment.py
varying_alpha_epsilon_experiment_deterministic_gridworld.py		varying_alpha_epsilon_experiment_deterministic_gridworld.py
varying_alpha_epsilon_experiment_stochastic_gridworld.py		varying_alpha_epsilon_experiment_stochastic_gridworld.py
varying_alpha_with_custom_sigma.py		varying_alpha_with_custom_sigma.py

Provide feedback