[BUG] Smoke test failure because of run_cell module in papermill

[yueguoguo]

Description

Running of smoke test failed because run_cell in nbconvert has been added with a new argument that breaks papermill. Discussion from other developers of this issue can be also found here.

The problem can be fixed if upgrading papermill to version >1.0. However, this will lead to another issue where the method record is not available in papermill (it is deprecated).

In which platform does it happen?

All platforms where papermill 0.18 is used.

How do we replicate the issue?

Run the smoke testing by

python -m pytest tests/smoke/test_notebooks_python.py -k test_sar_single_node_smoke

The following error will be generated

Expected behavior (i.e. solution)

• The easiest way to resolve the issue is to fix the versions of nbconvert and papermill to the old one (nbconvert 5.5.0 and papermill 0.18.2) with which we passed the tests
• Alternative may be using scrapbook for recording and reading the data in executing the notebooks.

Other Comments

[yueguoguo]

@miguelgfierro @gramhagen do we want to use scrapbook instead in the long-term run - it is used in NLP repo. Currently, using the fixed-versions mitigate the problem so that the testing can be passed

[miguelgfierro]

+1 to using scrapbook

[miguelgfierro]

related to #641

[yueguoguo]

Fixed version of nbconvert and papermill to resolve the issue.

Reopen the issue when scrapbook is being developed.