Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Test Execution for Google Collab #441

Merged
merged 18 commits into from
May 21, 2024
Merged

ENH: Test Execution for Google Collab #441

merged 18 commits into from
May 21, 2024

Conversation

kp992
Copy link
Contributor

@kp992 kp992 commented May 8, 2024

This PR adds google collab compatibility testing for the intro lecture series

Related to QuantEcon/meta#139.

Copy link

netlify bot commented May 8, 2024

Deploy Preview for taupe-gaufre-c4e660 ready!

Name Link
🔨 Latest commit 0a5716b
🔍 Latest deploy log https://app.netlify.com/sites/taupe-gaufre-c4e660/deploys/664ab13889fd5c0008787cf7
😎 Deploy Preview https://deploy-preview-441--taupe-gaufre-c4e660.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link

github-actions bot commented May 8, 2024

@github-actions github-actions bot temporarily deployed to pull request May 8, 2024 09:43 Inactive
@kp992
Copy link
Contributor Author

kp992 commented May 9, 2024

@mmcky Seems like larger machine is not assigned to run the job.

.github/workflows/collab.yml Outdated Show resolved Hide resolved
enabled larger runner for ubuntu
@mmcky
Copy link
Contributor

mmcky commented May 9, 2024

thanks @kp992 I have enabled ubuntu-latest-m as a larger runner for this repo. It will cost some money (as they aren't free) but it will be interesting to see how much this ends up being.

@github-actions github-actions bot temporarily deployed to pull request May 9, 2024 04:23 Inactive
@github-actions github-actions bot temporarily deployed to pull request May 9, 2024 04:44 Inactive
@mmcky
Copy link
Contributor

mmcky commented May 9, 2024

Thanks @kp992 I have reconfigured QuantEcon github and we will trial this repo with the larger runner to see how expensive it will be.

@github-actions github-actions bot temporarily deployed to pull request May 9, 2024 05:06 Inactive
@github-actions github-actions bot temporarily deployed to pull request May 9, 2024 05:25 Inactive
@github-actions github-actions bot temporarily deployed to pull request May 9, 2024 05:36 Inactive
@github-actions github-actions bot temporarily deployed to pull request May 9, 2024 05:54 Inactive
@mmcky
Copy link
Contributor

mmcky commented May 9, 2024

This error is currently being reported

pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
google-colab 1.0.0 requires pandas==1.5.3, but you have pandas 2.1.4 which is incompatible.

however this error doesn't show up if I use google collab to test directly by adding the same code into the colab version of the lecture: https://colab.research.google.com/github/QuantEcon/lecture-python-intro.notebooks/blob/master/inflation_history.ipynb

Screenshot 2024-05-09 at 4 30 52 PM

@kp992 any ideas?

@kp992
Copy link
Contributor Author

kp992 commented May 9, 2024

What if we install the latest version of pandas globally instead of installing in specific lecture?

@github-actions github-actions bot temporarily deployed to pull request May 9, 2024 10:58 Inactive
@github-actions github-actions bot temporarily deployed to pull request May 9, 2024 11:00 Inactive
@kp992
Copy link
Contributor Author

kp992 commented May 9, 2024

@mmcky Woaah!! It's green.

Comment on lines 33 to 34
if Version(pd.__version__) < Version('2.1.4'):
from importlib import reload
if Version(pandas.__version__) < Version('2.1.4'):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be removed now right?

@kp992
Copy link
Contributor Author

kp992 commented May 13, 2024

Sure, I will check on this by tomorrow.

@kp992
Copy link
Contributor Author

kp992 commented May 14, 2024

@mmcky Actually, I am also facing some issues upon installing docker on my mac. Should we try it on our Linux VM that we have? I didn't want to play with it because John might be using it and maybe something could break.

Do you know how frequently they update the google collab environment?

There are frequent updates because of changing dependencies among the libraries.

@github-actions github-actions bot temporarily deployed to pull request May 14, 2024 15:58 Inactive
@github-actions github-actions bot temporarily deployed to pull request May 14, 2024 16:21 Inactive
@kp992
Copy link
Contributor Author

kp992 commented May 14, 2024

Hi @mmcky, I was able to pull the docker image and following the steps mentioned there, I tested locally and I get pandas 1.5.3 so our job is doing the right thing. Just that the public image is not upated with their private image that they are using online maybe.

@kp992
Copy link
Contributor Author

kp992 commented May 15, 2024

We can do one thing. Let's just do a manual testing for a list of failing lectures(currently just one lectures/inflation_history.md) and skip this from collab automated testing job. Once the image is updated we can remove the lectures from this. If you want I can write a python script to do this.

@github-actions github-actions bot temporarily deployed to pull request May 15, 2024 13:14 Inactive
@github-actions github-actions bot temporarily deployed to pull request May 15, 2024 13:16 Inactive
@kp992
Copy link
Contributor Author

kp992 commented May 15, 2024

@mmcky How does this look 63d8503? I checked the logs and it was failing during reload and not during pip installation. So the workaround is to find pandas version without actually importing it so we don't need reload at all.

@mmcky
Copy link
Contributor

mmcky commented May 17, 2024

@mmcky How does this look 63d8503? I checked the logs and it was failing during reload and not during pip installation. So the workaround is to find pandas version without actually importing it so we don't need reload at all.

@kp992 I don't fully understand why reload is working on Google Collab and not in this action. From my inspection of the logs the main issue is pandas==1.5.3 is specified by some google-colab package.

@kp992
Copy link
Contributor Author

kp992 commented May 17, 2024

From the logs:

�[?25hRequirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas==2.1.4) (1.16.0)
----- stdout -----
Installing collected packages: tzdata, pandas
  Attempting uninstall: pandas
    Found existing installation: pandas 1.5.3
----- stdout -----
    Uninstalling pandas-1.5.3:
      Successfully uninstalled pandas-1.5.3
----- stdout -----
�[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
google-colab 1.0.0 requires pandas==1.5.3, but you have pandas 2.1.4 which is incompatible.�[0m�[31m
�[0mSuccessfully installed pandas-2.1.4 tzdata-2024.1
------------------

This shows though there was some dependency error, and pip was successfully able to install pandas-2.14. Now if we use google-colab by importing it, we can expect it to fail. Since we don't use it in our code, we can ignore.

The actual error is coming from the below line which is reload:

[0;31mImportError�[0m                               Traceback (most recent call last)
�[0;32m<ipython-input-2-e9f1b8a0cfbd>�[0m in �[0;36m<cell line: 4>�[0;34m()�[0m
�[1;32m      4�[0m �[0;32mif�[0m �[0mVersion�[0m�[0;34m(�[0m�[0mpandas�[0m�[0;34m.�[0m�[0m__version__�[0m�[0;34m)�[0m �[0;34m<�[0m �[0mVersion�[0m�[0;34m(�[0m�[0;34m'2.1.4'�[0m�[0;34m)�[0m�[0;34m:�[0m�[0;34m�[0m�[0;34m�[0m�[0m
�[1;32m      5�[0m   �[0mget_ipython�[0m�[0;34m(�[0m�[0;34m)�[0m�[0;34m.�[0m�[0msystem�[0m�[0;34m(�[0m�[0;34m'pip install pandas==2.1.4'�[0m�[0;34m)�[0m�[0;34m�[0m�[0;34m�[0m�[0m
�[0;32m----> 6�[0;31m   �[0mreload�[0m�[0;34m(�[0m�[0mpandas�[0m�[0;34m)�[0m�[0;34m�[0m�[0;34m�[0m�[0m

I checked some other repos who are also facing this similar issue when doing reload fails. So to avoid all reload and importing again, I bypassed it by checking version of pandas without importing.

@mmcky
Copy link
Contributor

mmcky commented May 17, 2024

thanks @kp992 what I don't understand is why I can do it in Google Collab though.

@mmcky
Copy link
Contributor

mmcky commented May 17, 2024

@kp992 given the discrepancies between the docker image and the inconsistencies between what you can do in the live environment and in this docker container -- I am not confident this is a good approach. We may want to instead use pip freeze on the live collab environment and make our own ubuntu based docker container.

If there is a way to interact with colab to get pip freeze results programmatically that would be amazing as then we can do automatic docker updates.

@mmcky mmcky added the on-hold label May 17, 2024
@kp992
Copy link
Contributor Author

kp992 commented May 19, 2024

Hmm, strange thing, when I tried today pulling the docker image from colab and getting a pip list on our linux server, I get pandas as 2.0.3. I did the same thing 3-4 days back when I got 1.5.3.

@kp992
Copy link
Contributor Author

kp992 commented May 19, 2024

Triggering the job again to see if it might have been updated recently.

@kp992
Copy link
Contributor Author

kp992 commented May 19, 2024

Hey @mmcky, I was right. Previously it was pandas 1.5.3, but see the latest run, we are getting 2.0.3. So the docker image is now updated in the last 2-3 days to the latest image that we see colab website.

@github-actions github-actions bot temporarily deployed to pull request May 19, 2024 12:29 Inactive
@mmcky
Copy link
Contributor

mmcky commented May 20, 2024

good news - thanks for letting me know @kp992 so it looks like the docker public runtime releases lag the online version a bit but I think we can accept that. 👍

@github-actions github-actions bot temporarily deployed to pull request May 20, 2024 02:18 Inactive
@github-actions github-actions bot temporarily deployed to pull request May 20, 2024 02:19 Inactive
@kp992
Copy link
Contributor Author

kp992 commented May 20, 2024

Yeah, looks good to me know. thanks @mmcky

@mmcky mmcky removed the on-hold label May 21, 2024
@mmcky
Copy link
Contributor

mmcky commented May 21, 2024

thanks @kp992.

@jstac this is probably the best we can do for google collab testing. I am going to run this here for a couple weeks to see how costly it is in terms of compute and then roll it out more widely

@mmcky mmcky merged commit fc5cb52 into main May 21, 2024
7 checks passed
@mmcky mmcky deleted the collab_test branch May 21, 2024 00:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants