Remove obsolete package versions #950
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Cleaned up older versions for two packages: urllib3 - deleted stale version folders python - deleted tests folder
MukuFlash03
pushed a commit
to MukuFlash03/e-mission-server
that referenced
this pull request
Jan 13, 2024
commit 54659fb Merge: cf0c9e2 1159eac Author: K. Shankari <[email protected]> Date: Thu Dec 21 20:17:15 2023 -0800 Merge pull request e-mission#951 from MukuFlash03/fix-vuln Bulk deletion of site-package/tests commit 1159eac Author: Mahadik, Mukul Chandrakant <[email protected]> Date: Thu Dec 21 20:43:39 2023 -0700 Bulk deletion of site-package/tests Added a one line pipe command to remove all occurrences of site-packages/tests occurring in miniconda main directory. commit cf0c9e2 Merge: d2f38bc 3be2757 Author: K. Shankari <[email protected]> Date: Thu Dec 21 17:47:27 2023 -0800 Merge pull request e-mission#950 from MukuFlash03/fix-vuln Remove obsolete package versions commit 3be2757 Author: Mahadik, Mukul Chandrakant <[email protected]> Date: Thu Dec 21 18:05:23 2023 -0700 Remove obsolete package versions Cleaned up older versions for two packages: urllib3 - deleted stale version folders python - deleted tests folder commit d2f38bc Merge: 978a719 c1b0889 Author: K. Shankari <[email protected]> Date: Wed Dec 20 14:31:09 2023 -0800 Merge pull request e-mission#949 from MukuFlash03/fix-vuln Fixing latest Docker image vulnerabilities commit c1b0889 Author: Mahadik, Mukul Chandrakant <[email protected]> Date: Mon Dec 18 11:04:25 2023 -0700 Upgraded Ubuntu base image Latest Ubuntu base image was just released officially by Docker which contains updated version of libc6 and libc-bin. commit 07747d0 Author: Mahadik, Mukul Chandrakant <[email protected]> Date: Fri Dec 15 18:38:12 2023 -0700 Fixing latest Docker image vulnerabilities AWS Inspector found the following vulnerable packages: CRITICAL perl HIGH nghttp2, libnghttp2-14 cryptography, libssl3 cryptography libc6, libc-bin Upgraded perl, libssl3, nghttp2 packages by upgrading base Ubuntu image to latest of the same LTS version - jammy (22.04). Cryptography package was fixed by mentioning required version to be installed using conda. Libc6, Libc-bin can be fixed by using apt-get upgrade but this upgrades all packages which is not recommended as a blanket upgrade fix.
shankari
added a commit
that referenced
this pull request
Feb 10, 2024
* Added details in logging statements Completed marked future fixes for PR-944 Provided more detailed info regarding number of distinct users, as well as number of predictions made for the trip_list. * Implemented logic to clear models Clearing or trimming number of models for a particular user from the Stage_updateable_models collection. Constant K_MODELS_COUNT defines the number of models to keep. Currently tested with model data available in stage dataset snapshot. Pending - Complete testing with assertions to be added for testing. * Added function call Called the trim_model_entries function in upsert_model to delete models before inserting new ones. * Squashed commit of the following: commit 54659fb Merge: cf0c9e2 1159eac Author: K. Shankari <[email protected]> Date: Thu Dec 21 20:17:15 2023 -0800 Merge pull request #951 from MukuFlash03/fix-vuln Bulk deletion of site-package/tests commit 1159eac Author: Mahadik, Mukul Chandrakant <[email protected]> Date: Thu Dec 21 20:43:39 2023 -0700 Bulk deletion of site-package/tests Added a one line pipe command to remove all occurrences of site-packages/tests occurring in miniconda main directory. commit cf0c9e2 Merge: d2f38bc 3be2757 Author: K. Shankari <[email protected]> Date: Thu Dec 21 17:47:27 2023 -0800 Merge pull request #950 from MukuFlash03/fix-vuln Remove obsolete package versions commit 3be2757 Author: Mahadik, Mukul Chandrakant <[email protected]> Date: Thu Dec 21 18:05:23 2023 -0700 Remove obsolete package versions Cleaned up older versions for two packages: urllib3 - deleted stale version folders python - deleted tests folder commit d2f38bc Merge: 978a719 c1b0889 Author: K. Shankari <[email protected]> Date: Wed Dec 20 14:31:09 2023 -0800 Merge pull request #949 from MukuFlash03/fix-vuln Fixing latest Docker image vulnerabilities commit c1b0889 Author: Mahadik, Mukul Chandrakant <[email protected]> Date: Mon Dec 18 11:04:25 2023 -0700 Upgraded Ubuntu base image Latest Ubuntu base image was just released officially by Docker which contains updated version of libc6 and libc-bin. commit 07747d0 Author: Mahadik, Mukul Chandrakant <[email protected]> Date: Fri Dec 15 18:38:12 2023 -0700 Fixing latest Docker image vulnerabilities AWS Inspector found the following vulnerable packages: CRITICAL perl HIGH nghttp2, libnghttp2-14 cryptography, libssl3 cryptography libc6, libc-bin Upgraded perl, libssl3, nghttp2 packages by upgrading base Ubuntu image to latest of the same LTS version - jammy (22.04). Cryptography package was fixed by mentioning required version to be installed using conda. Libc6, Libc-bin can be fixed by using apt-get upgrade but this upgrades all packages which is not recommended as a blanket upgrade fix. * Updated Test file with dummy models Was using pre-built models for testing that were present in stage_snapshot dataset but these won't be available in the normal dataset. Hence, took reference from emission.tests.modellingTests.TestRunGreedyModel.py for creating test models using dummy trip data. Creating multiple such models and then checking for trimming operation success. * Reverted commit 31626ba + Removed log statements Commit 31626ba included changes related to vulnerability fixes, hence reverted them. Had added some log statements related to scalability fixes for model loading. Removed them and will add in separate PR. * Changed imports to access K_MODEL_COUNT Removed redundant "esda" import added possibly during local testing. Removed get_model_limit() as K_MODEL_COUNT is a class variable and can be accessed directly using class name. * Revert "Squashed commit of the following:" This reverts commit 31626ba. Earlier commit f54e85f missed out the discarded reverted changes relating to the vulnerability fixes. Hence removing them in these. * Included maximum_stored_model_limit parameter in trip_model config file Decided upon threshold value for model count above which redundant models will be deleted. This was agreed upon to be 3 and is defined in the trip_model.config.json.sample file. Necessary changes have been made in the related files to use this config value. * Addressed latest review comments - Improved logging statements to included whether model count has changed. - Skipped check for saved test data in Test file as we clear teardown() related collections in the DB. * Restored inferrers.py Did this to revert a whitespace change which was possibly due to removing a line at the end of the file. We don't want it to be modified as it is unrelated to this set of PR changes. Restored using this command by pointing to specific commit hash for the first instance where the file was modified. git restore --source=79ad1cc~1 emission/analysis/classification/inference/labels/inferrers.py * Updated Test for Storing limited models Added test to ensure that only the latest models are stored by comparing the write_ts times. * Revert whitespace changes * Revert whitespace changes --------- Co-authored-by: Mahadik, Mukul Chandrakant <[email protected]> Co-authored-by: K. Shankari <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cleaned up older versions for two packages:
urllib3 - deleted stale version folders
python - deleted tests folder