Skip to content
This repository has been archived by the owner on May 22, 2024. It is now read-only.

[WIP] 672-Upgrade networkx to 2.6 #682

Closed
wants to merge 1 commit into from
Closed

Conversation

cnlucas
Copy link
Member

@cnlucas cnlucas commented Apr 21, 2022

Summary (required)

(Include a summary of proposed changes and connect issue below)
Networkx 2.6 upgrade highlights:
Replace deprecated method .node with .nodes in regulation-parser
(also updated gitdb and smmap version bc I was getting circular dependency issue when I pulled the parser repo)

Required reviewers

2-3 devs

Impacted areas of the application

General components of the application that this PR will affect:

  • Legal Resources/ Regulations

Related PRs

Regulations-Parser PR:
fecgov/regulations-parser#3

How to test

(Include any information that may be helpful to the reviewer(s). This might include links to sample pages to test or any local environmental setup that is unusual such as environment variable (never credentials), API version to point to, etc)

Terminal#1:

  1. create python 3.7.12 virtualenv for fec-eregs: run pyenv virtualenv 3.7.12 venv-eregs-3712
  2. pyenv activate venv-eregs-3712
  3. checkout branch
  4. open requirements.txt and change line 37 from:
    -e git+https://github.com/fecgov/regulations-parser.git@master#egg=regparser
    to:
    -e git+https://github.com/fecgov/[email protected]#egg=regparser
  5. open requirements-parsing.txt and change line 73 from:
    -e git+https://github.com/fecgov/regulations-parser.git@master#egg=regparser
    to:
    -e git+https://github.com/fecgov/[email protected]#egg=regparser
    (The above will point to my updated branch which I will merge into regulations parser if everyone approves this PR)
  6. install the requirements.txt: run pip install -r requirements.txt (make sure you're using version 22.0.4 of pip or upgrade according to warning and re-run) (also it will ask you if you want to use the old version or switch and you should type s)
  7. remove node_modules: run rm -rf node_modules
  8. If you have already parsed on your local (have a src file) run rm -rf src
  9. run npm i (you can make sure you're running the proper version with nvm install v.14.15.5)
  10. run npm audit fix
  11. run npm run build
  12. run dropdb eregs-db if the eregs database already exist.
  13. run createdb eregs-db (same name as defined in local_settings.py)
    create a new local_settings.py with the following configuration if one doesn't exist:
API_BASE = 'http://localhost:8000/api/'
DATABASES = {
  'default': {
    'ENGINE': 'django.db.backends.postgresql_psycopg2',
    'NAME': 'eregs-db',
    'HOST': '127.0.0.1',
    'PORT': '5432',
  }
}
  1. run python manage.py migrate
  2. run python manage.py compile_frontend (if you don't already have a compiled folder then mkdir compiled)
  3. run python manage.py runserver (leave this running)

Terminal#2:

  1. create python 3.7.12 virtualenv for parser: run pyenv virtualenv 3.7.12 venv-parser-3712
  2. pyenv activate venv-parser-3712
  3. install parser requirements: run pip install -r requirements-parsing.txt (same as above with pip version 22.0.4) NOTE: You will see a warning about networkx being y
  4. run snyk test --file=requirements-parsing.txt --package-manager=pip (you should not see the networkx issue anymore, but you will see the django and numpy issues that will be addresssed in [Snyk:High] django SQL Injection (due by 05/13/2022) #681)
  5. parser 2021 regs on to local db: run python load_regs/load_fec_regs.py local

This is my first time with e-regs, so please let me know if there's improvements I can make! Constructive criticism super welcome.

@cnlucas cnlucas changed the title [WIP] upgrade networkx to 2.6 672-Upgrade networkx to 2.6 Apr 21, 2022
@cnlucas cnlucas requested review from fec-jli, hcaofec, johnnyporkchops and pkfec and removed request for johnnyporkchops, pkfec, fec-jli and hcaofec April 21, 2022 17:41
@cnlucas cnlucas changed the title 672-Upgrade networkx to 2.6 [WIP] 672-Upgrade networkx to 2.6 Apr 21, 2022
@cnlucas
Copy link
Member Author

cnlucas commented Apr 26, 2022

Closing in favor of #683 (easier to review all at the same time)

@cnlucas cnlucas closed this Apr 26, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[March] eRegs parsing requirements
1 participant