672 & 681-upgrade django (3.2.13) and networkx (2.6.3)

[cnlucas]

Summary (required)

• Resolves [March] eRegs parsing requirements #672
• Resolves [Snyk:High] django SQL Injection (due by 05/13/2022) #681

(Include a summary of proposed changes and connect issue below)
Networkx 2.6.3 upgrade highlights:
Replace deprecated method .node with .nodes in regulation-parser. Also, removes low numpy snyk issues from 2.6
(also updated gitdb and smmap version bc I was getting circular dependency issue when I pulled the parser repo)

Upgraded django to 3.2.13 in regulations-parser, regulations-site, and regulations-core.

Required reviewers

2-3 devs

Impacted areas of the application

General components of the application that this PR will affect:

• Legal Resources/ Regulations

Related PRs

Regulations-Parser PR:
fecgov/regulations-parser#4
Regulations-Core PR:
fecgov/regulations-core#4
Regulations-Site PR:
fecgov/regulations-site#4

How to test

(Include any information that may be helpful to the reviewer(s). This might include links to sample pages to test or any local environmental setup that is unusual such as environment variable (never credentials), API version to point to, etc)

Terminal#1:

1. create python 3.7.12 virtualenv for fec-eregs: run pyenv virtualenv 3.7.12 venv-eregs-3712
2. pyenv activate venv-eregs-3712
3. checkout branch
4. open requirements.txt and change the lines for the parser, site, and core to point to my PR's (lines 36-44):

# regparser
-e git+https://github.com/fecgov/regulations-parser.git@upgrade-networkx-and-django#egg=regparser

# regsite
-e git+https://github.com/fecgov/[email protected]#egg=regulations

# regcore
-e git+https://github.com/fecgov/[email protected]#egg=regcore

6. open requirements-parsing.txt and do the same thing (lines 72-79):

# regparser
-e git+https://github.com/fecgov/regulations-parser.git@upgrade-networkx-and-django#egg=regparser

# regsite
-e git+https://github.com/fecgov/[email protected]#egg=regulations

# regcore
-e git+https://github.com/fecgov/[email protected]#egg=regcore

7. install the requirements.txt: run pip install -r requirements.txt (make sure you're using version 22.0.4 of pip or upgrade according to warning and re-run) (Also it will ask you if you want to use the old version or switch to my branches and you should type s)
8. remove node_modules: run rm -rf node_modules
9. run npm i (you can make sure you're running the proper version with nvm install 14.15.5)
10. run npm run build
11. run dropdb eregs-db if the eregs database already exist.
12. run createdb eregs-db (same name as defined in local_settings.py)
    create a new local_settings.py with the following configuration if one doesn't exist:

API_BASE = 'http://localhost:8000/api/'
DATABASES = {
  'default': {
    'ENGINE': 'django.db.backends.postgresql_psycopg2',
    'NAME': 'eregs-db',
    'HOST': '127.0.0.1',
    'PORT': '5432',
  }
}

14. run python manage.py migrate
15. run python manage.py compile_frontend (if you don't already have a compiled folder then mkdir compiled)
16. run python manage.py runserver (leave this running)

Terminal#2:

1. create python 3.7.12 virtualenv for parser: run pyenv virtualenv 3.7.12 venv-parser-3712
2. pyenv activate venv-parser-3712
3. install parser requirements: run pip install -r requirements-parsing.txt (same as above with pip version 22.0.4 and nvm version v.14.15.5) NOTE: You will see a warning about networkx being yanked that's ok
4. run snyk test --file=requirements-parsing.txt --package-manager=pip (you should not see the networkx issues or django issues anymore)
5. parse 2021 regs on to local db: run python load_regs/load_fec_regs.py local

This is my first time with e-regs, so please let me know if there's improvements I can make! Thank you very much!

[pkfec]

10. run npm audit fix

@cnlucas ^^ is not required to run while testing requirement file changes. Running npm audit fix updates package dependencies inside package-lock.json which is not needed. The only time we need package-lock.json to be modified is when we update npm packages inside package.json. Hope this help! You can remove Step#10 from your test instructions. Rest of the test instructions look right.

[cnlucas]

Thanks @Priya! Deleted. :)

[pkfec]

@cnlucas Your changes work great. Good job!

[fec-jli]

Great job. work as expected. Thanks