Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix security warnings #365

Merged
merged 3 commits into from
May 7, 2024
Merged

Fix security warnings #365

merged 3 commits into from
May 7, 2024

Conversation

oshoma
Copy link
Collaborator

@oshoma oshoma commented May 6, 2024

Description

Add Bandit and address Bandit security warnings

This change improves the security of our code.
When you run make lint, Bandit will now do some security analysis.

Run poetry install after picking up these changes.

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Maintenance
  • New release

Related issues

Tracked in SHR-46

Checklists

To speed up the review process, please follow these checklists:

Development

  • The Pull Request is small and focused on one topic
  • Lint rules pass locally (make format && make lint)
  • The code changed/added as part of this pull request has been covered with tests
  • All tests related to the changed code pass in development (make test)
  • The changes generate no new warnings (or explain any new warnings and why they're ok)
  • Commit messages are detailed
  • Changed code is self-explanatory and/or I added comments
  • I updated the documentation (docstrings, /docs)
    See the testing guidelines for help on tests, especially those involving web services.

Code review

  • This pull request has a descriptive title and information useful to a reviewer. There may be a screenshot or screencast attached.
  • I have performed a self-review of my code
  • Issue from task tracker has a link to this pull request

💔 Thank you for submitting a pull request!

oshoma added 2 commits May 6, 2024 16:35
@oshoma
Address Bandit security warnings
2b806ef 
Fixes the following Bandit errors seen with `make lint`:

1. Issue: [B104:hardcoded_bind_all_interfaces] Possible binding to all interfaces.
   Severity: Medium   Confidence: Medium
   CWE: CWE-605 (https://cwe.mitre.org/data/definitions/605.html)
   More Info: https://bandit.readthedocs.io/en/1.7.8/plugins/b104_hardcoded_bind_all_interfaces.html
   Location: apps/slackapp/slackapp/bolt_app.py:346:13

Change: suppress the warning with #nosec B104, since our use of
host '0.0.0.0' is intentional. Also add a security note.

2. Issue: [B113:request_without_timeout] Requests call without timeout
   Location: apps/slackapp/slackapp/bolt_app.py:340:23
   Location: sherpa_ai/scrape/extract_github_readme.py:75:19
   Location: sherpa_ai/utils.py:83:15
   Location: sherpa_ai/scrape/extract_github_readme.py:52:19
   Location: sherpa_ai/scrape/file_scraper.py:69:19

Change:  add timeouts for each of these. Adjustment may be needed.

3. Issue: [B310:blacklist] Audit url open for permitted schemes.
Allowing use of file:/ or custom schemes is often unexpected.
   Severity: Medium   Confidence: High
   CWE: CWE-22 (https://cwe.mitre.org/data/definitions/22.html)
   More Info: https://bandit.readthedocs.io/en/1.7.8/blacklists/blacklist_calls.html#b310-urllib-urlopen
   Location: sherpa_ai/tools.py:59:15
   Location: sherpa_ai/utils.py:294:12

Change: add logic to ensure URLs are HTTP or HTTPS. Add tests.
@oshoma
Remedy make format && make lint warnings
65f46c4 
poetry run flake8 --ignore=W503,E501,F401 sherpa_ai tests

sherpa_ai/utils.py:497:8: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`

sherpa_ai/utils.py:507:13: F841 local variable 'e' is assigned to but never used

sherpa_ai/utils.py:554:114: W291 trailing whitespace

tests/integration_tests/test_entity_citation_validator.py:53:5: F811 redefinition of unused 'get_llm' from line 13

tests/integration_tests/test_qa_agent_actions.py:46:49: F811 redefinition of unused 'get_llm' from line 8

tests/integration_tests/test_qa_agent_actions.py:72:55: F811 redefinition of unused 'get_llm' from line 8

tests/unit_tests/test_util.py:244:1: F811 redefinition of unused 'test_extract_numbers_from_text' from line 182

tests/unit_tests/test_util.py:299:1: F811 redefinition of unused 'test_extract_numbers_from_text' from line 244

tests/unit_tests/test_util.py:382:25: E712 comparison to True should be 'if cond is True:' or 'if cond:'

tests/unit_tests/test_util.py:390:25: E712 comparison to False should be 'if cond is False:' or 'if not cond:'

tests/unit_tests/test_util.py:405:1: F811 redefinition of unused 'test_text_similarity_with_entities_exist' from line 397
@oshoma oshoma requested a review from 20001LastOrder May 6, 2024 20:55
@oshoma
Rename test_util.py -> test_utils.py
ec3645b 
...to match the name of the file being tested.
20001LastOrder
20001LastOrder approved these changes May 7, 2024
View reviewed changes
Copy link
Collaborator

@20001LastOrder 20001LastOrder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, merging...

@20001LastOrder 20001LastOrder merged commit 8edeb7c into Aggregate-Intellect:main May 7, 2024
1 check passed
@oshoma oshoma deleted the fix-security-warnings branch May 7, 2024 21:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@20001LastOrder 20001LastOrder 20001LastOrder approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@oshoma @20001LastOrder