Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ScanCapa, Add Tests, Add Elf #277

Merged
merged 19 commits into from
Jan 12, 2023
Merged

Conversation

ryanohoro
Copy link
Collaborator

@ryanohoro ryanohoro commented Jan 8, 2023

Describe the change

Installs Fireeye Capa with pip, most recent version. Installs default rules, signatures.

Adds separate rules and signatures location options.

Adds three tests for ScanCapa.

Adds elf files to ScanCapa taste config.

This was intended to include code for #159, however, Fireeye Capa is not designed to be used as a module. Its structure and monolithic main.py makes it very unappealing.

Removes a print() from ScanDmg.

Describe testing procedures

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.2.0, pluggy-1.0.0
rootdir: /strelka
plugins: mock-3.10.0, unordered-0.5.2
collected 71 items

tests/test_required_for_scanner.py .
tests/test_scan_base64.py .
tests/test_scan_base64_pe.py .
tests/test_scan_batch.py .
tests/test_scan_bmp_eof.py .
tests/test_scan_bzip2.py .
tests/test_scan_capa.py ...
tests/test_scan_ccn.py .
tests/test_scan_dmg.py .
tests/test_scan_docx.py .
tests/test_scan_elf.py .
tests/test_scan_email.py .
tests/test_scan_encrypted_doc.py ....
tests/test_scan_encrypted_zip.py ..
tests/test_scan_entropy.py .
tests/test_scan_exiftool.py ..
tests/test_scan_footer.py .
tests/test_scan_gif.py .
tests/test_scan_gzip.py .
tests/test_scan_hash.py .
tests/test_scan_header.py .
tests/test_scan_html.py .
tests/test_scan_ini.py .
tests/test_scan_iso.py .
tests/test_scan_javascript.py .
tests/test_scan_jpeg.py ..
tests/test_scan_json.py .
tests/test_scan_libarchive.py ......
tests/test_scan_lnk.py .
tests/test_scan_lzma.py .
tests/test_scan_macho.py .
tests/test_scan_manifest.py .
tests/test_scan_msi.py .
tests/test_scan_ocr.py .
tests/test_scan_pdf.py .
tests/test_scan_pe.py .
tests/test_scan_png_eof.py ...
tests/test_scan_qr.py .
tests/test_scan_rar.py .
tests/test_scan_strings.py .
tests/test_scan_tar.py .
tests/test_scan_upx.py .
tests/test_scan_url.py .....
tests/test_scan_vhd.py ..
tests/test_scan_x509.py ..
tests/test_scan_xml.py .
tests/test_scan_yara.py .
tests/test_scan_zip.py ..

=============================== warnings summary ===============================
...
======================= 72 passed, 25 warnings in 13.22s =======================

Sample output

No changes.

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of and tested my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings

@ryanohoro ryanohoro changed the title Fix ScanCapa, Add Test, Re-enable in backend.yml, Add Elf Fix ScanCapa, Add Test, Re-enable, Add Elf Jan 8, 2023
@ryanohoro ryanohoro changed the title Fix ScanCapa, Add Test, Re-enable, Add Elf Fix ScanCapa, Add Test, Add Elf Jan 8, 2023
@ryanohoro ryanohoro changed the title Fix ScanCapa, Add Test, Add Elf Fix ScanCapa, Add Tests, Add Elf Jan 8, 2023
@@ -25,11 +26,11 @@ oletools==0.56.1
opencv-contrib-python==4.6.0.66
opencv-python==4.6.0.66
openpyxl==3.0.9
pefile==2019.4.18
pefile
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ryanohoro Do we want to pin a pefile version here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a dependency conflict with flare-capa that pip identified. It was automatically resolved by pip when I removed the pin. I'm not sure which version it picks, though I can go back and check which one pip picked.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a pin version suggested by pip.

@@ -8,6 +8,7 @@ cryptography==3.4.7
docker==5.0.0
eml-parser>=1.17
esprima==4.0.1
flare-capa==4.0.1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we using flare-capa in this PR?

Copy link
Collaborator Author

@ryanohoro ryanohoro Jan 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We switched from the binary release (which I think is just a python binary wrapper) to the python module release which is flare-capa in pip. It's still called from subprocess.POpen very similarly though. I was attempting to close #159 and used Method #2 for installation.

It turns out that capa isn't really built to be imported as a module into a script, the main script is a bit of a monolith and it wasn't obvious how to use it the way we want (not documented for that either).

@ryanohoro
Copy link
Collaborator Author

An issue where the build was failing occasionally has been fixed by extending scanner_timeout for ScanCapa.

@phutelmyer phutelmyer merged commit 3caa18c into target:master Jan 12, 2023
@ryanohoro ryanohoro deleted the scan_capa branch January 19, 2023 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants