Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rollback from using ppdeep to python-ssdeep (libfuzzy) #868

Merged
merged 2 commits into from
Jul 31, 2023

Conversation

psrok1
Copy link
Member

@psrok1 psrok1 commented Jul 31, 2023

Your checklist for this pull request

  • I've read the contributing guideline.
  • I've tested my changes by building and running the project, and testing changed functionality (if applicable)
  • I've added automated tests for my change (if applicable, optional)
  • I've updated documentation to reflect my change (if applicable)

What is the current behaviour?

Using ppdeep was really bad idea because Python-based implementation is awfully slow (almost 500x comparing to libfuzzy)

For 40MB file it takes 2 minutes to evaluate a hash:

libfuzzy
24576:Nvk8JqACY/kDtsyDULLk9iFySeLMu/lUHGg:SzACj78LkPR1Kmg 0.24330997467041016
ppdeep
24576:Nvk8JqACY/kDtsyDULLk9iFySeLMu/lUHGg:SzACj78LkPR1Kmg 113.24647259712219

What is the new behaviour?

MWDB uses libfuzzy-dev native dependency like in v2.8.0. It requires native dependency to be installed.

Closing issues

closes #865

@psrok1 psrok1 marked this pull request as ready for review July 31, 2023 10:57
Copy link
Contributor

@msm-code msm-code left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sad but necessary

@psrok1 psrok1 merged commit 18c1339 into master Jul 31, 2023
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Performance: ppdeep introduced in v2.9.0 is awfully slow causing worker timeout
2 participants