v2.14.0

New features and improvements:

• Performance improvements:
  • Set collation of some string fields to 'C' to better utilize btree index. Be aware that database migration may took a while (#959)
  • Add index on comment.object_id column (#988)
• Open-ID Connect improvements:
  • Fetching userinfo claims from userinfo endpoint, previously we assumed that they'll be send along with ID Token (#972)
  • Plugin-customizable OpenIDProvider class (#982)
  • OpenID Provider group is no longer a workspace and doesn't imply sharing objects within group (#974)
• Custom rate limit settings for specific group of users (#987)
• Allow to set limit of tags returned by TagListResource (#960)

Bugfixes:

• Fix internal server error when querying without field (by @msm-cert in #957)
• Fix: 'mwdb-core configure web' command after transition to Vite (#966)
• Fix: 'Back' doesn't work in RecentView because of navigation loop (#975)

Full Changelog: v2.13.0...v2.14.0

v2.13.0

This release is focused on further improvements of search performance and bugfixes.

It's recommended to upgrade your karton-system to v5.4.0 before applying this upgrade.

New features and improvements:

• Improved performance of object lists in Web UI (#949)
• Improved performance of wildcard queries for JSONB fields by @psrok1 in #943
• Karton is upgraded to v5.4.0 with much faster analysis status lookup method (#938)
• Tags are passed to Karton tasks (by @aBUDmdBQ in #934)
• Frontend: added warning banner when server version is different than client version, so user needs to clear cache (#950)
• Allow to set custom upload size limit via NGINX_MAX_UPLOAD_SIZE env var in mwdb-web Docker image
  (#930)

Bugfixes:

• Fix: ISE 500 on concurrent tag addition (#926)
• Fix: ISE 500 when non-numerical value appears in range search in JSON column by @psrok1 in #953
• Fix possible issues with plugins after replacing Flask-RESTful with own implementation (#937)
• Fix searching in diff mode (#941)
• Fix too eager schema for FileItemResponseSchema.latest_config field that affected performance of getting file items (#942)
• Fix unnecessary joined relationship for 'favorite' parameter affecting performance of searching and getting object lists (#948)

New Contributors

• @aBUDmdBQ made their first contribution in #934

Full Changelog: v2.12.0...v2.13.0

v2.12.0

Upgrade highlights:

If you use plugins that are adding new endpoints to the API, you need to fix Resource imports before upgrade: What's changed

New features and improvements:

• Support for Prometheus metrics (#908, Prometheus metrics docs)
• Refactored search engine to improve search performance for configs, attributes and file names (#906)
• UI: changed attribute adding modal to always show preview and make JSON values less concerning (#921)
• Limited default verbosity of logs (#909, see note about enable_debug_log)
• Added support for execute attribute that is passed to Karton to enable/disable execution in sandbox (#904, thanks @msm-cert)
• Flask-Limiter was replaced with direct use of limits library (#915)
• Dropped usage of Flask-Restful (#916)

Bug fixes:

• Fix: installation issues on Debian 12 (bumped psycopg2-binary to 2.9.9, #922)
• Fix: ISE 500 on user removal (#913)
• Fix: Don't treat 0 number as an empty attribute value (#920)
• Fix: MWDB doesn't surrender on Karton when can't be loaded eagerly and tries to load it lazily (#919)
• Web: Fixed race condition with applying request_timeout value (#905)

Full Changelog: v2.11.0...v2.12.0

v2.11.0

New features and improvements:

• Simple UI for uploading configurations and blobs (by @postrowinski in #803)
• Object tabs are extendable via plugins (by @yankovs in #896)
• Bump karton-core to v5.3.2 (by @yankovs in #902)

Full Changelog: v2.10.3...v2.11.0

v2.10.3

Bugfixes:

• Bumped Werkzeug to 3.0.1 including critical fix for very slow upload of huge files having speciifc layout (see #885)
• Fixed uneditable parent field in Upload view (#879)

v2.10.2

Bugfixes:

• Fixed default web_bundle_dir so webapp works correctly in standalone PyPi installation (#873)

v2.10.1

Most important change in this bugfix release is rollback to libfuzzy2 for ssdeep hash evaluation instead of ppdeep introduced in v2.9.0. It means that for standalone installation (from PyPi) you need to install libfuzzy2 before applying this upgrade.

Bugfixes:

• Removed ppdeep dependency in favor of previously used python-ssdeep (libfuzzy2) that caused worker timeouts when big file (>30MB) was uploaded (#868)
• Search in Search tab wasn't working (#871)
• Fixed flickering and showing wrong results when user navigated to query URL (#867)
• Fixed multiple issues in Rich attributes:
  • wrong search URL for searchable fields in lists (#869)
  • search link was not rendering correctly when searchable field value contained space (#870)

Other changes:

• included PID argument and before_request entries in log (#861)

v2.10.0

Small, minor release that provides bugfixes and Karton integration improvements:

It's recommended to upgrade karton-system to v5.2.0 before upgrading mwdb-core (if Karton is used within your pipeline).

Minor improvements:

• Karton: quality and share_3rd_party headers are using persistent headers and they're automatically added to all tasks within analysis (#858)
• Bumped Karton to v5.2.0

Bugfixes:

• Fixed issues related with "hex" sample preview mode (#859)
• Bumped PyYAML to v6.0.1 solving installation problems related with Cython release (#857)

v2.9.0

This release includes huge database migrations made for query optimization, which includes rewriting of object permission tables. Database backup is highly recommended before upgrade.

Also there is a long changelog ahead, so please read information about most important changes in What's changed section before upgrade.

Major changes:

• Huge improvements in Web part which includes:
  • Beautified login/registration pages (#726)
  • Usage of Vite and Rollup for building instead of Create React App and Webpack (#741). If you have in-house plugins, read the What's changed section in documentation.
  • Rewrite to TypeScript (#807, kudos @postrowinski!)
  • Closable error messages (#763)
• Search should be much much faster because of these changes:
  • Counting of all results before applying actual query is optional and disabled by default as it has huge impact on performance (#718)
  • When user has access_all_objects capability, exclusive object permissions are not even considered in query (#783). It also means that access_all_objects really gives access to all
    objects in system (it's not "autosharing" of all added objects as before), so everything group is effectively useless and is not created by default.
• Changes in shares representation, so it's more clear who is the actual uploader of the sample. It's better described here (#717)
• certpl/mwdb Docker image uses gunicorn instead of uwsgi, as uwsgi project was mostly abandoned (#735)
• v2.9.0 comes with additional small feature that enables you to ask your users for consent to share samples with 3rd party services (#801)
• Karton is bumped to v5.1.0 and its producer shows in services tab in Karton Dashboard
• Object listing endpoints are accepting count parameter, so you can load them in chunks bigger than 10 (#755)

Minor changes and improvements:

• Dedicated group is created for each OpenID Connect provider (#668)
• ssdeep is replaced with pure-Python implementation - ppdeep (#692)
• sharing_objects capability was renamed to sharing_with_all which better describes its real meaning (#696)
• Backslashes are better handled in configuration search (#690)
• Rich attributes: field can be rendered as search link using special {{@value}} syntax (#628)
• Sample preview downloads sample in obfuscated form (with negated bits) to not trigger EDR/AV solutions (#721, thanks @middleware99!)
• Added access_uploader_info capability to make users able to search for uploaders from the outside of our groups without giving powerful sharing_with_all capability (#705)
• Rich preview in AttributeAddModal (#724)
• Handle 'misc:' as a proper tag (#742, thanks @jasperla!)
• OAuth logout, so you can easily logout yourself from OAuth provider e.g. to switch accounts (#732)
• Configurable upload size (#756)
• Critical error in Web shows JS stack information (#790)
• Capabilities can be changed also in User/Group view instead of only Access control page (#770)
• User is warned in Relations tab when number of relations exceeds 1000 (#791)
• use_x_forwarded_for option in configuration to respect X-Forwarded-For header, enabled by default in Docker images (#845)

Bugfixes:

• NetworkError exceptions in Web are a bit better handled and they shouldn't crash whole application so often (#846)
• OpenID Connect: fixed provider registration (4e015b6, thanks @v-rzh!)

Special thanks to @yankovs for tracking some regressions during development!

And finally thanks to development team that worked on this release: @KWMORALE, @Repumba, @postrowinski, @olivergav, @nazywam.

Hopefully we'll be publishing stable releases a bit more often so the changelogs won't be that long 🥲

v2.8.2

This release contains bugfixes related mostly with S3 object storage. Regressions were introduced by migration from py-minio to Boto3 AWS SDK which apparently wasn't tested enough.

Bugfixes:

• Fix: Karton re-analysis was not possible when sample was stored on S3 (#695)
• Fix: AWS IAM authentication doesn't work since v2.8.0 (#704)
• Fix: Unable to delete objects with comments (#702)