Releases: OpenCTI-Platform/opencti
Version 2.1.3
Dear community, OpenCTI version 2.1.3 has been released! We mainly focused our work on the stability of the whole product as well as the resolution of some major bugs. However, several new features have also been implemented, in particular the possibility of exporting lists of entities in STIX2 or CSV format: exporting campaign indicators, reports about an intrusion sets, all malwares, etc. We have completed the refactoring of the Python library and added all the methods that were still missing on some entity types as well as the file upload feature. For the next release, we will carry on our huge refactor of workspaces and statistics in general. We also plan to conduct a documentation working session to improve it in depth.
Last but not least, we will soon plan the organization of a webinar that will be didicated to OpenCTI basics but also a very interesting use case for both red teams and blue teams: how to put all knowledge about an incident in OpenCTI and replay it using Caldera. Double advantage of such a scenario: the capitalization of knowledge related to the incident as well as the ability to test the reaction of SOC / CSIRT in the event of a similar attack. Your feedback and your impressions are precious, do not hesitate to send us your use cases and the difficulties you encounter!
⚠️ Breaking changes ⚠️
Grakn Core Server has been upgraded from 1.5.9 to version 1.6.2. The migration process of existing data cannot be done automatically even if you are using the Grakn Docker container. You have to follow the migration procedure Upgrading an Existing Installation to Grakn Core 1.6.x. available in the Grakn documentation. If you have any trouble to make this work, we are available to help you on our Slack channel or you can reach the Grakn team directly on their channel.
Enhancements:
- #424 Enhance searching of entities
- #412 Add OR/AND option to filter Observables & Indicators
- #396 Infinite loading lists in observables and entities of a report
- #391 Export lists of objects (intrusion sets, indicators, etc.)
- #390 Duplicate function askEnrich in the API
- #388 Display the number of entities in each view/lists
- #371 Migrate to Grakn 1.6.1
- #360 Redirect to the requested page after login
- #345 Add tagging for Tools/Vulnerabilities/Observables/Reports
- #334 Tags for reports
- #123 Implement CSV export
Bug Fixes:
- #432 Uploading a PDF threat report results in "Cannot read property 'toLowerCase' of null" error
- #427 Organisation type vs category
- #419 Filter with no tag not working anymore
- #416 infinite loading of reports using specific sort options
- #415 Mutation intrusionSetAdd creates an intrusion set with the first_seen value as the last_seen value
- #410 Exception when importing STIX 2 Indicators due to invalid default type in valid_from
- #407 Mandatory properties are not filled for inferred relations
- #405 Displaying an indicator constantly refreshes the page
- #404 Cannot create an Indicator of type Mac-Addr or Directory
- #381 New reports are added without a set "Processing Status"?
Version 2.1.2
Dear community, the OpenCTI platform version 2.1.2 has been released! This version provides users of the platform with many new features that will allow them to better modelize their CTI knowledge. We have resolved the gap that existed between the OpenCTI data model and the STIX2 schema by definitively separating the concepts of observables and indicators. It is now possible to create indicator in various formats: STIX Pattern, Snort, Sigma, YARA, etc.
All graphics and visuals are now based on ElasticSearch queries to increase performance. This will allow us to completely rework the workspaces and make them real monitoring tools in the next release. Also we have disabled all inferences rules on the platform by default, so a very important screen has been added to the settings section, allowing users to enable inferences rules if they really need it. The MISP connector has been fully refactored and a documentation is available. Finally, we have added many examples to the Python library.
Enhancements:
- #383 Improve performance by using Elastic when searching for relations (when we can)
- #375 Enhance display of relations and separate inferred/not inferred
- #366 Use ElasticSearch for all statistics (timeseries, distribution, etc.)
- #349 Enhance loaders/spinners everywhere
- #335 Detection rules (yara, snort, suricata, sigma, etc)
- #316 Split indicator/observable concepts, create indicators from observables
- #162 Refactor the observables schema to match STIX2 references
- #145 Enable/disable inferences rules in settings
- #58 Observables : expiration date
- #57 Observables : scoring/rating
Bug Fixes:
- #369 Store the remote IDs in some ES entities is useless and performance killer
Version 2.1.1
OpenCTI 2.1.1 has been released! This version is hotfixing 5 bugs (4 in the API/Frontend and 1 in the Python library) found after the last release. Thank you to all people who reported these bugs so we can now work on the next milestone. The next milestone will be focused on: improving performances of charts and relations display in the UI, development of many outputs and graphics (killchains, diamond model, PDF export of knowledge, full refactor/enhancement of workspaces, graph view of entities, comparison of threats TTPs/infrastructure and introduction of indicator concept.
Bug Fixes:
Version 2.1.0
Dear community, the OpenCTI platform version 2.1.0 has been released! This version is an important step for the future developments of OpenCTI as a full Cyber Threat Intelligence product. We have worked on major issues and features directly linked to what you can expect from OpenCTI and what we need, as a developers team, to build a powerful and durable application. We have done a lot of work on indexing in ElasticSearch and in general all the way API methods are organized (removing more than 7K lines of useless source code). Ingestion and reading performance have been improved by 12x or by 20x in some cases.
We have also completed the data model and have introduced very useful features to allow you to fully modelize threats that may target your organization. You are now able to directly link an observable on a relation "threat/incident => uses => TTP", for instance to indicate the registry key used for persistence or the sender email address of the phishing message. The attack patterns list has been reshaped to a true killchain with the description of each relation to ensure a better understanding of analysts. Observables can now be linked together, allowing you for instance to link hashes together if it corresponds to the same file, or link an IP address that resolves a domain name.
Last but not least, we have fully refactored the Python library and started to write a proper documentation, you have now access to many useful examples to interact with the OpenCTI platform in the Github repository. We will continue our efforts to make OpenCTI an indispensable tool for CTI, SOC and CSIRT teams around the world. We will soon publish usage and integration tips in existing workflows and plan a usecases-oriented webinar in January 2020.
Enhancements:
- #351 Be able to reset the state of a connector in the UI
- #339 ATT&CK techniques not searchable with their code
- #336 Add the ID of Attack Patterns
- #332 Observables must be able to indicate relations
- #319 Technical error thrown when not logged in
- #317 Observables filtering
- #315 Add Minio version in the "About tab"
- #314 Global performances improvement
- #308 Relations between observables
- #268 Global search in parameter of URL
- #266 Add a tags field on creation forms
- #245 Killchain view for Attack Patterns
- #219 Unable to add "localized in" relation
- #109 Import is really really slow
- #67 Export all entities to STIX2 JSON
Bug Fixes:
Version 2.0.2
OpenCTI 2.0.2 has been released! This version is mainly focused on fixing bugs, one affecting the graph database that could trigger out of memory issues in Grakn and the other leading to lost data in the workers if the API is not available. Please stay tuned for the next milestone (2.1.0) which will includes a huge work about data ingestion performances. We already know that this is the one of the most important weakness of the platform right now! Do not hesitate to send us your feedbacks on our last releases!
Enhancements:
- #309 Prevent worker for consuming messages if the API is down
- #298 Missing script for 'npm run schema'
Bug Fixes:
Version 2.0.1
We just released OpenCTI version 2.0.1! After the 2.0.0 some users reported us important bugs that are now fixed in this new version (especially on connectors & worker). We also introduced a persistent states in external import connectors such as MITRE or CVE to avoid re-sending messages to the queue each time the connector is restarted. Our work in the next milestones will be focused on improving the data ingestion speed and developing features to help users to massively handle entities and relationships in the platform (detect duplicates, merge, split, bulk delete, etc.).
Enhancements:
- #291 Add simple state management for connectors.
- #237 Improve search engine capacity
- #196 OpenCTI development environment documentation is outdated
Bug Fixes:
- #287 Subsectors cannot be added to sectors
- #285 Worker stopped consuming messages some processing
- #277 no inference relationship
- #276 Cannot export some reports
- #216 Searching for Entities returns inconsistent results
- #211 Multiple workers and PermanentBackendException: Permanent failure in storage backend
Version 2.0.0
We are proud to announce a new major release of the OpenCTI platform: 2.0.0 is out! Although the documentation is still under construction, this new version brings many features and improvements to users. It allows you to store and manage files, add tags to entities, easily create relationships to relationships in reports, and, depending on the available connectors, enable automatic enrichment on observables, extraction of indicators in PDF files and exports in different formats. Several bugs have been fixed and multiple improvements made in display and performance. We are waiting for your feedback and future contributions, especially on connectors!
⚠️ Breaking changes ⚠️
New dependency
- To handle file storage for import, export and files linked to entities, Minio has been introduced in the OpenCTI stack as a required component. In the future, any S3 storage system will be able to store the OpenCTI data and files.
- The file management system can be used by connectors to extract intelligence such as IoCs, TTPs or store any export from the platform (generated PDFs, STIX2, etc.).
Workers and connectors
- There is now only one worker for writing data coming from the RabbitMQ broker on the platform, so the
exportworker is deprecated. The worker remain the same base code, the parametertypeis no longer required. - To handle import and export (only STIX2 for the moment), 2 new connectors have been introduced.
- For the worker and connectors configuration, the RabbitMQ parameters are no longer needed, only the OpenCTI API hostname and token are required. RabbitMQ parameters are provided by the API through the Python helpers.
The new configuration of connectors is available in the dedicated documentation.
Enhancements:
- #254 Separate observables list of reports in a different QueryRenderer
- #249 Create new attack pattern to be associated to a report
- #244 Add a "drops" relation between malwares/tools.
- #241 Enhance the custom attributes management and update
- #236 Add version/build number and minimal system info in dashboard
- #232 Aliases display enhancement
- #229 Global tagging system
- #221 5 level certainty scale not adaptable
- #217 Better handling of concurrent integration
- #212 Remove "waiting behavior" from entrypoint, let docker restart the containers
- #204 Redesign the connector status page
- #191 Reduce opencti/platform docker image size
- #170 Add standalone observables
- #141 Observables don't appear when importing a file
- #130 Introduce file storage for export download
- #105 Add Kill Chain Phase selection when adding observable
- #69 Enhance knowledge graph of reports
- #61 Organisation : associated IP addresses, domain names, URL-s
- #48 Implement the observable enrichment
- #44 Attach files to report
- #43 Differenciate the display of sectors that are subsectors
- #42 Add relationships and knowledge everywhere
- #39 Add aliases to the generic entity creation form
- #38 Automatic graph organization on report
- #37 Display marking definitions in all entities / relations
- #34 Display entity information in a graph view
Bug Fixes:
- #235 The entity "Region" can't be added as the location property of a relation.
- #228 Inferred relations not displayed in the relationships lists
- #220 Inferred relation instrusion set - country - region
- #210 Unable to create a "Workspace" in the "Explore" view
- #209 Observables of entities cannot be sorted
- #136 Marking color
Version 1.1.2
Version 1.1.1
⚠️ Breaking changes ⚠️
ElasticSearch 6.X is no longer supported.
You need to upgrade your current elasticsearch deployment (docker or manual) to version 7.X.
Enhancements:
- #177 Remember views parameters for listing, sorting and searching
- #168 Adapt current github organization and documentation to improve release lifecycle
- #158 Refactor contextual list search
- #157 Speed-up statistics numbers on the dashboard
- #152 Add link to organization in report field "author"
- #151 Refactor all the infinite scroll list views
- #150 Migration to ES 7
- #7 Write an article about why we choose Grakn over Neo4j
Bug Fixes:
Version 1.1.0
⚠️ Breaking changes ⚠️
Integration and connectors
- The integration process
connectors_scheduler.pyand the Docker imageopencti/integrationhas been deleted and are no longer used. This has been replaced by the new connector architecture. - Connectors are no longer configured and enabled in the user interface, you have to launch them independently, please see the dedicated documentation on how to enable connectors.
Default credentials and token
- To launch the platform, you have to configure the default password and the default token of the platform, either in your
docker-compose.ymlenvironment variables or in theproduction.jsonconfiguration file. If you do not configure these parameters, the platform will not start and will raise an error.
Enhancements:
- #131 Keep UUIDs of STIX2 TLP marking definitions
- #127 OpenCTI and dependencies memory documentation
- #126 OpenCTI strategic roadmap
- #121 Integration connectors new architecture
- #104 Provide ability to add custom "Played Role" values when adding observables
- #90 Multiple documentation pages missing
- #88 OpenCTI fail to start with docker for windows
- #74 Admin account cannot be auto-created with a migration
- #73 Customizable report classes