Create final bundle (i.e. enterprise-attack.json)

[payamsamimi]

Hi guys,

recently, I am working on a python recipe, which should generate a final json dataset such as enterprise-attack.json for our attack navigator. I thought, it could be an easy task, thank to this library. However, it changed to be a nightmare.
Using the query function of Filter class, I am able to gather all entries from a domain. However, once I want to gather all of them in a bundle and write to the disc, I have various problems.

Issue:
1- filtering tactics returns dict. I have read many of comments, solved issues as well as documentation, still I cannot create a bundle with tactics.

2- When I ignore the customObjects, adding other stix objects cause:
Invalid value for Bundle 'objects': This property may only contain a non-empty dictionary or object

3- When I use Sink to push to the disc, instead of creating an entry in the appropriate folder, a new folder with an unique ID is created, and a json file with a random name in it.

I would appreciate, if you could help me.

# reading the domain
src = stix2.FileSystemSource(stix_dir="./attack", allow_custom=True, encoding='utf-8')

tactics = src.query([stix2.Filter('type','=','x-mitre-tactic')])
tecs = src.query([stix2.Filter('type','=','attack-pattern')])
rel = src.query([stix2.Filter('type','=','relationship')])
course_of_actions = src.query([stix2.Filter('type','=','course-of-action')])
identity = src.query([stix2.Filter('type','=','identitiy')])
adversaries = src.query([stix2.Filter('type','=','intrusion-set')])
malwares = src.query([stix2.Filter('type','=','malware')])
marking_definition = src.query([stix2.Filter('type','=','marking-definition')])
tools = src.query([stix2.Filter('type','=','tool')])
matrix = src.query([stix2.Filter('type', '=', 'x-mitre-matrix')]);


final_bundle = Bundle(tactics, tecs, rel, course_of_actions,identity, adversaries, malwares, marking_definition, tools, matrix )


print(final_bundle)

[clenk]

Hi @payamsamimi, sorry for the difficulties. Looking at your comment on #270, I think some of the problem is that the ATT&CK content uses STIX 2.0 but this library now defaults to using STIX 2.1. You can fix this by specifically importing the 2.0 version of the classes.

For example if you register the custom class:

from stix2 import Filter, FileSystemSource, properties
from stix2.v20 import CustomObject, Bundle

@CustomObject('x-mitre-tactic', [ ('x_mitre_shortname', properties.StringProperty(required=True)) ])
class Tactic(object):
    pass

src = FileSystemSource('../cti/enterprise-attack')
tactics = src.query([Filter('type', '=', 'x-mitre-tactic')])

final_bundle = Bundle(tactics)

Alternatively, you can use allow_custom=True on both the Source and Bundle:

from stix2 import Filter, FileSystemSource
from stix2.v20 import Bundle

src = FileSystemSource('../cti/enterprise-attack', allow_custom=True)
tactics = src.query([Filter('type', '=', 'x-mitre-tactic')])

final_bundle = Bundle(tactics, allow_custom=True)

For your second issue, the bundle is expecting each parameter to be a single object rather than a list. You can get around this by merging them all into a single list:

final_bundle = Bundle(tactics + tecs + rel + course_of_actions + identity + adversaries + malwares + marking_definition + tools + matrix)

We'll try to change the Bundle constructor to support how you were passing the objects, too.

For the third problem, the ATT&CK repository only contains a single version of each object, but the FileSystemStore is built to handle multiple versions of each object. The FileSystem stores objects in folders for each object type, which contain a folder for each object's ID, which then contain a JSON file for each version of the object. The filename of each of these is the object's modified timestamp property, with any non-numerical characters left out. The FileSystem DataStore can ingest data structured like in the ATT&CK repo, but it will always write it out like this so it can support versioning of the objects.

Hope this helps!

[payamsamimi]

great thanks @clenk for your comment. My problem is solved by only adding allow_custom attributes and the following line:
final_bundle = Bundle(tactics + tecs + rel + course_of_actions + identity + adversaries + malwares + marking_definition + tools + matrix)
put it differently, in my case importing from stix2.v20 was not necessary.