Skip to content
/ scraper Public

Scraper to automatically pull domain names from STIX files on the latest National Cyber Awareness System Analysis Reports

Notifications You must be signed in to change notification settings

SKIIDK/scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

scraper

Spider

To run the scraper, from the spiders directory use scrapy runspider scrape.py. The current default method is set to get all files, but it can also be configured to only get MIFR files. All of the reports will be saved to the reports folder.

XML Parsing

To run the xml parser, run ./xmlformat.py <file-path>. This will output the domains file for the corresponding XML file.

About

Scraper to automatically pull domain names from STIX files on the latest National Cyber Awareness System Analysis Reports

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages