Skip to content

Releases: TrevorBagels/PulsepointScraperV2

Incidents from 5/20 to 12/03 (2021)

10 Dec 02:07
e2d91d9
Compare
Choose a tag to compare

This is a backup of the data that was collected from 5/20/2021 to 12/03/2021. It contains over 4.4 million incidents scraped from 667 agencies. Import with mongorestore --archive="pulsepoint_dec_03" --nsFrom='pulsepoint.' --nsTo='pulsepoint.'

Note: Data between 09/22 and 10/13 is missing due to a ransom attack.
(File size is 3GB, so I had to compress it into a zip file)

Incidents from 5/20 to 11/03 (2021)

03 Nov 20:35
77026aa
Compare
Choose a tag to compare

This is a backup of the data that was collected from 5/20/2021 to 9/22/2021. It contains over 3.5 million incidents scraped from 666 agencies. Import with mongorestore --archive="pulsepoint_nov_03" --nsFrom='pulsepoint.' --nsTo='pulsepoint.'

Note: Data between 09/22 and 10/13 is missing due to a ransom attack.
(File size is 2.46GB, so I had to compress it into a zip file)

Incidents from 5/20 to 9/22 (2021)

23 Sep 01:56
77026aa
Compare
Choose a tag to compare

This is a backup of the data that was collected from 5/20/2021 to 9/22/2021. It contains over 3 million incidents scraped from 655 agencies. Import with mongorestore --archive="pulsepoint_sep_22" --nsFrom='pulsepoint.*' --nsTo='pulsepoint.*'

Update (10/13/2021):
Bad news to anyone that actually uses these releases, after making this backup I moved all of my operations to a virtual machine in the cloud. And at some point I wanted to use MongoDB Compass to view my data, which I struggled with, eventually leading to me making the database super insecure, and 8 days later it got hit with a ransom attack. So all data between 9/22 and 10/13 has been lost. To anyone that uses this data (I'm pretty sure nobody does at the time that I'm writing this), I apologize for my lack of security.

Incidents from 5/20 to 9/02

03 Sep 07:46
77026aa
Compare
Choose a tag to compare

This is a backup of the data that was collected from 5/20 to 9/02. It contains over 2.5 million incidents scraped from 632 agencies.
Import with mongorestore --archive="pulsepoint_sep_02" --nsFrom='pulsepoint.*' --nsTo='pulsepoint.*'

Incidents from 5/20 - 7/01

01 Jul 19:25
77026aa
Compare
Choose a tag to compare

This is a backup of the data that was collected from 5/20 to 7/01. It contains over 1 million incidents scraped from 632 agencies.
Import with mongorestore --archive="pulsepoint_jul_01" --nsFrom='pulsepoint.*' --nsTo='pulsepoint.*'

Missing data is a possibility, although there should be much less of it than there was in the first release, as the program has been improved in order to collect more data with minimal data loss.

Incidents from 5/20 - 6/10

11 Jun 17:40
Compare
Choose a tag to compare

This release is NOT a program.
Backup of the incident database from scraped PulsePoint data, contains data from 629 agencies across the U.S from 5/20 to 6/10.
Import with mongorestore --archive="pulsepoint_jun_10" --nsFrom='pulsepoint.*' --nsTo='pulsepoint.*'

There may be missing data (likely only a couple of days), this is because I still don't have a fully dedicated machine for running my programs on and because I started collecting this data during development.