How to merge is_notice=false with is_notice=true with schedule_e (independent expenditures)

[hobbes3]

I'm having a hard time understanding how to merge is_notice=true, which are the 24- and 48-hour reports, with is_notice=false for schedule_e (independent expenditures). For my understanding, I simply see is_notice=true as more recent, but temporary reports, whereas I see is_notice=false as archived and permanent reports.

Right now I pulled both types of reports (for Clinton and Trump only) and summed up the chart by expenditure amount:

And by count:

So in order to get all the recent reports that haven't been "converted" from is_notice=true to is_notice=false then is it say to say I should take all is_notice=false reports from September and merge it with all is_notice=true from October and November (the only two months where the bar is all orange). But in reality October and November aren't completely is_notice=true, it's just more like 99% of it (so you can't see the blue part in the chart).

The actual last day (most recent) is_notice=false is November 24th, but the spending and count for is_notice=false for October and November is tiny compared to is_notice=true for those two months:

         false         true
2016-06	 $  6,489,033  $  24,783,350
2016-07	 $ 22,980,842  $  39,818,757
2016-08	 $ 26,523,522  $  54,267,538
2016-09	 $ 10,996,667  $  54,834,798
2016-10	 $    303,817  $ 148,336,866
2016-11	 $        210  $  17,871,342

         false   true
2016-06	  1187   1146
2016-07	  4572   4560
2016-08	 11420  13004
2016-09	 16342  17218
2016-10	    14  26416
2016-11	     1	 6950

Can someone (perhaps @LindsayYoung) give some more insight in this? :-)

[LindsayYoung]

You are off to a good start. One thing to note is that you want to add the notices after the coverage date of the last report the last reporting period was the pre-general. It was due October 27 and the coverage date on that ended October 19th The next report is due Dec 8 and will cover Oct 20- Nov 28.

For now, you want is_notice=false for 2016-10-19 and before combined with is_notice=true for 2016-10-20 and after.

[hobbes3]

Thanks for the response!

How do I find out what the coverage dates are? For example, how do I now coverage date was due Oct 27 and ended Oct 19?

Is there documentation or some kind of formula to determine when a coverage is due and what period it affects?

[LindsayYoung]

The reporting deadlines and coverage dates are here:
http://www.fec.gov/info/report_dates_2016.shtml

Different kinds of groups file on different schedules so It can get tricky in cases where not all groups report at the same time.

[hobbes3]

Thanks!

[hobbes3]

Hello @LindsayYoung, could you or someone else verify that the report for Oct 20-Nov 28 actually made it?

I did a recent pull (from a few days ago) and I still only see the latest is_notice=false to be 10/18 sub_id= 4102420161344575779 (with one outlier at 11/24 sub_id= 4101220161340820027). But is_notice=true has many events in November and some in December.

[hobbes3]

Here is how many events I see for is_notice:

[LindsayYoung]

Thanks for reporting this!

I took a quick look and I am not seeing those records. I think it will be resolved by: https://github.com/18F/openFEC/pull/2052 which is almost done but in testing.

We will take another look at that on Monday.

[noahmanger]

Do we know if #2052 resolved this issue?

[hobbes3]

@noahmanger I believe so. I just did a recently pull with only is_notice=false and the most recent event is now 12/30.

Date        Count
2015-05	    183
2015-06	    320
2015-07	    131
2015-08	    104
2015-09	    122
2015-10	    226
2015-11	    247
2015-12	    270
2016-01	    566
2016-02	    1179
2016-03	    1061
2016-04	    1150
2016-05	    1298
2016-06	    1283
2016-07	    4897
2016-08	    11814
2016-09	    17466
2016-10	    2723
2016-11	    1119
2016-12	    13

I'm a bit confused on how to read the reporting deadline and coverage date link that @LindsayYoung provided (http://www.fec.gov/info/report_dates_2016.shtml) so I can't comment on if the recent date of 12/30 is right.