[WIP] State page prototype

[shawnbot]

😎 preview on Federalist

This is a very work in progress PR that adds state pages with data from our new database workflow in #1353. The goal here is to determine whether we're on the right track in our wireframing exercise, #1347.

There's no index of all of the state pages yet, but you can see them by changing the URL, for instance with Utah and Texas. Right now we've only got revenue and production on federal land, but all lands production and jobs are on the way.

[houndci-bot]

Use %w or %W for an array of words.

[shawnbot]

As of now, you can see some listings for the "top products" of each state in terms of revenue, all lands, and federal production. For instance: California, Utah, and Texas. You can see some edge cases of very little data in states like Massachusetts, Rhode Island, Delaware, and Georgia.

[gemfarmer]

@shawnbot --> eee5880 adds Exports data. Not sure if we want to include more (namely rank), but this should be a start

[shawnbot]

Heads up, @RyanSibley: I just pushed an update that adds percentage figures to the state revenue and production historical tables, and adds some CSS to right-align numbers so that they're easier to read. Federalist should be done building it in a couple of minutes.

Next up: disbursements!

[shawnbot]

Okay, now we've got "top-level" state disbursements by fund, which only includes "Onshore" and "Offshore" fund sources. I've broken these down in a table like so:

Year	Total	Onshore	Offshore
2012	$111,618,105	$103,072,839	$8,545,266
2013	$102,272,570	$93,817,155	$8,455,415

I only see 2012 and 2013 data in this spreadsheet, which is not super interesting, but it's something. Next step is to see how complicated it'll be to incorporate the fund-specific data, such as historic preservation.

[meiqimichelle]

@shawnbot it begins! I'm excited. As for the two years -- yep, not super interesting. I remember on our beta site back in the day we did the bubble chart -- and switching between the years turned out to be a pretty big yawn because the funds don't change much year to year. The story here is not likely in the year-over-year change.

[shawnbot]

Heads up, @gemfarmer: you'll need to npm install to update the dependencies next time you pull down this branch to work on data stuff. The new version of tito does map and filter transforms and can handles the "unrolling" of columns natively now, so there's no need for custom script.

[shawnbot]

So I have the historic preservation fund data in our db now, but I'm not quite sure what to do with it on the state pages. The HPF (Historic Preservation Fund) data goes from 2011-2014, so it overlaps with the summary data, but I feel like it would be weird to show only HPF data for 2011 and 2014. Maybe we only show HPF data for the years that we have summary data?

It's worth trying out, but I have a feeling that the HPF values are going to be confusing because—according to @mentastc, and the summary spreadsheet—they're part of the Offshore fund. So right now Onshore + Offshore = Total, but adding HPF muddies the picture a bit.

[shawnbot]

Update for you @gemfarmer: I've moved all of the database import and site data generation scripts to the new project-level Makefile. The data/Makefile still exists, but will slowly be deprecated in favor of the new database approach. To make the database you now just run:

make db

from the project root. You can then update the site's Jekyll data (the new stuff in _data) with:

make site-data

There are a couple of things that are annoying about Makefiles:

1. "Some makes" don't support leading underscores in targets, so I've worked around this by making fake targets for the YAML outputs without their leading underscore, e.g. this one.
2. Because each line in make is treated as a separate shell, we have to escape line breaks in multi-line calls. This makes longer inline SQL queries kind of ugly, and a little tricky to debug when you forget them. There might be a better way to do this, but I kind of like just having all of the queries in one file.

[shawnbot]

@gemfarmer I'm particularly interested in whether if, after you run the above commands, there are any git diffs in the _data directory. Hopefully there aren't!

[gemfarmer]

@shawnbot didn't see this. Will run the commands and get back to you.

[gemfarmer]

@shawnbot looks great except I'm running into this error: /bin/sh: ./node_modules/.bin/tito: No such file or directory which is preventing all the tables from being added.

[gemfarmer]

Did you add nodemodules/.bin to your PATH?

[gemfarmer]

@shawnbot just took another stab at this. I originally installed the dev dependencies with node v4.2.6 instead of v5.0.0, and for whatever reason it didn't add tito to the node_modules/.bin directory. I tried again using v5.0.0 and all the correct dependencies were added and the commands run perfectly.

Here is a sample from my diff git diff origin/state-pages _data. Basically the only difference I saw was slight adjustment to the yml syntax. Is that what you were hoping?

[gemfarmer]

Closing in favor of #1651