[scanner integration] Handle redirection

[harrislapiroff]

In the process of investigating something unrelated I noticed that one of our directory entries has a landing page URL https://organization.tld/securedrop/ which redirects to https://securedrop.organization.tld. The landing page scanner should detect a redirect like this and behave accordingly.

This might actually be a sort of complicated problem, since presently we extract the domain from the landing page URL as a substring and ask pshtt to scan it as step 1 and then do an HTML scrape of the actual URL as step 2. We may need to reverse those steps in order to follow a redirect and scan the correct domain. (This will also stop us from using pshtt's bulk scan API, which probably doesn't actually give us a performance boost anyway, since it requires the same number of HTTP requests.)

[eloquence]

Looks like a duplicate of #494.

[harrislapiroff]

Whoops, right you are. Though I have some questions, I'll add to that ticket.