Skip to content

kolonuk/sbcwaste.top

Repository files navigation

sbcwaste
What this does:

Using chromdp, (reasons become apparent), this interrogates the Swindon Borough Council residential waste collection,
and extracts the waste collectioon types, along with the next collection dates. There is no API for collections available
directly from the site, so HTML parsing it is.

There are a number of issues with the SBC site becuase it is very badly designed and uses IIS (probably?) ASP, wiht loads
of overkill JS and CSS.

Issues with the SBC site I countered:
1. The HTML is returned complete with the dates and no obvious queryable API, hence the HTML parsing.
2. The images of the waste types are only available after the javascript has been run, so the use of a compiled
   HTML/webpage is required, hence the use of chromedp rather than collyv2
3. The address lookup is required because, strangely, the address is just passed as a URL text parameter to the final
   HTML page, so, yes, you can have any address you like, or blank. Thats a bad design.
4. The output here is in one of two formats, JSON or iCalendar (ics). The format is controlled via the format section of
   the path parameter.
5. The ics RFC does not allow for the use of calendar entry icons, so the image is included as an attachment in case
   your calendar app does manage to somehow display it.

Usage:
1. Find your UPRN (Unique Property Reference) by going to https://maps.swindon.gov.uk/ and searching for your address.
   The UPRN is the number that shows up on the left hand side once you have done a search.
2. Use the UPRN in the URL path, e.g. http://localhost:8080/[UPRN]/[ics|json]
   You should be able to just plug the URL with the ics as the format into your calendar app. The JSON option is if
   you want to do something else with the data.
3. I will provide a public URL for this at somepoint, but for now you can run it locally. I aim to get a caching service
   in place to avoid hitting the SBC website too much - I think a day is enough.

Dev environment
Install gcloud cli: https://cloud.google.com/sdk/docs/install-sdk    (just follow the instructions for installation at this stage)
Whilst the install is progressing, now create a google cloud project: https://console.cloud.google.com/projectcreate
(You can see the main dashboard here: https://console.cloud.google.com/projectselector2/home/dashboard)
And then enable the Artifact Registy API (make sure the project is selected from the drop down at the top): https://console.cloud.google.com/apis/library/artifactregistry.googleapis.com

You can see the Cloud Run instance here: https://console.cloud.google.com/run
You can see the Artifactory repo here: https://console.cloud.google.com/artifacts


Deploy Github actions
1. Github actions workflow availabe in repo - Change params to suit. 
2. Run deploy-githubactions.sh - Change params and run, making sure that gcloud sdk is installed and authenticated first (see the second part of the setup script). Make sure you also create the secrets. The permissions the script creates are most likelu a little overkill, but I don't have the time (or patience!) to strip it down.
3. Once you push this to your own repo, the github actions will run, and it should deploy using your google cloud resources.