Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Instance]: Portland, OR #79

Closed
1 task done
dphoria opened this issue Feb 21, 2022 · 12 comments
Closed
1 task done

[Instance]: Portland, OR #79

dphoria opened this issue Feb 21, 2022 · 12 comments
Assignees
Labels
new instance New instance deployment process

Comments

@dphoria
Copy link
Contributor

dphoria commented Feb 21, 2022

Municipality Name

Portland

Governing Body Type

city council

Maintainer GitHub Name

dphoria

Legistar Client Id

No response

Municipality Timezone

America/Los_Angeles

Municipality Slug

portland

Firestore Region

us-west1

Code of Conduct

  • I agree to follow this project's Code of Conduct
@dphoria dphoria added the new instance New instance deployment process label Feb 21, 2022
@github-actions
Copy link

github-actions bot commented Feb 21, 2022

Hello! 👋
Thanks for initiating the process to configure a new CDP instance.

I am a bot that will validate the information provided in your form. If any check fails, please update your issue by opening the '⋯' dropdown in the top-right-corner of your GitHub Issue and selecting 'Edit'. I will automatically rerun the checks after you update the issue to validate the changes.

A member from the CDP team will respond as soon as possible!

Form Validation Results

✅ Governing body type is an accepted value.
@dphoria has been marked as the instance maintainer.
CouncilDataProject/portland is available.
✅ An existing scraper for 'Portland' was found in cdp-scrapers (cdp_scrapers.instances.get_portland_events). If this scraper was selected incorrectly, please update the Municipality Slug field with more specificity (i.e. 'seattle-wa' instead of 'seattle').

✅ All checks successful 🎉

All Cookiecutter Parameters

{
 "municipality": "Portland",
 "governing_body_type": "city council",
 "municipality_slug": "portland",
 "python_municipality_slug": "portland",
 "infrastructure_slug": "cdp-portland-d2bbda97",
 "maintainer_or_org_full_name": "dphoria",
 "hosting_github_username_or_org": "CouncilDataProject",
 "hosting_github_repo_name": "portland",
 "hosting_github_url": "https://github.com/CouncilDataProject/portland",
 "hosting_web_app_address": "https://councildataproject.github.io/portland",
 "firestore_region": "us-west1"
}

Steps for Internal CDP Team

To proceed with the deployment process, please do the following:

  • Run make login in cdp-backend/dev-infrastructure and login to the CDP gcloud and pulumi accounts
  • Run make init project=cdp-portland-d2bbda97 in cdp-backend/dev-infrastructure
  • Comment "/cdp-deploy" on this issue and follow the rest of the instructions

More details on the make commands can be found in cdp-backend.

This comment was written by a bot!

@github-actions
Copy link

Generated Infrastructure Slug

cdp-portland-d2bbda97

This comment was written by a bot!

@evamaxfield
Copy link
Member

@dphoria will deploy tomorrow but just checking, what all is included in this scraper right now? Votes? Minutes items? (I wish we had the run-scraper bot on this repo too 😂)

@dphoria
Copy link
Contributor Author

dphoria commented Feb 21, 2022

https://gist.github.com/dphoria/42f133917ac7f474bdfcfc319e5b5a03

Beyond the required sessions, we get data like EventMinuteItem.decision, Matter.result_status, sponsors, supporting_files, votes. We also get minutes_item but that may be same as matter at the moment.

@evamaxfield
Copy link
Member

From the above JSON dump I think everything looks correct there except that the third session is from 2/17 not 2/16.

@dphoria
Copy link
Contributor Author

dphoria commented Feb 22, 2022

Yes that is actually an oft-observed pattern from Portland. The last session of a meeting often takes place on the second day. See https://www.portland.gov/council/agenda/all; many meetings span multiple days. I don't know why they do it this way. i.e. Why include the session from the "next" day on the same agenda?

Regardless, do you think it's better for CDP purposes, to treat that session as part of a different EventIngestionModel?

@evamaxfield
Copy link
Member

That is a perfect answer. Seems like they really do have multi-day meetings. Cool.

Who knows, but if that is what it says on the official agenda then that is what we should do as well.

Awesome. I will start the deployment process in ~1 hour

@CouncilDataProject CouncilDataProject deleted a comment from github-actions bot Feb 22, 2022
@CouncilDataProjectServiceAccount

/cdp-deploy

@github-actions
Copy link

github-actions bot commented Feb 22, 2022

Deployment Status

🎉 🎉 Repository Created 🎉 🎉

A new CouncilDataProject Instance Repository was created (CouncilDataProject/portland), external collaborator added (@dphoria), and cookiecutter files generated and pushed to repository.

The instance is setting itself up right now and the process will take around 10 minutes to complete. Once completed, a CDP maintainer will comment on this issue with your instance's website link. See the instance's GitHub Action job history for more details on the deployment setup progress.

Your CDP instance will be populated with data within 6 hours of website creation.

At any point in the future if you would like to destroy this instance, please just add a comment to this thread and a maintainer will help you.


Steps for Internal CDP Team

Final Setup
  • Run make gen-key project=cdp-portland-d2bbda97
  • Use the generated key as the repository secret for GOOGLE_CREDENTIALS
  • Rerun the infrastructure setup job, then:
    • Run make set-cors project=cdp-portland-d2bbda97
    • Enable [Google Speech-to-Text Data Logging](https://console.cloud.google.com/apis/api/speech.googleapis.com/data_logging?project={{ cookiecutter.infrastructure_slug }})
    • Navigate to Firebase Console, login to the CDP Google account, select the cdp-portland-d2bbda97 Firebase project
      • Navigate to "Firestore Database", select the "Rules" tab, paste the following in:
        rules_version = '2';
        service cloud.firestore {
          match /databases/{database}/documents {
            match /{document=**} {
              allow read;
            }
          }
        }
        
      • Navigate to "Storage", select the "Rules" tab, paste the following in:
        rules_version = '2';
        service firebase.storage {
          match /b/{bucket}/o {
            match /{allPaths=**} {
              allow read;
            }
          }
        }
        
  • Enable GitHub Pages
  • Comment on this issue with "Deployment Status - Complete" and the instance URL
Deletion Steps (Future Reference)
  • Delete the instance repository
  • Run make login and login to the CDP gcloud and pulumi accounts
  • Run make destroy project=cdp-portland-d2bbda97

More details on the make commands can be found in cdp-backend.

@CouncilDataProjectServiceAccount

Deployment Status - Complete

Instance available at: http://councildataproject.org/portland/#/
Running first data gather: https://github.com/CouncilDataProject/portland/runs/5283363985?check_suite_focus=true

@evamaxfield
Copy link
Member

@dphoria looks like this is all ready to go with the exception that this PR (CouncilDataProject/cdp-backend#169) needs to be merged into cdp-backend and a new patch released to allow portland to go through.

@dphoria
Copy link
Contributor Author

dphoria commented Feb 22, 2022

Thank you Jackson! We are almost there @Shak2000!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new instance New instance deployment process
Projects
None yet
Development

No branches or pull requests

3 participants