Skip to content

Project for ingest pypi packages data from BigQuery and send to DataDog for analysis and insights with dashboards, monitors and more

License

Notifications You must be signed in to change notification settings

ivdatahub/pypi-package-stats

Repository files navigation

Google Cloud BigQuery: BigQuery SQL Repository

Project Status License

BigQuery Executor

Project Summary

This project contains SQL scripts for creating and managing BigQuery tables, views, and other objects. It is intended to be used as a repository for storing and sharing SQL code across multiple projects.

Development Stack

My Skills

Cloud Stack (GCP)

BigQueryBigQuery

  • Cloud IAM: Identity and Access Management (IAM) service provided by GCP for managing user access to cloud resources.
  • BigQuery: Fully managed, serverless data warehouse provided by GCP for storing and analyzing large datasets.

Continuous Integration and Continuous Deployment (CI/CD, DevOps)

My Skills

Contributing

See the following docs:

Project Highlights:

  • Modular Architecture: Separation of SQL files and execution scripts for easier maintenance and updates.

  • Automated SQL Execution: GitHub Actions workflows to automatically execute SQL scripts on BigQuery based on defined triggers.

  • Configuration Management: Use of environment variables and configuration files for managing execution parameters and credentials.

  • CI/CD Pipelines: Automated build, test, and deployment processes using GitHub Actions to keep SQL scripts up-to-date.

  • Comprehensive Documentation: Detailed documentation including installation instructions, usage examples, and troubleshooting guides.

About

Project for ingest pypi packages data from BigQuery and send to DataDog for analysis and insights with dashboards, monitors and more

Topics

Resources

License

Code of conduct

Stars

Watchers

Forks

Sponsor this project