v0.4.1-rc04

fixes #90

v0.4.1-rc03

fixes #126

v0.4.1-rc02

fixes #129

v0.4.1-rc01

fixes #129

v0.4.0

v0.4.0

Steps to update:

• Update ram package version in go.mod file, commit and push
• go build ram.go to compile the cli on this version
• ./ram -config to create settings for new services
• Review changes to revert or add changed setting files based on your environment need, commit and push
• Optionally update solution.yaml with new setting hosting/bigquery/views/intervalDays, see below for details
• ./ram -pipe -environment=dev -ramclisa=ramcli@<your_dev_pj_id>.iam.gserviceaccount.com -service=setlogmetrics to deploy this new service
• ./ram -check -pipe -environment=dev -ramclisa=ramcli@<your_dev_pj_id>.iam.gserviceaccount.com regular Cloud build triggers control
• git tag -l "ram-*-dev" to find what is your last tag version
• git tag -a ram-vA.B.C-dev -m "build: ram v0.4.4" where A,B,C depend on the result of the previous step
• git push --tags
• Wait for deployments in Cloud Build to complete
• ./ram -check -deploy -environment=dev -ramclisa=ramcli@<your_dev_pj_id>.iam.gserviceaccount.com regular deployment control
• Launch RAM Cloud Scheduler Jobs
• Check on the ram monitoring dashboard that the peak of cloud function activity is gone
• Check results in logs and reports

refactor

• refactor: getDisplayName
  • previously: returns "unknown" when not able to find an asset display name
  • now: returns the asset name with underscore replacing slash
• refactor: GCI assets the ancestry path and ancestors list
  • previously:
    • missing ancestry path for groupSettings
    • groupMember ancestry = directory id + group ID
  • now:
    • all GCI asset have a consistent ancestryPath or ancestors list as "directories/xxx", where xxx is the customer directory ID
• refactor: SQL view
  • previously: as GCI assets are not persisted in asset table the following fileds were NULL in the last_compliancestatus view: assetType, ancestryPath and level0 to 9
  • now: these filed are built by the view SQL code from the GCI asset name
• refactor: setdashboards
  • previously: widgets related to distribution metrics use line visualization
  • now: widgets related to distribution metrics use heatmap + percentile lines visualization

feat

• feat: solution.yaml has a new optional setting hosting/bigquery/views/intervalDays as a number of day to query in the tables. Aligning this setting the the frequency of the job starting the batch export of GCI assets, for example 7 days, enable to reduce the Bigquery cost in the same proportion: e.g. 7 instead of 365. Thanks to @MarcFundenberger for the idea.fixes #53
• feat: add projectID to the assets table and surface it in the related views fixes #54
• feat: splitdump default memory = 4GB to deal with large dumps, fixes #111
• feat: add a filed 'compliant' of type boolean in 'monitor' finish log entries fixes #109
• feat: new microserviceservice setlogmetrics create metrics form logs fixes #116
  • Four log based metrics:
    • ram_execution_status: count function execution good (finish...) vs bad (no retry)
    • ram_latency: elapsed time for a cloud function execution
    • ram_latency_e2d: elapsed time since the originating event (eg. asset change captured by CAI)
    • ram_trigger_age: elapsed time between the trggering event is published to Pubsub and the cloud function starts to run
  • All metrics have label by microservice, by instance and by environment.
  • log metric deployments are idempotent
• feat: add new monitoring dashboard for each microservice showing:
  • end to end latency heatmap and percentile line
  • latency heatmap and percentile line
  • trigger age heatmap and percentile line
  • pubsub oldest unacked message line
  • active instances line
  • execution count stacked bars
  • execution time heatmap and percentile line
  • memory usage heatmap and percentile line
• feat: log feed origin in finish messages and use it as a label in latency metrics

build

• build: cloud.google.com/go from 0.72.0 to 0.76.0
• build: bigquery v1.13.0 to v1.15.0
• build: firestore v1.3.0 to v1.4.0
• build: logging v1.1.2 to v1.2.0
• build: pubsub v1.8.3 to v1.9.1
• build: storage v1.12.0 to v1.13.0
• build: uuid v1.1.2 to v1.2.0
• build opa v0.24.0 to v0.26.0
• build: oauth2 v0.0.0-20210113205817-d3ed898aa8a3 to v0.0.0-20210201163806-010130855d6c
• build: api v0.38.0 to v0.39.0
• build: genproto v0.0.0-20210202153253-cf70463f6119 to v0.0.0-20210204154452-deb828366460

test

• test: unit test and integration test for setlogmetrics
• test: activate dependabot

v0.4.0-rc23

prepare release

v0.4.0-rc22

dependencies

v0.4.0-rc21

upd dependencies

v0.4.0-rc20

feat: log assetInventoryOrigin

v0.4.0-rc19

feat: log assetInventoryOrigin