feat: Write Metrics to BigQuery

[tristanvuong2021]

feat: Write Metrics to BigQuery

[wfa-reviewable]

This change is 

[kungfucraig]

Reviewed 2 of 14 files at r13, 1 of 6 files at r17.
Reviewable status: 24 of 36 files reviewed, 3 unresolved discussions (waiting on @SanjayVas, @stevenwarejones, and @tristanvuong2021)

---

src/main/kotlin/org/wfanet/measurement/kingdom/deploy/gcloud/spanner/readers/MeasurementReader.kt line 403 at r17 (raw file):

        .trimIndent()

    private val FULL_VIEW_SQL =

As part of testing, can we look at the query plans for the various queries we run?

[tristanvuong2021]

Reviewable status: 23 of 36 files reviewed, 3 unresolved discussions (waiting on @kungfucraig, @SanjayVas, and @stevenwarejones)

---

src/main/kotlin/org/wfanet/measurement/kingdom/deploy/gcloud/spanner/readers/MeasurementReader.kt line 403 at r17 (raw file):

Previously, kungfucraig (Craig Wright) wrote…

As part of testing, can we look at the query plans for the various queries we run?

I did look at it. That is why I made query changes in StreamMeasurements.kt.

[tristanvuong2021]

Dismissed @SanjayVas and @kungfucraig from 2 discussions.
Reviewable status: 23 of 36 files reviewed, 1 unresolved discussion (waiting on @kungfucraig, @SanjayVas, and @stevenwarejones)

[tristanvuong2021]

Reviewed 1 of 22 files at r1, 4 of 21 files at r2, 5 of 16 files at r4, 1 of 4 files at r5, 4 of 10 files at r10, 1 of 5 files at r12, 3 of 14 files at r13, 1 of 3 files at r14, 5 of 8 files at r16, 5 of 6 files at r17, 2 of 5 files at r18, 2 of 3 files at r19, all commit messages.
Reviewable status: 35 of 36 files reviewed, 1 unresolved discussion (waiting on @kungfucraig, @SanjayVas, and @stevenwarejones)

[tristanvuong2021]

Reviewed 1 of 3 files at r19, 4 of 4 files at r20, all commit messages.
Dismissed @kungfucraig from a discussion.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @SanjayVas)

[tristanvuong2021]

Dismissed @SanjayVas from a discussion.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @SanjayVas)

[tristanvuong2021]

Dismissed @SanjayVas from a discussion.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @SanjayVas)

[SanjayVas]

Reviewed 1 of 3 files at r14, 8 of 8 files at r16, 5 of 6 files at r17, 2 of 5 files at r18, 3 of 3 files at r19, 4 of 4 files at r20, all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @tristanvuong2021)

---

src/main/kotlin/org/wfanet/measurement/kingdom/deploy/gcloud/spanner/readers/MeasurementReader.kt line 403 at r17 (raw file):

        .trimIndent()

    private val FULL_VIEW_SQL =

I thought COMPUTATIONS and FULL give the same result, meaning they use the same query aside from ordering (which isn't part of this constant).

---

src/main/proto/wfa/measurement/internal/kingdom/bigquerytables/operational_metrics_dataset.proto line 58 at r5 (raw file):

}

message ComputationParticipantsTableRow {

Doesn't this need to get added back in? DuchyMeasurmentLogEntries includes non-error entries with the stages.

[tristanvuong2021]

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @SanjayVas)

---

src/main/kotlin/org/wfanet/measurement/kingdom/deploy/gcloud/spanner/readers/MeasurementReader.kt line 403 at r17 (raw file):

Previously, SanjayVas (Sanjay Vasandani) wrote…

I thought COMPUTATIONS and FULL give the same result, meaning they use the same query aside from ordering (which isn't part of this constant).

The query is the same aside from ordering, but the building of the result isn't. Although it is only a small difference right now, I will need to include all the log entries. The result of the computation view only includes errors.

---

src/main/proto/wfa/measurement/internal/kingdom/bigquerytables/operational_metrics_dataset.proto line 58 at r5 (raw file):

Previously, SanjayVas (Sanjay Vasandani) wrote…

Doesn't this need to get added back in? DuchyMeasurmentLogEntries includes non-error entries with the stages.

I was planning to follow up this PR with a new table. This PR has been alive for awhile and I can see some new discussions regarding the new table.

[SanjayVas]

Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @tristanvuong2021)