Skip to content
This repository has been archived by the owner on Jul 12, 2023. It is now read-only.

allow for max records to be overridden on a per-export-config basis #1215

Merged
merged 1 commit into from
Dec 7, 2020

Conversation

mikehelmick
Copy link
Contributor

@mikehelmick mikehelmick commented Dec 7, 2020

Fixes #1214

Proposed Changes

  • Allow exportconfig entries to override max records

Release Note

Allow for individual exportconfigs to have variable sizes for the max records setting.

@google-cla google-cla bot added the cla: yes Auto: added by CLA bot when all committers have signed a CLA. label Dec 7, 2020
@google-oss-robot google-oss-robot added approved Auto: added by prow when enough reviewers approve. size/L Auto: large number of changes. labels Dec 7, 2020
Copy link
Member

@sethvargo sethvargo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

logger.Infof("Processing export batch %d (root: %q, region: %s), max records per file %d", eb.BatchID, eb.FilenameRoot, eb.OutputRegion, s.config.MaxRecords)
maxRecords := eb.EffectiveMaxRecords(s.config.MaxRecords)
if maxRecords != s.config.MaxRecords {
logger.Debugw("max records is being overridden", "batchID", eb.BatchID)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe log exp vs configured?

@google-oss-robot google-oss-robot added the lgtm Auto: added by prown with a reviewer LGTMs label Dec 7, 2020
@google-oss-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mikehelmick, sethvargo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [mikehelmick,sethvargo]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-oss-robot google-oss-robot merged commit 29b041d into google:main Dec 7, 2020
@mikehelmick mikehelmick deleted the issue1214 branch December 7, 2020 23:36
ThruDate string `form:"thrudate"`
ThruTime string `form:"thrutime"`
SigInfoIDs []int64 `form:"siginfo"`
MaxRecordsOverride int `form:"MaxRecordsOverride"`
}
Copy link
Contributor

@whaught whaught Dec 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's too late for this, but weird how we've mixed lowercase and CamelCase. I tend to prefer case-insensitive kebab or snake case if it's coming from web.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can change this pretty easily - it's an internal API

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 9, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
approved Auto: added by prow when enough reviewers approve. cla: yes Auto: added by CLA bot when all committers have signed a CLA. lgtm Auto: added by prown with a reviewer LGTMs size/L Auto: large number of changes.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow max records on export to be configurable per-exportconfig
4 participants