Require tlog_id when log_id_ranges is passed in #739
Conversation
cmd/rekor-server/app/flags.go
Outdated
| } | ||
|
|
||
| // The last entry is special and should not have a terminating range, because this is active. | ||
| // It is specified by the tlog_id. | ||
| inputRanges = append(inputRanges, sharding.LogRange{ |
There was a problem hiding this comment.
Instead of appending the active ID, we can add a ActiveTreeID int field to the LogRangesFlag struct that'll make it easier to access it
There was a problem hiding this comment.
@priyawadhwa That's a good idea. This also implies we may have to move methods like func (l *LogRanges) ResolveVirtualIndex(index int) and func (l *LogRanges) TotalLength() int64 to be methods on logRangesFlag instead of LogRanges in order to work across all shards (active + inactive), does that seem reasonable?
There was a problem hiding this comment.
I've actually got a better way of doing it, I'm turning the LogRanges struct into this:
type LogRanges struct {
inactive []LogRange
active int64
}
That should be the simplest way, stay tuned!
cmd/rekor-server/app/serve.go
Outdated
| if rangeMap != "" { | ||
| if err := logRangeMap.Set(rangeMap); err != nil { | ||
| log.Logger.Fatal("unable to set logRangeMap from flag: %v", err) | ||
| if tlogID == 0 { |
There was a problem hiding this comment.
we don't need this check because we check for it later in logRangeMap.Set again
lkatalin
force-pushed
the
flags
branch
6 times, most recently
from
f3c0a78
to
0f31017
Compare
Codecov Report
@@ Coverage Diff @@
## main #739 +/- ##
=======================================
Coverage 47.25% 47.26%
=======================================
Files 66 66
Lines 5743 5755 +12
=======================================
+ Hits 2714 2720 +6
- Misses 2740 2746 +6
Partials 289 289
Continue to review full report at Codecov.
|
lkatalin
force-pushed
the
flags
branch
2 times, most recently
from
01c2213
to
1b5a745
Compare
cmd/rekor-server/app/root.go
Outdated
| @@ -63,7 +63,7 @@ func init() { | |||
|
|
|||
| rootCmd.PersistentFlags().String("trillian_log_server.address", "127.0.0.1", "Trillian log server address") | |||
| rootCmd.PersistentFlags().Uint16("trillian_log_server.port", 8090, "Trillian log server port") | |||
| rootCmd.PersistentFlags().Uint("trillian_log_server.tlog_id", 0, "Trillian tree id") | |||
| rootCmd.PersistentFlags().String("trillian_log_server.tlog_id", "0", "Trillian tree id") | |||
There was a problem hiding this comment.
sorry I wasn't clear when we chatted offline, I think this should actually remain a Uint for backwards compatibility.
There was a problem hiding this comment.
Aaahhh, got it. No worries, easy to fix.
pkg/sharding/log_index.go
Outdated
| } | ||
|
|
||
| var virtualIndex int64 | ||
| for _, r := range ranges.GetRanges() { | ||
| // TODO: Are the tree IDs guaranteed to be ordered deterministically in the Ranges? |
There was a problem hiding this comment.
yup, we can assume the user correctly set up the config so that the shards exist in order
There was a problem hiding this comment.
Perfect. Thank you!!
pkg/sharding/ranges.go
Outdated
| @@ -44,6 +45,9 @@ func NewLogRanges(path string, treeID string) (LogRanges, error) { | |||
| if err != nil { | |||
| return LogRanges{}, errors.Wrapf(err, "%s is not a valid int64", treeID) | |||
| } | |||
| if id == 0 { | |||
| return LogRanges{}, errors.New("non-zero tlog_id required when passing in shard config filepath") | |||
There was a problem hiding this comment.
We could add the fix to this error message to make it easier for users:
please set the active tree ID via the `--trillian_log_server.tlog_id` flag
pkg/sharding/ranges.go
Outdated
| } | ||
|
|
||
| func (l *LogRanges) NoInactive() bool { | ||
| return l.inactive == nil | ||
| } | ||
|
|
||
| func (l *LogRanges) Empty() bool { |
There was a problem hiding this comment.
i think this might be unused at this point and can be removed?
lkatalin
force-pushed
the
flags
branch
2 times, most recently
from
787903a
to
ab51bfa
Compare
tlog_id specifes the active shard and is kept for backwards compatibility. To avoid replicating information, the shard config file is used only to specify inactive shards and must be used in conjunction with a tlog_id flag. Together, these build the logRanges type in the sharding module. Signed-off-by: Lily Sturmann <[email protected]>
Summary
Based on the conversation @priyawadhwa in #727 , this separates concerns more cleanly between the
log_id_rangesflag andtlog_idflag.tlog_idnow specifies the active tree (shard) and can be used by itself.log_id_rangesspecifies inactive trees (shards) and must be used in conjunction withtlog_id.