wal: check out of range slice in "ReadAll", entry limits in "decodeRecord" #11793
Conversation
wal/decoder.go
Outdated
| @@ -79,6 +80,9 @@ func (d *decoder) decodeRecord(rec *walpb.Record) error { | |||
| } | |||
|
|
|||
| recBytes, padBytes := decodeFrameSize(l) | |||
| if recBytes >= math.MaxInt64-padBytes { | |||
There was a problem hiding this comment.
i think we have a max wal size, also a max key-value pair size. i feel these can be set as a better limit.
There was a problem hiding this comment.
i think we have a max wal size, also a max key-value pair size. i feel these can be set as a better limit.
Do we mean define one for max wal entry size? We have request size limit, but I don't see anywhere we define limit?
Lines 777 to 786 in 1166b1f
There was a problem hiding this comment.
we have a request limit. also we know that the throughput wont be unlimited. we probably will only be able to batch at most 1000 requests into one WAL entry at most, usually far less... So I guess a 2GB wal entry limit is large enough...
There was a problem hiding this comment.
Added a check for entry upper limit. Here I assume no project ever allows more than 10 MB raft message entry.
gyuho
changed the title
| @@ -412,7 +415,12 @@ func (w *WAL) ReadAll() (metadata []byte, state raftpb.HardState, ents []raftpb. | |||
| case entryType: | |||
| e := mustUnmarshalEntry(rec.Data) | |||
| if e.Index > w.start.Index { | |||
| ents = append(ents[:e.Index-w.start.Index-1], e) | |||
There was a problem hiding this comment.
can we simply check 0 <= e.Index-w.start.Index - 1 < len(ents)?
wal: add slice bound checks in decoder CHANGELOG-3.5: add wal slice bound check CHANGELOG-3.5: add "decodeRecord" Signed-off-by: Gyuho Lee <[email protected]>
Codecov Report
@@ Coverage Diff @@
## master #11793 +/- ##
==========================================
- Coverage 66.69% 66.00% -0.69%
==========================================
Files 403 403
Lines 36976 36981 +5
==========================================
- Hits 24660 24411 -249
- Misses 10822 11068 +246
- Partials 1494 1502 +8
Continue to review full report at Codecov.
|
|
lgtm |
…-upstream-release-3.4 Automated cherry pick of #11793
…-upstream-release-3.3 Automated cherry pick of #11793
Signed-off-by: Gyuho Lee [email protected]
Please read https://github.com/etcd-io/etcd/blob/master/CONTRIBUTING.md#contribution-flow.