Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(replication): fullsync phase write to sync on noop #3084

Merged
merged 3 commits into from
May 27, 2024
Merged

fix(replication): fullsync phase write to sync on noop #3084

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
80210d2
fix replication: fullsync phase write to sync on noop
adiholden May 26, 2024
6c68ae5
add comment
adiholden May 27, 2024
4494e2e
add comment
adiholden May 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions src/server/snapshot.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -330,12 +330,17 @@ void SliceSnapshot::OnDbChange(DbIndex db_index, const DbSlice::ChangeReq& req)
// no database switch can be performed between those two calls, because they are part of one
// transaction.
void SliceSnapshot::OnJournalEntry(const journal::JournalItem& item, bool await) {
// We ignore EXEC and NOOP entries because we they have no meaning during
// We ignore EXEC entries because we they have no meaning during
// the LOAD phase on replica.
if (item.opcode == journal::Op::NOOP || item.opcode == journal::Op::EXEC)
if (item.opcode == journal::Op::EXEC)
return;

serializer_->WriteJournalEntry(item.data);
// To enable journal flushing to sync after non auto journal command is executed we call
// TriggerJournalWriteToSink. This call uses the NOOP opcode with await=true. Since there is no
// additional journal change to serialize, it simply invokes PushSerializedToChannel.
if (item.opcode != journal::Op::NOOP) {
Copy link
Collaborator

@romange romange May 26, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, maybe we can comment here @adiholden ?

Does it mean we pass NOOP with await=true and the fix will allow the flow to reach PushSerializedToChannel(false);

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so maybe add a comment here that we allow TriggerJournalWriteToSink to flush the serialised chunk because it calls this function with (NOOP, await=true)

serializer_->WriteJournalEntry(item.data);
}

if (await) {
// This is the only place that flushes in streaming mode
Expand Down
9 changes: 6 additions & 3 deletions src/server/transaction.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1413,21 +1413,24 @@ void Transaction::LogAutoJournalOnShard(EngineShard* shard, RunnableResult resul
return;
}

// If autojournaling was disabled and not re-enabled, skip it
// If autojournaling was disabled and not re-enabled the callback is writing to journal.
// We do not allow preemption in callbacks and therefor the call to RecordJournal from
// from callbacks does not allow await.
// To make sure we flush the changes to sync we call TriggerJournalWriteToSink here.
if ((cid_->opt_mask() & CO::NO_AUTOJOURNAL) && !re_enabled_auto_journal_) {
TriggerJournalWriteToSink();
return;
}

// TODO: Handle complex commands like LMPOP correctly once they are implemented.
journal::Entry::Payload entry_payload;

string_view cmd{cid_->name()};
if (unique_shard_cnt_ == 1 || kv_args_.empty()) {
entry_payload = journal::Entry::Payload(cmd, full_args_);
} else {
entry_payload = journal::Entry::Payload(cmd, GetShardArgs(shard->shard_id()).AsSlice());
}
// Record to journal autojournal commands, here we allow await which anables writing to sync
// the journal change.
LogJournalOnShard(shard, std::move(entry_payload), unique_shard_cnt_, false, true);
}

Expand Down
Loading