Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

3437 improve handling of slow archives #4090

Merged
merged 1 commit into from
Jan 3, 2024
Merged

3437 improve handling of slow archives #4090

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/metrics.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ history.check.success | meter | history archive status c
history.publish.failure | meter | published failed
history.publish.success | meter | published completed successfully
history.publish.time | timer | time to successfully publish history
history.get.throughput | meter | bytes per second of history archive retrieval
history.get.failure | meter | history archive downloads failed
ledger.age.closed | bucket | time between ledgers
ledger.age.current-seconds | counter | gap between last close ledger time and current time
ledger.apply.success | counter | count of successfully applied transactions
Expand Down
7 changes: 7 additions & 0 deletions src/historywork/GetRemoteFileWork.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
#include "history/HistoryArchiveManager.h"
#include "history/HistoryManager.h"
#include "main/Application.h"
#include "util/Fs.h"
#include "util/GlobalChecks.h"
#include "util/Logging.h"

Expand All @@ -22,6 +23,10 @@ GetRemoteFileWork::GetRemoteFileWork(Application& app,
, mRemote(remote)
, mLocal(local)
, mArchive(archive)
, mFailuresPerSecond(
app.getMetrics().NewMeter({"history", "get", "failure"}, "failure"))
, mBytesPerSecond(
app.getMetrics().NewMeter({"history", "get", "throughput"}, "bytes"))
{
}

Expand Down Expand Up @@ -52,13 +57,15 @@ void
GetRemoteFileWork::onSuccess()
{
releaseAssert(mCurrentArchive);
mBytesPerSecond.Mark(fs::size(mLocal));
RunCommandWork::onSuccess();
}

void
GetRemoteFileWork::onFailureRaise()
{
releaseAssert(mCurrentArchive);
mFailuresPerSecond.Mark(1);
CLOG_ERROR(History,
"Could not download file: archive {} maybe missing file {}",
mCurrentArchive->getName(), mRemote);
Expand Down
3 changes: 3 additions & 0 deletions src/historywork/GetRemoteFileWork.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
#pragma once

#include "historywork/RunCommandWork.h"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This include should not have changed.

#include "medida/medida.h"

namespace stellar
{
Expand All @@ -18,6 +19,8 @@ class GetRemoteFileWork : public RunCommandWork
std::shared_ptr<HistoryArchive> const mArchive;
std::shared_ptr<HistoryArchive> mCurrentArchive;
CommandInfo getCommand() override;
medida::Meter& mFailuresPerSecond;
medida::Meter& mBytesPerSecond;

public:
// Passing `nullptr` for the archive argument will cause the work to
Expand Down
2 changes: 1 addition & 1 deletion src/ledger/readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ behaves just like pointers in typical data structures but with added
security guarantees.

See the protocol file for the object definitions.
[`src/xdr/Stellar-ledger.x`](../xdr/Stellar-ledger.x)
[`src/protocol-curr/xdr/Stellar-ledger.x`](../protocol-curr/xdr/Stellar-ledger.x)
ThomasBrady marked this conversation as resolved.
Show resolved Hide resolved

One can think of the historical chain as a linked list of LedgerHeaders:

Expand Down