Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

services/horizon/internal/actions: Add latest ingested ledger in experimental ingestion endpoints #1830

Merged
merged 2 commits into from
Oct 15, 2019

Conversation

tamirms
Copy link
Contributor

@tamirms tamirms commented Oct 11, 2019

Include a Latest-Ledger header with the sequence number of the last processed ledger by the
experimental ingestion system. Also, ensure the endpoint responses contains only data consistent
with the sequence number included in the header

PR Checklist

PR Structure

  • This PR has reasonably narrow scope (if not, break it down into smaller PRs).
  • This PR avoids mixing refactoring changes with feature changes (split into two PRs
    otherwise).
  • This PR's title starts with name of package that is most changed in the PR, ex.
    services/friendbot, or all or doc if the changes are broad or impact many
    packages.

Thoroughness

  • This PR adds tests for the most critical parts of the new functionality or fixes.
  • I've updated any docs (developer docs, .md
    files, etc... affected by this change). Take a look in the docs folder for a given service,
    like this one.

Release planning

  • I've updated the relevant CHANGELOG (here for Horizon) if
    needed with deprecations, added features, breaking changes, and DB schema changes.
  • I've decided if this PR requires a new major/minor version according to
    semver, or if it's mainly a patch change. The PR is targeted at the next
    release branch if it's not a patch change.

Summary

Add Latest-Ledger header with the sequence number of the last processed ledger by the experimental ingestion system. The endpoints built using the experimental ingestion system will always respond with data which is consistent with the ledger in Latest-Ledger.

Fixes #1791

Goal and scope

See #1791 for motivation

Copy link
Contributor

@bartekn bartekn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I think it's different for /order_book, the action code should probably overwrite the header to send ledger sequence from in-memory order book graph. Also, please add tests to check if header is sent from DB and order book endpoints.

services/horizon/internal/actions/session.go Outdated Show resolved Hide resolved
@tamirms tamirms force-pushed the latest-ledger-header branch 6 times, most recently from fc1bed8 to e9c73cf Compare October 11, 2019 17:12
Include a Latest-Ledger header with the sequence number of the last processed ledger by the
experimental ingestion system. Also, ensure the endpoint responses contains only data consistent
with the sequence number included in the header
@tamirms
Copy link
Contributor Author

tamirms commented Oct 11, 2019

@bartekn @abuiles this is ready for review now

Copy link
Contributor

@bartekn bartekn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@@ -60,6 +60,10 @@ func (tx *orderBookBatchedUpdates) apply() error {
}
tx.committed = true

if ledger < tx.orderbook.lastLedger {
return errOutdatedLedger
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably check: tx.orderbook.lastLedger != 0 && ledger != tx.orderbook.lastLedger + 1 to make sure no ledger is skipped.

GetResource(r *http.Request) (actions.StreamableObjectResponse, error)
GetResource(
r *http.Request,
w actions.HeaderWriter,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about flipping the order so it matches ServeHTTP?

@@ -28,7 +28,7 @@ type Path struct {
// Finder finds paths.
type Finder interface {
// Returns path for a Query of a maximum length `maxLength`
Find(q Query, maxLength uint) ([]Path, error)
Find(q Query, maxLength uint) ([]Path, uint32, error)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the comment to explain what returned uint32 is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants