Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QueryAPI Logging Features + UI/UX changes #388

Merged
merged 9 commits into from
Nov 16, 2023
Merged

Conversation

roshaans
Copy link
Contributor

@roshaans roshaans commented Nov 14, 2023

This PR addresses a few painpoints of logging and QueryAPI Indexer development.

  1. Logs can now be seen through the QueryAPI editor window via a Show Logs button.
    - No need to switch between Indexer Status & Editor window anymore saving time and frustration with the editor taking a few seconds to load.
    - Enables more powerful integrations with the editor for future PRs (e.g: adding blocks to debug directly from the indexer Logs table)

  2. Logs are easier to digest now and search now.
    - Logs are grouped by block_height so now the user sees all the messages belonging to a block's execution together.
    - Additionally the user can use the search functionality to search for any block_height where the function was executed to view its log in entirety.
    - Logs have relative dates now e.g (4 mins ago...)
    - BlockHeights can be clicked on to take you to the explorer link

  3. Simplifying QueryAPI links.
    - There is no need for &view=status/editor-window to be passed in anymore.
    - As long as selectedIndexerPath is defined, the user is sent to the editor window. If &view=status is passed the user is redirected to the indexer logs page within the editor window.
    - Note: I have a PR on queryapi/docs to reflect these changes: chore: change links to reflect changes on queryapi docs#1584

Notes: Instead of using GraphQL to fetch logs, we are using a feature of Hasura which restifies your graphql queries.
You can find details here: https://near-queryapi.api.pagoda.co/console/api/rest/list

Other small updates

  • If a invalid indexer path is chosen, the editor will show an error
  • QueryAPI is avalaible for users in read only mode even if they are not logged in on near.org

This PR does not add WS support. This will be done in a future PR.

image
image
queryapi logs testing

@roshaans roshaans requested a review from a team as a code owner November 14, 2023 15:10
</ButtonGroup>
</Col>
</Row>
{debugMode && heights.length > 0 && (
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a future PR will add feature to add blocks to debug mode directly from the indexer logs table

Copy link
Collaborator

@darunrs darunrs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are awesome changes! I think they address a lot of the friction points of the indexer logs. Just have two questions about the changes.


const processLogs = (data) => {
const logEntries = data.indexer_log_entries.map((row) => ({
block_height: row.block_height,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to confirm, if the indexer is stuck on a block and it replays the block again and again in that state, will the logs show multiple rows with the same block height? Or merge them all under one block height?

This parlays into my next question. If the block heights do get repeated, then is there any value in having timestamps for each log? Each block height would correspond to a single runner execution, so logs would be quite close to each other.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would merge all attempted execution logs of a block under one entry. So for block x, you could see multiple duplicate logs signifying that it had been attempted multiple times.

This is the reason why the timestamps exist. To be able to get a mental model of how long each block invocation took

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh I see. I think that's fine, as long as all the logs inside the block are still ordered by timestamp. Maybe we can take a look at existing blocked indexers and see what the experience looks like.

@roshaans roshaans merged commit 59cc68d into main Nov 16, 2023
2 checks passed
@roshaans roshaans deleted the logs-table-features branch November 16, 2023 03:43
@morgsmccauley morgsmccauley mentioned this pull request Nov 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants