R4R: generalize query response with height #4573

colin-axner · 2019-06-17T22:22:05Z

Addition to #4536, no longer specific to account queries. Allows for validator endpoints to return height in the response. closes #4609

Targeted PR against correct branch (see CONTRIBUTING.md)
Linked to github-issue with discussion and accepted design OR link to spec that describes this work.
Wrote tests
Updated relevant documentation (docs/)
Added a relevant changelog entry: clog add [section] [stanza] [message]
rereviewed Files changed in the github PR explorer

For Admin Use:

Added appropriate labels to PR (ex. wip, ready-for-review, docs)
Reviewers Assigned
Squashed all commits, uses message "Merge pull request #XYZ: [title]" (coding standards)

codecov · 2019-06-17T22:47:36Z

Codecov Report

Merging #4573 into master will increase coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #4573      +/-   ##
==========================================
+ Coverage   52.65%   52.66%   +0.01%     
==========================================
  Files         261      261              
  Lines       16410    16410              
==========================================
+ Hits         8640     8642       +2     
+ Misses       7124     7122       -2     
  Partials      646      646

codecov · 2019-06-17T22:47:37Z

Codecov Report

Merging #4573 into master will increase coverage by 0.1%.
The diff coverage is 64%.

@@            Coverage Diff            @@
##           master    #4573     +/-   ##
=========================================
+ Coverage   53.62%   53.72%   +0.1%     
=========================================
  Files         260      260             
  Lines       16177    16197     +20     
=========================================
+ Hits         8675     8702     +27     
+ Misses       6856     6846     -10     
- Partials      646      649      +3

.pending/improvements/sdk/4573-adds-height-in-

types/rest/rest.go

alessio

I'd appreciate some test cases

types/rest/rest.go

alexanderbez

This is API breaking for all endpoints that will do this. Is this intended? If so, we must make sure swagger is also updated.

x/distribution/types/delegator.go

alexanderbez

Tested. There is nothing that actually sets the height @colin-axner.

e.g.

	res, _, err := ar.querier.QueryWithData(fmt.Sprintf("custom/%s/%s", QuerierRoute, QueryAccount), bs)
	if err != nil {
		return nil, err
	}

Where do we set the height when returning the result?

colin-axner · 2019-06-24T22:38:09Z

@alexanderbez ohh yea... There are a few other places this happens as well where a helper function does the data retrieval and just returns the result/error. Seems to me, if you want height to be returned in a response, then it it needs to be passed all the way down to the client. This would mean changing GetAccount to return height, as well as any other data retrieval functions to always return result, height, error. Should I update it to this format?

alexanderbez · 2019-06-24T23:21:37Z

@colin-axner not sure I completely follow, but this is what needs to happen I think:

Remove the recently added AccountWithHeight custom type and logic (essentially revert the PR)
Update all relevant query REST handlers as follows:

res, _, err := cliCtx.QueryWithData(route, bz)
if err != nil {
    rest.WriteErrorResponse(w, http.StatusInternalServerError, err.Error())
    return
}

rest.PostProcessResponse(w, cliCtx, res)

Now becomes:

res, height, err := cliCtx.QueryWithData(route, bz)
if err != nil {
    rest.WriteErrorResponse(w, http.StatusInternalServerError, err.Error())
    return
}

cliCtx = cliCtx.WithHeight(height)
rest.PostProcessResponse(w, cliCtx, res)

I know (2) will be a huge PIA, but this is how we get the responses to have the height.

Does this make sense?

colin-axner · 2019-06-24T23:44:32Z

@alexanderbez Both those have been done, I think the issue is here where you have:

account, err := accGetter.GetAccount(addr)
if err != nil {
	rest.WriteErrorResponse(w, http.StatusInternalServerError, err.Error())
	return
}

rest.PostProcessResponse(w, cliCtx, account)

because GetAccount() doesn't return height and therefore (2) can't be done to set the height in context

alexanderbez · 2019-06-25T12:50:04Z

@colin-axner phenomenal, just checked and you're correct -- my mistake!

Wrt to the account query, I guess this is a special case huh? No biggie, let's update AccountRetriever to also include a GetAccountWithHeight method.

colin-axner · 2019-06-25T19:08:04Z

@alexanderbez updated, the only issue is not all of the gov endpoints will return the query height. I can't find a work around for querying with tags without rewriting a lot of code. If these endpoints query for a specific height then the height will be included in the response. If they are querying using the latest height then the height won't be included because it is not set.

Specifically in QueryByTxsTags can't return height because:

resTxs, err := node.TxSearch(query, prove, page, limit)
if err != nil {
	return nil, err
}

does not return height.

Furthermore, a lot of the gov endpoint code is intermixed with searching by tags like the following:

if !(propStatus == types.StatusVotingPeriod || propStatus == types.StatusDepositPeriod) {
	res, err = gcutils.QueryDepositsByTxQuery(cliCtx, params) // calls QueryByTxsTags
} else {
	res, _, err = cliCtx.QueryWithData("custom/gov/deposits", bz)
}

fedekunze · 2019-06-25T21:28:00Z

@colin-axner yeah, we need to refactor those queries

fedekunze

LGTM

types/rest/rest_test.go

Co-Authored-By: Federico Kunze <[email protected]>

alexanderbez

ACK -- great job @colin-axner!

wrt to the specific governance queries, there is nothing we can do. Those specific type of queries were requested by the community and are "special" because they're ad-hoc and do tag (event) based queries underneath the hood. By definition the results can all have various heights. This PR is good as-is 👍

fedekunze · 2019-06-28T20:55:14Z

is this PR blocked?

alexanderbez · 2019-06-30T20:13:12Z

Nope, just need another ACK from @alessio

alessio

Something major needs fixing

alessio · 2019-06-30T22:46:39Z

types/rest/rest_test.go

 	"github.com/cosmos/cosmos-sdk/types"
+	authtypes "github.com/cosmos/cosmos-sdk/x/auth/types"


Top level packages must not depend on x sub packages.

Offf, good catch -- I missed this!

We can probably remove this test all together @colin-axner and move it to the LCD tests in Gaia.

We don’t need LCD tests for queries as those are covered by the contract tests

Makes sense, should I remove and move the tests elsewhere or should I refactor the test to use a mock version of account?

I like the latter

alexanderbez · 2019-07-01T13:00:58Z

types/rest/rest_test.go

 	"github.com/cosmos/cosmos-sdk/types"
+	authtypes "github.com/cosmos/cosmos-sdk/x/auth/types"


Offf, good catch -- I missed this!

alexanderbez · 2019-07-01T13:02:25Z

types/rest/rest_test.go

 	"github.com/cosmos/cosmos-sdk/types"
+	authtypes "github.com/cosmos/cosmos-sdk/x/auth/types"


We can probably remove this test all together @colin-axner and move it to the LCD tests in Gaia.

…ner/cosmos-sdk into colin/generalize-height-queries

alexanderbez

ACK

alessio

Nice one!

colin-axner added 2 commits June 17, 2019 15:16

add height in validator endpoint response

60e0267

generalize response with height

bb1a498

colin-axner requested review from alessio, alexanderbez, jackzampolin and rigelrozanski as code owners June 17, 2019 22:22

pending log

3c0faea

fedekunze reviewed Jun 18, 2019

View reviewed changes

.pending/improvements/sdk/4573-adds-height-in- Outdated Show resolved Hide resolved

alessio reviewed Jun 18, 2019

View reviewed changes

types/rest/rest.go Outdated Show resolved Hide resolved

alessio suggested changes Jun 18, 2019

View reviewed changes

alexanderbez reviewed Jun 18, 2019

View reviewed changes

types/rest/rest.go Outdated Show resolved Hide resolved

alexanderbez suggested changes Jun 18, 2019

View reviewed changes

alexanderbez added T: API Breaking Breaking changes that impact APIs and the SDK only (not state machine). ready-for-review labels Jun 18, 2019

beginning part of test for rest

975073f

alexanderbez added the S:blocked Status: Blocked label Jun 20, 2019

updated postprocessresponse to encode in height, added wip test

62fcc39

colin-axner closed this Jun 21, 2019

colin-axner mentioned this pull request Jun 21, 2019

Returning height in JSON Response #4609

Closed

4 tasks

colin-axner added 5 commits June 21, 2019 11:34

added passing postprocessresponse test

8aacfa5

merge and updated post process response call to include height

fd4e7f3

update delegator json tag

3a61eb7

revert indenting to two spaces

cc115ef

update pending log

ca2618c

colin-axner reopened this Jun 21, 2019

fix function comment

73c8f81

colin-axner commented Jun 21, 2019

View reviewed changes

x/distribution/types/delegator.go Show resolved Hide resolved

rm unused fmt import

5a845b2

height set in ctx instead of passed in func

1619f90

alexanderbez suggested changes Jun 24, 2019

View reviewed changes

add GetAccountWithHeight

2d4e65a

colin-axner added 2 commits June 25, 2019 12:12

fix merge conflict

4f86411

fix build errors

8d06eb0

fedekunze approved these changes Jun 25, 2019

View reviewed changes

types/rest/rest_test.go Outdated Show resolved Hide resolved

misspelling

5809c5e

Co-Authored-By: Federico Kunze <[email protected]>

alexanderbez approved these changes Jun 26, 2019

View reviewed changes

alexanderbez requested a review from alessio June 26, 2019 12:21

rigelrozanski mentioned this pull request Jun 26, 2019

CI Bot: Autopost a Changelog Preview #4627

Closed

colin-axner mentioned this pull request Jun 28, 2019

supply querier #4592

Merged

5 tasks

alexanderbez removed the S:blocked Status: Blocked label Jun 30, 2019

alessio suggested changes Jun 30, 2019

View reviewed changes

alexanderbez suggested changes Jul 1, 2019

View reviewed changes

colin-axner added 2 commits July 1, 2019 09:36

address @alessio pr comments; remove x/auth dependency

18cbfbd

Merge branch 'colin/generalize-height-queries' of github.com:colin-ax…

99b2c65

…ner/cosmos-sdk into colin/generalize-height-queries

alexanderbez requested a review from alessio July 1, 2019 16:43

alexanderbez approved these changes Jul 1, 2019

View reviewed changes

alessio approved these changes Jul 1, 2019

View reviewed changes

alessio merged commit 8d8fd9d into cosmos:master Jul 1, 2019

alexanderbez mentioned this pull request Jul 24, 2019

Correctly Returning Height in Queries #4773

Closed

4 tasks

colin-axner mentioned this pull request Aug 1, 2019

Add cli commands for the coinswap module. #4719

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

R4R: generalize query response with height #4573

R4R: generalize query response with height #4573

colin-axner commented Jun 17, 2019 •

edited

Loading

codecov bot commented Jun 17, 2019

codecov bot commented Jun 17, 2019 •

edited

Loading

alessio left a comment

alexanderbez left a comment

alexanderbez left a comment

colin-axner commented Jun 24, 2019

alexanderbez commented Jun 24, 2019 •

edited

Loading

colin-axner commented Jun 24, 2019

alexanderbez commented Jun 25, 2019

colin-axner commented Jun 25, 2019 •

edited

Loading

fedekunze commented Jun 25, 2019

fedekunze left a comment

alexanderbez left a comment

fedekunze commented Jun 28, 2019

alexanderbez commented Jun 30, 2019

alessio left a comment

alessio Jun 30, 2019

alexanderbez Jul 1, 2019

alexanderbez Jul 1, 2019

fedekunze Jul 1, 2019

colin-axner Jul 1, 2019 •

edited

Loading

alessio Jul 1, 2019

alexanderbez Jul 1, 2019

alexanderbez Jul 1, 2019

alexanderbez left a comment

alessio left a comment

		"github.com/cosmos/cosmos-sdk/types"
		authtypes "github.com/cosmos/cosmos-sdk/x/auth/types"

R4R: generalize query response with height #4573

R4R: generalize query response with height #4573

Conversation

colin-axner commented Jun 17, 2019 • edited Loading

codecov bot commented Jun 17, 2019

Codecov Report

codecov bot commented Jun 17, 2019 • edited Loading

Codecov Report

alessio left a comment

Choose a reason for hiding this comment

alexanderbez left a comment

Choose a reason for hiding this comment

alexanderbez left a comment

Choose a reason for hiding this comment

colin-axner commented Jun 24, 2019

alexanderbez commented Jun 24, 2019 • edited Loading

colin-axner commented Jun 24, 2019

alexanderbez commented Jun 25, 2019

colin-axner commented Jun 25, 2019 • edited Loading

fedekunze commented Jun 25, 2019

fedekunze left a comment

Choose a reason for hiding this comment

alexanderbez left a comment

Choose a reason for hiding this comment

fedekunze commented Jun 28, 2019

alexanderbez commented Jun 30, 2019

alessio left a comment

Choose a reason for hiding this comment

alessio Jun 30, 2019

Choose a reason for hiding this comment

alexanderbez Jul 1, 2019

Choose a reason for hiding this comment

alexanderbez Jul 1, 2019

Choose a reason for hiding this comment

fedekunze Jul 1, 2019

Choose a reason for hiding this comment

colin-axner Jul 1, 2019 • edited Loading

Choose a reason for hiding this comment

alessio Jul 1, 2019

Choose a reason for hiding this comment

alexanderbez Jul 1, 2019

Choose a reason for hiding this comment

alexanderbez Jul 1, 2019

Choose a reason for hiding this comment

alexanderbez left a comment

Choose a reason for hiding this comment

alessio left a comment

Choose a reason for hiding this comment

colin-axner commented Jun 17, 2019 •

edited

Loading

codecov bot commented Jun 17, 2019 •

edited

Loading

alexanderbez commented Jun 24, 2019 •

edited

Loading

colin-axner commented Jun 25, 2019 •

edited

Loading

colin-axner Jul 1, 2019 •

edited

Loading