Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DRAFT] Create new getstatus API method #4502

Closed
wants to merge 20 commits into from

Conversation

ghubstan
Copy link
Contributor

@ghubstan ghubstan commented Sep 8, 2020

This PR will never leave DRAFT status, and will be replaced by a clean PR.

A daemon will return an int32 = 1 (the pong) to the CLI if successful, then an api service is available message will be displayed in the CLI's console. If a valid pong is not returned, an api service is not available message is displayed in the console.

TODO: Experiments will be made to see if this implementation can be used to check server availabilty for other requests. This server ping implementation checks for a non-null wallet balance before returning a pong. If this ping impl's wallet check throws an exception to the calling API method, the calling method should throw a gRPC exception to the CLI instead of completing the operation, and the user should see an "api service is not available" message. No other API methods call ping with this change; experiments will be done later, giving other devs time to consider the idea first.

The premise is that if the wallet balance is available, so is the server, else the server is not avaiable. This may be a simple solution to the 'p2p not ready' problem affecting daemons.

A ping test was added to apitest/scripts/mainnet-test.sh.

This cannot be merged, but the code is safely backed up and the concept could be reviewed by other devs.

This PR adds a new CoreDisputeAgentService for handling dispute agent
registration in arbitrator daemons.  API methods for dispute agent
registration in mainnet mode are out of scope of this PR;  these
changes are for supporting the auto-registration of a mediator and
refund agent in regtest/dao mode for apitest cases.

Some minor, unrelated changes are included.

 * Inject new CoreDisputeAgentService into CoreApi.

 * Change a DisputeAgentManager 'dispute agent saved' log statement
   to indicate type of agent being saved.

 * Add import static bisq.common.app.Version.VERSION to CoreApi, and
   add @SuppressWarnings("SameReturnValue") annotation to the getVersion
   method.

 * Fix comment in Scaffold class.
Commit 40dd1a9 @injects a new CoreDisputeAgentService into CoreApi, but
it was unused by CoreApi, resulting in a legit codacy issue: "Perhaps
'coreDisputeAgentService' could be replaced by a local variable."

This change adds two pass through methods to CoreApi that use
the new CoreDisputeAgentService.  It should fix the codacy problem.
These new pass through methods are not useful yet, and may be
removed in a future PR for registering dispute agents with the API,
but CoreDisputeAgentService is needed now for apitest cases.

The CoreDisputeAgentService instance adds a listener to the
P2PService that tells it when the p2p network is bootstrapped (upon
receiving an onDataReceived event), then it automatically registers
dispute agents (if the daemon is an arbitrator node running in
regtest/dao mode).
Also moved GetVersion proto def to bottom of file (for alphabetic ordering).
This method cannot be used on mainnet, it is to support auto
registration of mediators and refund agents in api test cases.
It also does not support registration of test arbitrators -- not
needed for api's support for simple trading functionality.
This method cannot be used on mainnet, and is intended for developer
use only.

Some formatting was done to shorten line lengths.
Api test cases can now make api calls to arbitrator, bob & alice
daemons, not just alice's daemon.
This reverts commit c5639e6,
which failed to resolve the file conflict.
The jfoenix upgrade in PR 4443 was reverted in PR 4463 due to
incompatibility with JDK 10.  This change avoids an accidental
re-upgrade during merge.
A daemon will return an int32 = 1 (pong) to the CLI if successful,
then an "api service is available" message will be displayed in the
CLI's console.  If a valid pong is not returned, an "api service is
not available" message is displayed in the console.

Experiments will be made to see if this implementation can be used
to check server availabilty for other requests.  This server ping
implementation checks for a non-null wallet balance before returning
the pong (1) response.  If the ping's wallet check throws an exception
to the calling API method, the calling API method should throw a gRPC
exception to the CLI instead of completing the operation, and the user
should see an "api service is not available" message.

The premise is that if the wallet balance is available, so is
the server, else the server is not avaiable.  This may be a simple
solution to the 'p2p not ready' problem affecting daemons.

A ping test was added to apitest/scripts/mainnet-test.sh.
@ghubstan ghubstan changed the title [WIP] Create new ping API method [DRAFT] Create new ping API method Sep 8, 2020
@chimp1984
Copy link
Contributor

This may be a simple solution to the 'p2p not ready' problem affecting daemons.

I doubt that this is a feasible approach. There are several state in the p2p and wallet which need be treated differently depending on use case.
P2p:

  • getInitialDataResponse -> can show offers but is not listening to network as no hidden service published yet.
  • HS published -> ok not we are part of the network
  • UpdatedDataResonse -> process the maybe missed data in the duration between initiaal response and HS published

Wallet:

  • initialize -> wallet is set up
  • sufficient peers connected -> it is safe to broadcast txs
  • block download complete -> all synced, not needed to use the app but some txs might not show confirmed or we might miss balance if we received funds

So you see this complexity cannot be simulated with a single method. I would not suggest to use ping for more then simple connectivity tests.

@ghubstan
Copy link
Contributor Author

ghubstan commented Sep 9, 2020

@chimp1984,
@cbeams sent his thoughts about this problem some months ago in a keybase video before he handed this API project to me. I thought about it, put other work in front, and now trying to solve it before I finish out the "simplest trading script" impl. Chris wanted me to find a way to tell a user if a daemon was ready to perform all functions or no functions, and not show a CLI user anything more detailed than "server up" or "server not ready". (But details would be logged on the server.) And it needs to be used by some server methods (like create offer, take offer,...) so they know the request can be made or not.

So you see this complexity cannot be simulated with a single method. I would not suggest to use ping for more then simple connectivity tests.

The problem is the requirement to do precisely that -- to communicate the "ready=true/false" state with a single method. A simple gRPC connectivity check isn't useful for a user or some API methods that should throw an exception when "server not ready".

I thought a non-null balance was enough to determine if the server was ready or not, because a non-null balance means that allBasicServicesInitialized=true (see BisqSetup#initDomainServices)

As you said, it's not that simple; a non-null wallet balance is not enough to indicate the daemon is ready for trading.

What do you think of this solution for this non-trivial type of "ping"?

if(!bsqNode.isP2pNetworkReady())
	throw new IllegalStateException("p2p network not ready");

if(!p2PService.isBootstrapped())
	throw new IllegalStateException("p2p service not bootstrapped");

if(!daoStateService.isParseBlockChainComplete())
	throw new IllegalStateException("dao block chain not complete");

check for sufficient # of connect peers... a minimum of 4?

then the wallet balance check...

If these checks pass, ping returns a pong, else thows an exception.

This change adds a new CorePingService that does a series of checks
before returning a pong, indicating the server is ready to perform
trading functions.

	* Check bsqNode.isP2pNetworkReady == true
	* Check p2PService.isBootstrapped == true
	* Check daoStateService.isParseBlockChainComplete == true
	* Check p2PService.numConnectedPeers >= 4
	* Check walletsManager.areWalletsAvailable == true
	* Check balances.availableBalance != null

An exception with a reason msg is thrown if any check fails.  That
exception will be logged in the server, and the CLI user will
see an "api service is not available" message in the console.

A public isP2pNetworkReady() acessor method was added to BsqNode to
make the bsqNode.isP2pNetworkReady == true possible.
@chimp1984
Copy link
Contributor

@ghubstan
Those checks when all is ready are there. It just combines all the required substates. It can be added to your core API class and you simply request that. I would not mix that with ping/pong as that has another meaning (testing network availibility or keep connections alive).

The balance though might be a bit tricky. You can create an offer in Bisq UI with 0 BTC balance but need to fund it then in the create (or take) offer process. So in the UI it is not required to have balance as you also dont know in advance how much is required before the user defines the details of the offer (amount, sec. deposit).
To change too much from the way it works in the UI carries the risk that we get 2 apps to maintain. I would not recommend that, specially if there are good reasons why it is like that.

@ghubstan
Copy link
Contributor Author

ghubstan commented Sep 9, 2020

@chimp1984

RE: The balance though might be a bit tricky. You can create an offer in Bisq UI with 0 BTC balance but need to fund it then in the create (or take) offer process. So in the UI it is not required to have balance as you also dont know in advance how much is required before the user defines the details of the offer (amount, sec. deposit).

The balance check is for a non-null value. A 0 balance passes the test. When create & take offer is implemented, it will have to check funds for the offer, but that's not in the scope of this PR. (Those API methods don't exist yet.)

... It can be added to your core API class and you simply request that. I would not mix that with ping/pong as that has another meaning (testing network availibility or keep connections alive).

I agree ping is not the proper name for the method/meaning.

We still need to discuss this issue... I'm not sure you know the requirement (now) is to provide a single method for determining if the server is ready for any & all operations -- that all the criteria in the checks are met, or the server is not ready to take requests. There is no requirement (yet) for fine grained checks particular to a specific operation, excepting the existing checks in the CoreWalletsService. That is out of the scope of getting a "simplest trading script" ready to show other devs and users. I'm supposed to keep the demo'd API as simple as possible.

So, ping should be renamed, not exposed to the user, and only be used by other api methods to check readiness for the more complex operations. Would that be OK for you?

Ping was an incorrect name for the intent of this method, which is
to determine general network and wallet status, not the availability
of the gRPC server.

The new getstatus method does the following checks, returning true
if all pass, false if one or more of the checks fail:

* Check is p2p network ready, log warning if not
* Check is p2p network bootstrapped, log warning if not
* Check is block chain complete, log warning if not
* Check >= 4 peers are connected, log warning if not
* Check wallets are available, log warning if not
* Check balance is not null, log warning if not
@ghubstan ghubstan changed the title [DRAFT] Create new ping API method [DRAFT] Create new getstatus API method Sep 11, 2020
@ghubstan
Copy link
Contributor Author

Replaced by 4534.

@ghubstan ghubstan closed this Sep 17, 2020
@ghubstan ghubstan deleted the 3-ping branch September 19, 2020 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants