Skip to content

Commit

Permalink
Refine deploy target for better use of screen
Browse files Browse the repository at this point in the history
Problem: Prior to this change, it was necessary to first create and
attach to a screen session and then to run `make deploy` within it. This
meant extra steps for the user and was generally error-prone.

Solution: Usage of screen has been refined such that a screen session
named 'localnet' is created on the users behalf without any need to
attach to it. Individual node deployment targets such as `make
bitcoind`, `make alice`, et al. are issued to new windows within the
localnet screen session, and the user is free to attach or not whenever
they choose. The result is that a new user can clone the repository and
type nothing more than `make deploy` to get up and running with their
localnet.

This also reverts the changes in commit 97dd342 ("Make build target
phony") for the following reasons:

 - As mentioned in that commit message, Gradle was not deleting the its
   'build' directory when running `gradle clean`, meaning that the
   'build' target was always up-to-date, even after running `make
   clean`. This made it impossible to get a correct rebuild workflow. On
   analysis, howewer, this situation was because of a badly behaving
   Kotlin plugin not cleaning up after itself, leaving a subdirectory at
   build/kotlin and preventing the build directory itself from being
   deleted altogether. To address this, the `make clean` target has been
   updated to `rm -rf build` instead of calling `build gradle`. While
   it's a workaround until we back out the Kotlin changes that caused
   this, it does have the added benefit of being faster than invoking
   `gradle clean`.

 - By making the 'build' target PHONY, this meant that `./gradlew build`
   was getting invoked every time a dependent target was called. For
   example, `make alice` depends on the 'setup' target, which in turn
   depends on the 'build' target. When calling such targets in
   isolation, this arrangement works out fine, because the phony 'build'
   target always runs, invoking `./gradle build`, and the Gradle build
   completes quickly assuming everything is up-to-date. The problem
   arises when calling a number of these targets in rapid succession, as
   we do when calling `make deploy` and running each individual node
   target in its own screen window. This causes contention in two ways.
   The first is that these multiple, simultaneous Gradle processes
   compete for access to an available Gradle daemon, and because each
   process needs its own, it ends up that as many Gradle daemons get
   created as Bisq nodes we need to deploy (5 in total). This is a big
   waste of time and resources. The second way it causes not only
   contention but outright failure is that each of these builds are
   operating in the same directory, and while most aspects of the build
   are in fact up-to-date and therefore not modified in any way, there
   are exceptions to this rule. The result is that build artifacts, e.g.
   jars are getting deleted and rebuilt from underneath competing Gradle
   processes, and all manner of chaos ensues, such as NoClassDefFound
   errors and much more. This change (reverting 'build' back to a
   normal, non-phony target) avoids these problems entirely. When
   running `make deploy`, we run the 'build' target once as a function
   of the 'deploy' target depending on it. At this point, the 'build'
   directory exists, and all subsequent node deployment targets, e.g.
   'alice', 'bob', etc do not re-run the build target because it is
   up-to-date. For workflows where the user definitely wants to rebuild
   prior to redeploying a given node, they can either run `make
   clean-build`, or drop down to issuing Gradle build commands directly,
   e.g. `./gradlew :desktop:build` followed by `make desktop`.
  • Loading branch information
cbeams committed Dec 2, 2019
1 parent c347d9c commit 5fb4b21
Showing 1 changed file with 16 additions and 15 deletions.
31 changes: 16 additions & 15 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ setup: build .localnet
clean: clean-build clean-localnet

clean-build:
./gradlew clean
rm -rf build

clean-localnet:
rm -rf .localnet ./dao-setup
Expand Down Expand Up @@ -143,20 +143,21 @@ localnet: .localnet

# Deploy a complete localnet by running all required Bitcoin and Bisq
# nodes, each in their own named screen window. If you are not a screen
# user, you'll need to run each of the make commands manually in a
# separate terminal or as a background job.
#
# NOTE: You MUST already be attached to a screen session for the
# following commands to work properly.
# user, you'll need to manually run each of the targets listed below
# commands manually in a separate terminal or as background jobs.
deploy: setup
screen -t bitcoin make bitcoind
sleep 2 # wait for bitcoind rpc server to start
make block # generate a block to ensure Bisq nodes get dao-synced
screen -t seednode make seednode
screen -t seednode2 make seednode2
screen -t alice make alice
screen -t bob make bob
screen -t mediator make mediator
# create a new screen session named 'localnet'
screen -dmS localnet
# deploy each node in its own named screen window
targets=('bitcoind' 'seednode' 'seednode2' 'alice' 'bob' 'mediator'); \
for t in "$${targets[@]}"; do \
screen -S localnet -X screen -t $$t; \
screen -S localnet -p $$t -X stuff "make $$t\n"; \
done;
# give bitcoind rpc server time to start
sleep 5
# generate a block to ensure Bisq nodes get dao-synced
make block

bitcoind: .localnet
bitcoind \
Expand Down Expand Up @@ -242,4 +243,4 @@ block:
-rpcpassword=bsq \
generatetoaddress 1

.PHONY: build seednode
.PHONY: seednode

0 comments on commit 5fb4b21

Please sign in to comment.