Bundle a chaincode-as-a-service external builder into the hyperledger/fabric Docker image

[jkneubuh]

The Fabric Operations Guide describes a process by which a Fabric network administrator may install an external builder to support the lifecycle of external chaincode service endpoints. This involves updating the peer's config file with the location of the external builder, and installing the build, detect, and release binaries on the peer file system.

The documentation guide provides a reference sample of external builder scripts (bash), but this path is difficult to realize in orchestrated clusters (e.g. Kubernetes.) A couple of pain points include:

• The sample bash scripts rely on jq for manipulating the metadata and connection json files, but jq is not available on the hyperledger/fabric-peer image.
• Deploying custom scripts and files into the peer image is possible using a combination of Kubernetes configmaps, init containers, or volume shares, but the process adds an additional, tedious configuration step for the network administrator.

The issues outlined above add just enough structural friction to delay or defer the migration to chaincode-as-a-service (CCaaS) deployments. Let's make this trivial for end-users to update to the new pattern by:

1. Including reference chaincode-as-a-service builder binaries into the hyperledger/fabric-peer Docker image.

or

2. Include a CCaaS build pipelines into the peer's Golang runtime.

With option (1), the network will enable the default CCaaS builder by modifying core.yaml, referencing a known (documented) location for the system builder. E.g.:

externalBuilders:
  - name: fabric-ccs-builder
    path: /usr/local
    propagateEnvironment:
      - HOME
      - CORE_PEER_ID
      - CORE_PEER_LOCALMSPID

The fabric-peer image, in turn, will need to be supplemented with fabric-ccs-builder (or equivalent) binaries on the docker image. In the configuration above, the image file system includes:

- usr
    +- local
        +- bin
            +- peer 
            +- build 
            +- detect 
            +- release 

In option (2) above, instead of distributing external binaries with the hyperledger/fabric-peer image, the peer binary will be supplemented with logic suitable for emulating the build, detect, release lifecycle without exec-ing a new process. With this pattern, the network administrator may register the default builder by specifying an attribute in the externalBuilders configuration stanza, e.g.:

  - name: fabric-ccs-builder
    type: ccsas
    propagateEnvironment:
      - HOME
      - CORE_PEER_ID
      - CORE_PEER_LOCALMSPID

With the latter configuration, when the peer detects an external builder with the ccaas type, the routines internally bypass the fork/exec, invoking native Golang routines to manipulate the chaincode descriptors.

Choose one of these approaches (or a third...) and bundle an external builder into the base fabric image. This will be a valuable time-saver and convenience for running chaincode-as-a-service in Fabric networks.

[mbwhite]

FYI - Just having a look at the Fabric codebase to get an idea of which would be easiest route here.

[mbwhite]

This is one possible way this could be done; draft only PR... #2990

It effectively builds the external builder and then packages as part of the peers docker image.
If you take just the peer binary then it knows nothing about the external chaincode.

[arsulegai]

@jkneubuh @mbwhite I've a slightly modified version of this PR. It accommodate filling up package parameters via the environment variables. Reason: CC_ID uniqueness for all the peers of an organization becomes critical at approval lifecycle step.

https://github.com/arsulegai/fabric-cc-external-builder

Let me know if it sounds good and I'm happy to work together on improving it further.

[mbwhite]

thanks @arsulegai - I'll take a look at the repo.

The other approach is to make the concept of 'external' and 'chaincode-as-a-service' implicitly understood by the peer. Though I'm undecided on which is the better approach.

[arsulegai]

I like that idea, the concept of making the peer implicitly understanding chaincode-as-a-service sounds good. The current model on external builder can be left as is, given also the docker discontinuity in Kubernetes. For a peer's chaincode-as-a-service, it can be a separate template configuration under core.yaml.

However, chaincode builder has to package a chaincode server. Builder parameters will have to be dynamically configured. Storing these parameters on-chain does not make sense because they are local to the node. A peer CLI pre-creating the overlay-package, which the peer can understand, peer node then proceeds to create a chaincode package as usual is another approach. But this needs to be carefully orchestrated.

[mbwhite]

@arsulegai the primary advantage of the implicit knowledge of the chaincode-as-a-service is that a 'plain' non-dockerised peer binary would work 'out-of-the-box' with chaincode-as-a-service.

An assumption that I've made, that forgive me, I don't think I've clearly explained - is that the builder code, for either approach has nothing smarter than the basics. It's job being to soley curate the information from the chaincode package to the peer. Nothing more than https://github.com/hyperledgendary/fabric-ccs-builder

This could of course get more complex as you note - the builder code itself could setup K8S deployments or trigger something else. However the peer doesn't want to get involved with this level of complexity. Hence the purpose of the external builders initially.

[mbwhite]

Reason: CC_ID uniqueness for all the peers of an organization becomes critical at approval lifecycle step.
This is a very good point. To recap - today, in one organization multiple peers can have the same chaincode package installed (with it's hash-created CC_ID). A single approval is made per organization, and each peer will then use the chaincode package to spin up the chaincode container. There's a 1:1 between peer and a chaincode instance.

However with the chaincode-as-a-server, the package is also fixed and unchanged between peers - but the peer might want to talk to a different instance of the chaincode?

The reason for multiple peers is permit HA/load balancing - so this potentially falls down with one chaincode instance running. Not to mention the potential loading on one instance.

We believe that within k8s this is solved using standard service patterns etc. use a service behind which there could be many instances.

IBP for example takes a slightly different route and augments the url of the chaincode service with the peer id.

---

To summarise though.. I'm strongly coming down in favour of NOT giving the peer any more knowledge of types of chain code. Rather stay with the configuration route - and as here, including a few basic external builders within the published docker container.

And increasing documentation for how to configure a peer to determine what builder to use.

[mbwhite]

I've updated the PR at #2990 - this two notable changes

1. Rather than using the environment variable override, modify the core.yaml directly
2. In connection.json that is packaged in the chaincode package, each string is now being treated as a golang template string. The context from this string is taken from EXTERNAL_CHAINCODE_BUILDER_CONFIG environment variable (formatted as JSON).

The reason for this later change is to give the ability to configure on a peer-to-peer basis the config.. eg different TLS certs, different Chaincode URLs.

[mbwhite]

Links to related issues/prs.

Fabric

• Implementation PR Adding chaincode-as-a-service external builder to peer's docker image #2990
• TODO - Read-The-Updates

Fabric-Samples updates

• Use the builder in the test-network-k8s Adding examples of CCAAS and support into the test-network-k8s fabric-samples#527
• Use the builder in the test-network-nano-bash Possible update to nano-bash network for external builders fabric-samples#507
• TODO: Update CC examples. BasicAssestTransfer as both, or BasicAssetTransfer-as-a-service style

Chaincode updates

• Update Java to include - the as-service-main https://github.com/hyperledger/fabric-chaincode-java/pulls
• TODO- Update GoLang to include - the as-service-main
• (Node already has this ability)