Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding chaincode-as-a-service external builder to peer's docker image #2990

Merged
merged 1 commit into from
Dec 9, 2021

Conversation

mbwhite
Copy link
Member

@mbwhite mbwhite commented Oct 19, 2021

Signed-off-by: Matthew B White [email protected]

Type of change

  • Bug fix
  • New feature
  • Improvement (improvement to code, performance, etc)
  • Test update
  • Documentation update

Description

Additional details

Related issues

Resolves #2884

@jt-nti
Copy link
Member

jt-nti commented Oct 20, 2021

It would be really really nice to get an out of the box external builder, although that's only part of the story... e.g. peer chaincode package would need to know about it. Including the external builder in the docker image seems like a good option but it would also be good to update the fabric-samples nano test network to demonstrate how to configure it outside docker. It might also be nice to support prebuilt/binary chaincode in the build. For example, the fabric integration tests include a builder for that -> https://github.com/hyperledger/fabric/tree/main/integration/externalbuilders

@mbwhite
Copy link
Member Author

mbwhite commented Oct 21, 2021

Candidate update to the fabric-samples to use these builders. hyperledger/fabric-samples#507

@mbwhite
Copy link
Member Author

mbwhite commented Oct 21, 2021

Reference for Running Chaincode-as-a-service

node.js
All the code is part of the node libraries (use >2.2.1). Chaincode-as-a-service can be started by, for example, adding these two lines to the package.json scripts section

"start:server-nontls": "set -x && fabric-chaincode-node server --chaincode-address=$CHAINCODE_SERVER_ADDRESS --chaincode-id=$CHAINCODE_ID",
"start:server": "set -x && fabric-chaincode-node server --chaincode-address=$CHAINCODE_SERVER_ADDRESS --chaincode-id=$CHAINCODE_ID --chaincode-tls-key-file=/hyperledger/privatekey.pem --chaincode-tls-client-cacert-file=/hyperledger/rootcert.pem --chaincode-tls-cert-file=/hyperledger/cert.pem"

java
For Java use the 2.3 or main branch libraries. A separate main class needs to be written and set as the main class. (this is instead of the org.hyperleder.fabric.contract.ContractRouter)

/*
 * SPDX-License-Identifier: Apache-2.0
 */

package org.example;

import java.io.IOException;
import org.hyperledger.fabric.contract.ContractRouter;
import org.hyperledger.fabric.shim.ChaincodeServer;
import org.hyperledger.fabric.shim.ChaincodeServerProperties;
import org.hyperledger.fabric.shim.NettyChaincodeServer;

public class ContractBootstrap {

    private static final String CHAINCODE_SERVER_ADDRESS = "CHAINCODE_SERVER_ADDRESS";
    private static final String CHAINCODE_ID = "CHAINCODE_ID";
    private static final String CORE_PEER_TLS_ENABLED = "CORE_PEER_TLS_ENABLED";
    private static final String CORE_PEER_TLS_ROOTCERT_FILE = "CORE_PEER_TLS_ROOTCERT_FILE";
    private static final String ENV_TLS_CLIENT_KEY_FILE = "CORE_TLS_CLIENT_KEY_FILE";
    private static final String ENV_TLS_CLIENT_CERT_FILE = "CORE_TLS_CLIENT_CERT_FILE";


    public static void main(String[] args) throws Exception {
        ChaincodeServerProperties chaincodeServerProperties = new ChaincodeServerProperties();

        final String chaincodeServerPort = System.getenv(CHAINCODE_SERVER_ADDRESS);
        if (chaincodeServerPort == null || chaincodeServerPort.isEmpty()) {
            throw new IOException("chaincode server port not defined in system env. for example 'CHAINCODE_SERVER_ADDRESS=0.0.0.0:9999'");
        }

        final int port = Integer.parseInt(chaincodeServerPort.split(":")[1]);
        chaincodeServerProperties.setPortChaincodeServer(port);

        final String coreChaincodeIdName = System.getenv(CHAINCODE_ID);
        if (coreChaincodeIdName == null || coreChaincodeIdName.isEmpty()) {
            throw new IOException("core peer address not defined in system env. for example 'CHAINCODE_ID=externalcc:06d1d324e858751d6eb4211885e9fd9ff74b62cb4ffda2242277fac95d467033'");
        }

        boolean tlsEnabled = Boolean.parseBoolean(System.getenv(CORE_PEER_TLS_ENABLED));
        if (tlsEnabled) {
            // String tlsClientRootCertPath = System.getenv(CORE_PEER_TLS_ROOTCERT_FILE);
            String tlsClientKeyFile = System.getenv(ENV_TLS_CLIENT_KEY_FILE);
            String tlsClientCertFile = System.getenv(ENV_TLS_CLIENT_CERT_FILE);

            // set values on the server properties
            chaincodeServerProperties.setTlsEnabled(true);
            chaincodeServerProperties.setKeyFile(tlsClientKeyFile);
            chaincodeServerProperties.setKeyCertChainFile(tlsClientCertFile);
        }

        ContractRouter contractRouter = new ContractRouter(new String[] {"-i", coreChaincodeIdName});
        ChaincodeServer chaincodeServer = new NettyChaincodeServer(contractRouter, chaincodeServerProperties);

        contractRouter.startRouterWithChaincodeServer(chaincodeServer);
    }

}

Note - this should be added to the java libraries properly

golang
Like Java, this requires a separate 'main' file to be written.. for example

/*
SPDX-License-Identifier: Apache-2.0
*/

package main

import (
	"encoding/base64"
	"encoding/json"
	"fmt"
	"io/ioutil"
	"log"
	"os"
	"strconv"

	"github.com/hyperledger/fabric-chaincode-go/shim"
	"github.com/hyperledger/fabric-contract-api-go/contractapi"
	"github.com/hyperledger/fabric-samples/asset-transfer-basic/chaincode-go/chaincode"
)

func main() {

	fmt.Printf("Starting chaincode\n")

	assetChaincode, err := contractapi.NewChaincode(&chaincode.SmartContract{})
	if err != nil {
		log.Panicf("Error creating asset-transfer-basic chaincode: %v", err)
	}
	fmt.Printf("Created Contract\n")

	tlsProps, err := getTlsProps()
	if err != nil {
		log.Panicf("Error creating getting TLS properties: %v", err)
	}
	fmt.Printf("Creating Server\n")

	server := &shim.ChaincodeServer{
		CCID:     os.Getenv("CHAINCODE_ID"),
		Address:  os.Getenv("CHAINCODE_SERVER_ADDRESS"),
		CC:       assetChaincode,
		TLSProps: tlsProps,
	}

	out, err := json.Marshal(server)
	if err != nil {
		panic(err)
	}
	fmt.Println(string(out))

	fmt.Printf("Starting server\n")
	err = server.Start()
	if err != nil {
		fmt.Printf("Error starting go-contract chaincode: %s", err)
	}

}

func getTlsProps() (shim.TLSProperties, error) {

	tlsEnabled, err := strconv.ParseBool(os.Getenv("CORE_PEER_TLS_ENABLED"))
	if err != nil {
		tlsEnabled = false
	}

	if !tlsEnabled {
		return shim.TLSProperties{Disabled: true}, nil
	}

	var key []byte
	path, set := os.LookupEnv("CORE_TLS_CLIENT_KEY_FILE")
	if set {
		key, err = ioutil.ReadFile(path)
		if err != nil {
			return shim.TLSProperties{}, fmt.Errorf("failed to read private key file: %s", err)
		}

	} else {
		data, err := ioutil.ReadFile(os.Getenv("CORE_TLS_CLIENT_KEY_PATH"))
		if err != nil {
			return shim.TLSProperties{}, fmt.Errorf("failed to read private key file: %s", err)
		}
		key, err = base64.StdEncoding.DecodeString(string(data))
		if err != nil {
			return shim.TLSProperties{}, fmt.Errorf("failed to decode private key file: %s", err)
		}
	}

	var cert []byte
	path, set = os.LookupEnv("CORE_TLS_CLIENT_CERT_FILE")
	if set {
		cert, err = ioutil.ReadFile(path)
		if err != nil {
			return shim.TLSProperties{}, fmt.Errorf("failed to read public key file: %s", err)
		}
	} else {
		data, err := ioutil.ReadFile(os.Getenv("CORE_TLS_CLIENT_CERT_PATH"))
		if err != nil {
			return shim.TLSProperties{}, fmt.Errorf("failed to read public key file: %s", err)
		}
		cert, err = base64.StdEncoding.DecodeString(string(data))
		if err != nil {
			return shim.TLSProperties{}, fmt.Errorf("failed to decode public key file: %s", err)
		}
	}

	root, err := ioutil.ReadFile(os.Getenv("CORE_PEER_TLS_ROOTCERT_FILE"))
	if err != nil {
		return shim.TLSProperties{}, fmt.Errorf("failed to read root cert file: %s", err)
	}

	return shim.TLSProperties{
		Disabled:      false,
		Key:           key,
		Cert:          cert,
		ClientCACerts: root,
	}, nil
}

@@ -0,0 +1,5 @@
module hyperledgendary/fabric-ext-builder/v2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrong module name ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah yes you're correct

@davidkel
Copy link
Contributor

davidkel commented Oct 22, 2021

I really think that this should support non chaincode-as-a-service as well to support existing users with existing chaincode implementations. I feel the initial benefit of this is the removal of dependencies on the internal Docker builder and launcher, plus there are an awful lot of existing chaincode implementations that would need to be changed over to chaincode as a service

@mbwhite mbwhite changed the title Possible way of adding external builders Possible way of adding chaincode-as-a-service external builder Oct 29, 2021
@mbwhite
Copy link
Member Author

mbwhite commented Oct 29, 2021

Sorry I'd misrepresented the scope of this suggestion in the title. this is only about supporting the issue #2884 - that is chaincode-as-a-service.

The aim is to provide, in the Fabric Peer as standard, an external builder for processing chaincode-as-a-service. rather than have to configure the peer specifically to add this.

@jt-nti @davidkel both very correct in your points. The ways an external builder (not using chaincode-as-a-service) are vast, though not directly called from the peer, code needs to be running in the same container as the peer to build/create the chaincode. This is now responsible for orchestrating the starting of the chaincode. It may no longer be docker, but it would need to be something.

This specific PR is exploring one of the ways that an external builder could be provided as standard - that is via including it preconfigured in the built docker image. This is a simple, and effective solution. It does not however mean a 'standalone' peer binary would know, though that could be configured to use the same if you wished.

jkneubuh
jkneubuh previously approved these changes Nov 5, 2021
Copy link
Contributor

@jkneubuh jkneubuh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like this approach. The compromise of installing a chaincode-as-a-service external builder into the peer image strikes the right balance of simplicity vs flexibility. Even with some rough edges around the docker build, it seems infinitely better than the alternative approach of altering the peer golang routines for cc lifecycle.

Using the new default builder, it was very easy to get up and running with external chaincode services running on Kubernetes. Matthew's (fabric-samples) PR 527 shows:

  1. An update to core.yaml to enable the builder:
    externalBuilders:
        - name: ccaas_builder
          propagateEnvironment:
            - EXTERNAL_CHAINCODE_BUILDER_CONFIG
          path: /opt/hyperledger/ccaas_builder
  1. yaml updates to launch the chaincode containers (compiled out of band)

  2. template substitution within connection.json

In contrast with the previous approach for registering an external chaincode builder, this is very, very straightforward. This PR reduces the config burden for switching to external chaincode-as-a-service significantly. I approve!

Copy link
Contributor

@SamYuan1990 SamYuan1990 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mbwhite , I added some comments here. please take a look at.

Makefile Outdated Show resolved Hide resolved
ccaas_builder/go.mod Show resolved Hide resolved
ccaas_builder/cmd/build/main.go Show resolved Hide resolved
ccaas_builder/cmd/detect/main.go Outdated Show resolved Hide resolved
@mbwhite mbwhite force-pushed the intergated-external-builder branch 2 times, most recently from e91c480 to 35c4301 Compare November 15, 2021 11:13
@mbwhite mbwhite changed the title Possible way of adding chaincode-as-a-service external builder Adding chaincode-as-a-service external builder to peer's docker image Nov 16, 2021
@mbwhite
Copy link
Member Author

mbwhite commented Nov 17, 2021

@hyperledger/fabric-core-maintainers this is ready for review.. thankyou.

@mbwhite mbwhite force-pushed the intergated-external-builder branch from 35c4301 to 89e557f Compare November 17, 2021 15:44
Copy link
Contributor

@jkneubuh jkneubuh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new approach works really well but there are a couple of issues in this PR:

  • ccass -> ccaas (to match target of metadata.json)

  • detect is detecting ccass but build/main.go is still checking for external

  • The default core.yaml should propagate the updated env var name CHAINCODE_AS_A_SERVICE_BUILDER_CONFIG

After making these updates in the core images it was smooth sailing.

ccaas_builder/cmd/build/main.go Outdated Show resolved Hide resolved
ccaas_builder/cmd/detect/main.go Outdated Show resolved Hide resolved
ccaas_builder/cmd/detect/main.go Outdated Show resolved Hide resolved
ccaas_builder/cmd/release/main.go Outdated Show resolved Hide resolved
images/peer/Dockerfile Outdated Show resolved Hide resolved
@mbwhite mbwhite force-pushed the intergated-external-builder branch from 89e557f to cb280eb Compare November 18, 2021 16:02
@mbwhite
Copy link
Member Author

mbwhite commented Nov 18, 2021

thank you @jkneubuh; I've grepped the code as well looking for the words external ccaas ccaass ccas etc make sure there's nothing hidden.. should be good now

@jkneubuh
Copy link
Contributor

👍

@mbwhite mbwhite force-pushed the intergated-external-builder branch 3 times, most recently from ab0e576 to 2d411a8 Compare December 1, 2021 09:13
Copy link
Contributor

@denyeart denyeart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking pretty good now, I'll go through the code in some more detail, but looks like it is structured well. Just a few nits for now...

# propagateEnvironment:
# - ENVVAR_NAME_TO_PROPAGATE_FROM_PEER
# - GOPROXY
externalBuilders:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Above, it would be worthwhile to comment about the new out-of-the-box ccaas, in addition to the default Docker builder.


FROM peer-base
ENV FABRIC_CFG_PATH /etc/hyperledger/fabric
VOLUME /etc/hyperledger/fabric
VOLUME /var/hyperledger
COPY --from=peer /go/src/github.com/hyperledger/fabric/build/bin /usr/local/bin
COPY --from=peer /go/src/github.com/hyperledger/fabric/sampleconfig/msp ${FABRIC_CFG_PATH}/msp
COPY --from=peer /go/src/github.com/hyperledger/fabric/sampleconfig/core.yaml ${FABRIC_CFG_PATH}
COPY --from=peer /go/src/github.com/hyperledger/fabric/sampleconfig/core.yaml ${FABRIC_CFG_PATH}/core.yaml
COPY --from=peer /go/src/github.com/hyperledger/fabric/release/linux-amd64/bin/ccaas_builder/bin/ /opt/hyperledger/ccaas_builder/bin/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you build the image locally and confirm it has the correct content and can be used?
e.g. make docker-clean docker

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have done; and will repeat the exercise before final merge

Copy link
Member

@jt-nti jt-nti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me although it seems a bit odd that ccaas_builder is a completely separate go module whereas there are existing commands under /cmd which are part of the top level module. What are the pros and cons of doing it each way?

@mbwhite
Copy link
Member Author

mbwhite commented Dec 2, 2021

@jt-nti yes a good question; I don't have the experience of knowledge of the build/codebase structure philosophy to be able to definitively respond. Will take the direction of the maintainers here, as to what is the most approraite.

Copy link
Contributor

@denyeart denyeart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed Detect first. Many of the comments there apply to Build and Release as well.

Comment on lines +354 to +356
cd ccaas_builder && go test -v ./cmd/detect && go build -o ../$(MARCH:%=release/%)/bin/ccaas_builder/bin/ ./cmd/detect/
cd ccaas_builder && go test -v ./cmd/build && go build -o ../$(MARCH:%=release/%)/bin/ccaas_builder/bin/ ./cmd/build/
cd ccaas_builder && go test -v ./cmd/release && go build -o ../$(MARCH:%=release/%)/bin/ccaas_builder/bin/ ./cmd/release/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need go test here.

Is there a reason you'd put the built executables in $(BUILD_DIR)/bin with the others?
e.g. /build/bin/ccaas_builder/bin

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it correct to assume that the make unit-test will find all the tests needed?

The path you provide in the core.yaml needs to point to a directory that contains a bin directory with the apps in it. Hence the ccaas_builder/bin but as to where exactly the source lives and the binaries built to. happy to take guidance.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: I ran the make unit-tests and it didn't pick up these tests automatically? Should it be updated to do this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would have expected all unit tests under /fabric to get picked up automatically.

"strings"
)

var logger = log.New(os.Stderr, "", 0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everywhere else in Fabric Go code we use github.com/hyperledger/fabric/common/flogging, e.g.
var logger = flogging.MustGetLogger("kvledger")
But since this doesn't pull in any Fabric dependencies, I guess it's ok to use the built in logger.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes -as you say these aren't part of say the Peer, Orderer etc. they are external applications; all the examples in the docs use bash; but they could be anything.

They could pull in the Fabric dependencies such as the flogging if you wish.

ccaas_builder/cmd/detect/main.go Outdated Show resolved Hide resolved
chaincodeMetaData := os.Args[2]
mdbytes, err := ioutil.ReadFile(filepath.Join(chaincodeMetaData, "metadata.json"))
if err != nil {
return err
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We typically add context to the error so the user knows what was going on instead of just returning the underlying Go error message. Typically errors.WithMessage() is used for this:
https://pkg.go.dev/github.com/pkg/errors#WithMessage

Do you want to check file existence like you do in build program?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding context is very useful thanks. that's for pointing out the withMessage. Will update across the board with this.

Though (a) if these files aren't there then the peer's probably gone wrong and (b) the peer doesn't really report much back to if one these programs fails..

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking of a scenario where the user forgot to include metadata.json in the chaincode package.

ccaas_builder/cmd/detect/main.go Outdated Show resolved Hide resolved
ccaas_builder/cmd/build/main.go Outdated Show resolved Hide resolved
ccaas_builder/cmd/build/main_test.go Outdated Show resolved Hide resolved
ccaas_builder/cmd/build/main_test.go Show resolved Hide resolved
fileMetadata.WriteString(`{
"type":"ccaas"
}`)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also add negative test for no metadata.json and no connection.json.

if err := json.Unmarshal(connectionFileContents, &connectionData); err != nil {
return err
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to check for required properties that may not have gotten set in the JSON? Or does the template do that?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added the error if missing key option to the template for exactly this case.

@mbwhite mbwhite force-pushed the intergated-external-builder branch from 2d411a8 to 1a26117 Compare December 6, 2021 11:59
@denyeart
Copy link
Contributor

denyeart commented Dec 6, 2021

Looks good to me although it seems a bit odd that ccaas_builder is a completely separate go module whereas there are existing commands under /cmd which are part of the top level module. What are the pros and cons of doing it each way?

Since these are completely standalone programs with no Fabric dependencies I think it is appropriate to keep them separate.

ccaas_builder/cmd/detect/main.go Show resolved Hide resolved
metadataFile := filepath.Join(chaincodeMetaData, "metadata.json")
_, err := os.Stat(metadataFile)
if err != nil {
return fmt.Errorf("%s not found ", metadataFile)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even for these we usually use errors.New().

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just been updating these, but now have VSCode warnings...

should use fmt.Errorf(...) instead of errors.New(fmt.Sprintf(...)) (S1028) go-staticcheck

ccaas_builder/cmd/build/main.go Show resolved Hide resolved
@jt-nti
Copy link
Member

jt-nti commented Dec 7, 2021

Looks good to me although it seems a bit odd that ccaas_builder is a completely separate go module whereas there are existing commands under /cmd which are part of the top level module. What are the pros and cons of doing it each way?

Since these are completely standalone programs with no Fabric dependencies I think it is appropriate to keep them separate.

So a follow on question is whether it would be better for the new external builder module to live in a separate repo. This PR would essentially convert the fabric repository into a monorepo. What are the implications of that in Go?

@denyeart
Copy link
Contributor

denyeart commented Dec 8, 2021

Looks good to me although it seems a bit odd that ccaas_builder is a completely separate go module whereas there are existing commands under /cmd which are part of the top level module. What are the pros and cons of doing it each way?

Since these are completely standalone programs with no Fabric dependencies I think it is appropriate to keep them separate.

So a follow on question is whether it would be better for the new external builder module to live in a separate repo. This PR would essentially convert the fabric repository into a monorepo. What are the implications of that in Go?

Since users wouldn't import the new programs, I don't see an impact to users.
As maintainers, we'd have to manage the new programs as part of the Fabric release in terms of fixes and release tagging.
It's really no different than delivering the other stand alone command line utilities as part of the Fabric release - it hasn't been an issue.
That being said, in an ideal world the new build programs would go into their own repository, and even the Fabric image build and publish could go into its own repository to keep the core Fabric repo slim and pure. The question is whether that purity is worth the cost of the additional CI machinery that would have to be maintained to tie the various repos together. I've been leaning towards keeping the precedent that everything the image depends on goes in a single repo to keep it simple, but it could be argued either way.

@mbwhite
Copy link
Member Author

mbwhite commented Dec 8, 2021

That being said, in an ideal world the new build programs would go into their own repository, and even the Fabric image build and publish could go into its own repository to keep the core Fabric repo slim and pure. The question is whether that purity is worth the cost of the additional CI machinery that would have to be maintained to tie the various repos together. I've been leaning towards keeping the precedent that everything the image depends on goes in a single repo to keep it simple, but it could be argued either way.

For the record, I completely agree with the desire to keep things slim/pure; Fabric is a now a product to be used in production environments, so it's consumability is of paramount importance. This PR is intended to make this easier, and make migration to a K8S environment significantly easier. (agree this is not being disputed).

However I don't think this PR is the place to refactor the repository; the question is it worth not having this consumeability improvement until such a refactor takes place?

Copy link
Member

@jt-nti jt-nti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the current precedent is to keep everything the image depends on in a single repo, it makes sense to get this PR in and discuss whether we want to revisit that strategy and refactor the repository later.

I would prefer a minor change to follow the precedent of the existing commands being part of the same Fabric module, rather than introducing a new separate module just for the builder, but also happy for the PR to go in as is if that's the consensus.

@needhamj
Copy link

needhamj commented Dec 9, 2021

Hi, I'm interested in this. So just to be clear, where does this leave external Java chaincode? Does this PR cover that? Is there any chance of a working image we can use on our project as it's pretty critical for us.

Copy link
Contributor

@denyeart denyeart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Matthew. This is a great leap in consumability for chaincode-as-a-service. It does muddy the waters of the Fabric repo a bit, but we can always move it out later if it becomes a problem or if we decide on another strategy for building and publishing the Fabric peer docker image.

@denyeart denyeart merged commit 4b39cb2 into hyperledger:main Dec 9, 2021
@denyeart
Copy link
Contributor

denyeart commented Dec 9, 2021

@Mergifyio backport release-2.4

@mergify
Copy link

mergify bot commented Dec 9, 2021

backport release-2.4

✅ Backports have been created

@denali49
Copy link
Contributor

denali49 commented Dec 9, 2021 via email

@mergify
Copy link

mergify bot commented Dec 9, 2021

backport release-2.4

✅ Backports have been created

@uqix
Copy link

uqix commented Apr 19, 2022

is it possible to use ccaas_builder in fabric v2.2? @mbwhite

@mbwhite
Copy link
Member Author

mbwhite commented Apr 19, 2022

@uqix yes; that should be possible. The binaries can be used as is, but you will need to edit the config.yaml used by the peer inline with the Fabric 2.4 version. So they pick up the required ccaas binaries.

@uqix
Copy link

uqix commented Apr 25, 2022

For v2.2, we needed to build custom peer image:

Dockerfile:

ARG fabric_version

FROM hyperledger/fabric-peer:2.4.3

FROM hyperledger/fabric-peer:$fabric_version
COPY --from=0 /opt/ /opt/
RUN sed -i 's/externalBuilders: \[\]/externalBuilders:\n        - name: ccaas_builder\n          path: \/opt\/hyperledger\/ccaas_builder/' /etc/hyperledger/fabric/core.yaml

Build:

dockerfile_version=1.0.1
fabric_version=2.2.2
image_name=uqix/hyperledger-fabric-peer:$dockerfile_version-$fabric_version

docker build \
    --build-arg fabric_version=$fabric_version \
    -t $image_name \
    . \
  && docker push $image_name

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bundle a chaincode-as-a-service external builder into the hyperledger/fabric Docker image
10 participants