Skip to content

Latest commit

 

History

History
2730 lines (1776 loc) · 185 KB

CHANGELOG.md

File metadata and controls

2730 lines (1776 loc) · 185 KB
Raw

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog and this project adheres to Semantic Versioning.

6.0.0-dev7

Changed

  • ccf::http::get_query_value() now supports bool types with "true" and "false" as values.
  • Service certificates and endorsements used for historical receipts now have a pathlen constraint of 1 instead of 0, reflecting the fact that there can be a single intermediate in endorsement chains. Historically the value had been 0, which happened to work because of a quirk in OpenSSL when Issuer and Subject match on an element in the chain.

Fixed

  • Services upgrading from 4.x to 5.x may accidentally change their service's subject name, resulting in cryptographic errors when verifying anything endorsed by the old subject name. The subject name field is now correctly populated and retained across joins, renewals, and disaster recoveries.

6.0.0-dev6

Added

  • Added a ccf::any_cert_auth_policy (C++), or any_cert (JS/TS), implementing TLS client certificate authentication, but without checking for the presence of the certificate in the governance user or member tables. This enables applications wanting to do so to perform user management in application space, using application tables (#6608).
  • Added OpenAPI support for std::unordered_set (#6634).
  • Added "cose_signatures" entry in the configuration, which allows setting "issuer" and "subject" at network start or recovery time (#6637).

6.0.0-dev5

Added

  • Updated ccf::cose::edit::set_unprotected_header() API, to allow removing the unprotected header altogether (#6607).
  • Updated ccf.cose.verify_receipt() to support checking the claim_digest against a reference value (#6607).

6.0.0-dev4

Added

Removed

  • Remove SECP256K1 support as a part of the migration to Azure Linux (#6592).

6.0.0-dev3

Changed

  • Set VMPL value when creating SNP attestations, and check VMPL value is in guest range when verifiying attestation, since recent updates allow host-initiated attestations (#6583).
  • Added ccf::cose::edit::set_unprotected_header() API, to allow easy injection of proofs in signatures, and of receipts in signed statements (#6586).

6.0.0-dev2

Added

  • Introduced ccf::describe_cose_endorsements_v1(receipt) for COSE-endorsements chain of previous service identities (#6500).
  • Ignore time when resolving did:x509 against x5chain, resolution establishes a point-in-time endorsement, not ongoing validity (#6575).

6.0.0-dev1

Changed

  • Output of ccf::describe_merkle_proof_v1(receipt) has been updated, and is now described by ccf-tree-alg schema.
  • Improved error message when attempting to obtain receipts for a past epoch during a recovery (#6507).

6.0.0-dev0

Changed

  • The set_jwt_issuer governance action has been updated, and no longer accepts key_filter or key_policy arguments (#6450).
  • Nodes started in Join mode will shut down if they receive an unrecoverable condition such as StartupSeqnoIsOld or InvalidQuote when attempting to join (#6471, #6489).
  • In configuration, attestation.snp_endorsements_servers can specify a max_retries_count. If the count has been exhausted without success for all configured servers, the node will shut down (#6478).
  • When deciding which nodes are allowed to join, only UVM roots of trust defined in public:ccf.gov.nodes.snp.uvm_endorsements are considered (#6489).

Removed

  • SGX Platform support.

Added

  • Provided API for getting COSE signatures and Merkle proofs (#6477).
    • Exposed COSE signature in historical API via TxReceiptImpl.
    • Introduced ccf::describe_merkle_proof_v1(receipt) for Merkle proof construction in CBOR format.
  • Added COSE signatures over the Merkle root to the KV (#6449).
    • Signing is done with service key (different from raw signatures, which remain unchanged and are still signed by the node key).
    • New signature reside in public:ccf.internal.cose_signatures.

5.0.4

Bug fix

  • JWT authentication correctly parses certificates that contain other certificates (#6440)

5.0.3

Changed

  • Improved JWT authentication error messages (#6427).

Bug fix

  • In GET gov/service/javascript-app, openApi now correctly returns the schema set for the endpoint (#6430)

5.0.2

Developer API

C++

  • RSAKeyPair::sign and RSAKeyPair::verify now use RSA-PSS instead of RSASSA-PKCS1-v1_5.
    • Users can specify salt_length (defaulted to 0).

TypeScript/JavaScript

  • ccfapp.crypto.sign() and ccfapp.crypto.verifySignature() no longer support RSASSA-PKCS1-v1_5, instead RSA-PSS has been added.
    • SigningAlgorithm has been extended with optional saltLength, defaulted to 0 if not passed.

Bug Fixes

  • The /tx endpoint returns more accurate error messages for incorrectly formed transactions ids (#6359).

5.0.1

Bug Fixes

  • All public headers now correctly set pragma once (#6388, #6389)

Dependencies

  • Base image refresh for containers (#6394, #6395)
  • Python cryptography package requirement raised to 43.* (#6385)

5.0.0

In order to upgrade an existing 4.x service to 5.x, CCF must be on the latest 4.x version (at least 4.0.19). For more information, see our documentation

Important

  • 5.0.0 is the last long term support release for Intel SGX. Live code upgrades to AMD SEV-SNP on Confidential Containers on Azure Container Instances and Confidential Containers Azure Kubernetes Service are supported.
  • The forwarding of requests between nodes is deprecated in favour of HTTP redirects, and will be removed in 6.0.0.
  • Recovery shares created with releases older than 4.0.9 are deprecated, and support will be removed completely in 6.0.0. Ledgers created on older versions must re-share at least once after having upgraded to a release newer than 4.0.9.
  • 5.0.0 introduces an Azure-compliant governance REST API, the old API is deprecated, and will be removed in 6.0.0.
  • Containers are now published to the GitHub Container Registry. The platform has moved from the tag to the image name, to enable meaningful usage of GitHub attestation, and the tag now matches the git tag used to cut the release. For example, the SGX Development container for this release is ghcr.io/microsoft/ccf/app/dev/sgx:ccf-5.0.0-rc0.

Developer API

C++

TypeScript/JavaScript Programmability
  • Reusable functionality allowing applications to expose TypeScript/JavaScript-implemented endpoints has been added to the public C++ API. Applications should subclass ccf::js::DynamicJSEndpointRegistry to get similar behaviour to the existing JS Generic app, see samples/app/programmability for samples usage.
  • Introduce DynamicJSEndpointRegistry::record_action_for_audit_v1 and DynamicJSEndpointRegistry::check_action_not_replayed_v1 to allow an application making use of the programmability feature to easily implement auditability, and protect users that make use of signature-based authentication to update the application against replay attacks (#6285).
  • The programmability sample app also demonstrates how applications can define their own extensions, create bindings between C++ and JS state, and allow JS endpoints to call functions implemented in C++.
Redirect
  • CCF now supports a mode where HTTP redirect responses are returned, rather than relying on internal forwarding. See docs for description of redirection behaviour and migration instructions.
  • Endpoints now support a ToBackup redirection strategy, for requests which should never be executed on a primary. These must also be read-only. These are configured similar to ToPrimary endpoints, with a to_backup object (specifying by-role or statically-addressed targets) in each node's configuration.
Other
  • Introduced ccf::historical::read_only_adapter_v4 and ccf::historical::read_write_adapter_v4. Users can now pass a custom error handler to the adapter to customise RPC responses for internal historical queries errors, which are listed in ccf::historical::HistoricalQueryErrorCode enum.
  • ccf::historical::adapter_v3 is deprecated in favour of _v4 version, ccf::historical::adapter_v2 is removed.
  • Public namespaces are all under ::ccf
    • ::ds is now ::ccf::ds
    • ::siphash is now ::ccf::siphash
    • ::threading is now ::ccf::threading, and ccf/ds/thread_ids.h has moved to ccf/threading/thread_ids.h
    • ::consensus is now ::ccf::consensus
    • ::tls is now ::ccf::tls
    • ::http is now ::ccf::http
    • ::nonstd is now ::ccf::nonstd
    • ::crypto is now ::ccf::crypto
    • ::kv is now ::ccf::kv
    • ::logger is now ::ccf::logger
    • ::ccfapp is now ::ccf
  • There is now a contains_globally_committed(k) method on kv::Set<K>, with the same semantics as get_globally_committed(k) on kv::Map<K, V> (#5928).
  • ccf::EnclaveAttestationProvider has been removed. It is replaced by ccf::AttestationProvider
  • JWT verifiers are now automatically cached, for increased performance (#5575)

TypeScript/JavaScript

  • Added support for reusing JS interpreters, persisting global state. See docs for more detail.
  • Added TypeScript TypedKvSet and ccfapp.typedKv<K> to facilitate set handling from application code.
  • Added ccf.SnpAttestation.verifySnpAttestation() for TypeScript apps. (#5653)
  • Removed unused openenclave.verifyOpenEnclaveEvidence API from JS/TS.
  • Added a ccfapp.checkedJson converter to the CCF TypeScript package, which will raise errors when given objects which cannot be roundtrip-converted through JSON (currently Map and Date). There is a slight cost to checking this on each instance during encode, so the behaviour is opt-in (not directly replacing ccfapp.json), but it is recommended that most tables update to use this converter.
  • JS endpoints marked as "mode": "readonly" are prevented from writing to the KV. Attempting to call map.set(k, v), map.delete(k), or map.clear() on any KV table in such an endpoint will now result in an error being thrown (#5921).
  • ccf.crypto.generateEddsaKeyPair, pubEddsaPemToJwk and eddsaPemToJwk now support x25519 as well as curve25519 (#5846).
  • ccf.crypto.unwrapKey() has been added to the JS API (#5792).
  • In governance contexts, JS runtimes now only use runtime limits from the public:ccf.gov.js_runtime_options map if they are strictly higher than the defaults (#5730).
  • Add HMAC support to JS API. Call with ccf.crypto.sign({"name": "HMAC", "hash": "SHA-256"}, key, data).

Authentication

  • Added token.iss claim validation to JWT authentication (#5809). Must-knows:
    • Supports both the OpenID requirements and the Entra specification of it.
    • All keys fetched after the upgrade will not work against tokens missing the iss claim if the issuer has been specified in the .well-known/openid-configuration/.
    • Due to an internal schema change, networks that are in the process of upgrading to this version may see inconsistent authorization behaviour while the network contains nodes of different versions (depending which node executes the auto-refresh, any nodes on the other version will not use any newly provided keys). We recommend a full upgrade to this version, removing any nodes on prior versions, followed by a key and issuer refresh.
    • A future release will remove the old tables entirely. Until then, some redundant state will be retained in the ledger. This is tracked in #6222.
  • Authentication policies can now be conjoined (AND) together, in addition to the previous disjoint (OR) behaviour. The new ccf::AllOfAuthnPolicy takes a collection of other policies, all of which must be true for this auth policy to pass. In JS, this can be configured in the app.json as "authn_policies": [{ "all_of": ["policy_a", "policy_b"] }].

Governance

  • proposalId is now passed to resolve(proposal, proposerId, votes, proposalId), allowing proposals to consider other pending proposals in their resolution process. (#5995)
  • The current state of an accepted proposal is written to the KV so that it can be accessed in the constitution's apply(proposal, proposalId) function (#6114).
  • POST /recovery/members/{memberId}:recover is now authenticated by COSE Sign1, making it consistent with the other POST endpoints in governance, and avoiding a potential denial of service where un-authenticated and un-authorised clients could submit invalid shares repeatedly. The submit_recovery_share.sh script has been amended accordingly, and now takes a --member-id-privk and --member-id-cert (#5821).
  • A new versioned governance API is now available, with the api-version=2024-07-01 query parameter. This will fully replace the previous governance endpoints, which will be removed in a future release. A guide to aid in upgrading from the previous API is available here
  • New endpoints GET /gov/service/javascript-modules and GET /gov/service/javascript-modules/{moduleName} to retrieve the raw JS code of the currently installed app. Note that the {moduleName} path parameter will need to be URL-encoded to escape any / characters (eg - /foo/bar.js should become %2Ffoo%2Fbar.js).

Operations

Configuration

  • The cchost configuration file now includes an idle_connection_timeout option. This controls how long the node will keep idle connections (for user TLS sessions) before automatically closing them. This may be set to null to restore the previous behaviour, where idle connections are never closed. By default connections will be closed after 60s of idle time.
  • A soft size limit can now be set for the historical store cache in the node configuration: historical_cache_soft_limit. The default value is 512Mb.
  • Path to the enclave file should now be passed as --enclave-file CLI argument to cchost, rather than enclave.file entry within configuration file.
  • SNP collateral must now be provided through the snp_security_policy_file, snp_uvm_endorsements_file and snp_endorsements_servers configuration values. See documentation for details and platform-specific configuration samples.
  • The url field in snp_endorsements_servers can now contain environment variables that will be resolved at startup, such as "$Fabric_NodeIPOrFQDN:2377" (#5862).
  • Add a new snp_security_policy_file configuration value under attestation, superseding the lookup from $UVM_SECURITY_CONTEXT_DIR. The value can contain environment variables, for example: "snp_security_policy_file": "$UVM_SECURITY_CONTEXT_DIR/security-policy-base64".
  • Add a new snp_uvm_endorsements_file configuration value under attestation, superseding the lookup from $UVM_SECURITY_CONTEXT_DIR. The value can contain environment variables, for example: "snp_uvm_endorsements_file": "$UVM_SECURITY_CONTEXT_DIR/reference-info-base64". This value can come from an untrusted location, like snp_security_policy_file and AMD endorsements (fetched from snp_endorsements_servers), because the CCF code contains pre-defined roots of trust.
  • snp_endorsements_servers now supports a THIM type, which is the recommended value when running in Confidential AKS preview.
  • Added a consensus.max_uncommitted_tx_count configuration option, which specifies the maximum number of transactions that can be pending on the primary. When that threshold is exceeded, a 503 Service Unavailable is temporarily returned on all but the /node/* paths (#5692).
Logging
  • Added 2 new log lines which may be helpful diagnostics in production deployments, both including a new [rollback] tag.
    • [rollback] ... Dropping conflicting branch may be emitted after network partitions, and indicates that some Pending (non-committed) transactions have been lost. This is expected, but worth investigating if it occurs regularly - it is a sign of elections impacting service availability.
    • [rollback] ... Ignoring conflicting AppendEntries could also be emitted after a network partition, but should be reported to the CCF development team. It is a sign of an unexpected execution path, which could lead to loss of liveness (inability to advance commit).
Endpoints
  • Added a GET /node/backup endpoint, returning 200 when backup and 404 when not, for load balancers to use (#5789).
  • Add /node/ready/app and /node/ready/gov endpoints for the use of load balancers wanting to check if a node is ready to accept application or governance transactions. See Operator RPC API for details.
Other
  • On retirement, nodes that are primary now request that their most likely successor triggers and instant election, without waiting for a timeout. This speeds up some reconfigurations, particularly code updates since they result in all the nodes being replaced. (#5697)

Client API

Removed

  • Removed the existing metrics endpoint and API (GET /api/metrics, get_metrics_v1). Stats for request execution can instead be gathered by overriding the EndpointRegistry::handle_event_request_completed() method.
  • Removed automatic msgpack support from JSON endpoint adapters, and related include/ccf/serdes.h file.
  • The scurl.sh script has been removed. With #5137 removing support for HTTP signed requests, it is no longer needed.

Dependencies

  • Updated Open Enclave to 0.19.7.
  • SNP attestation UVM endorsements SVNs are now treated as alphanumerically ordered strings.
  • Secret sharing used for ledger recovery now relies on a much simpler implementation that requires no external dependencies. Note that while the code still accepts shares generated by the old code for now, it only generates shares with the new implementation. As a result, a DR attempt that would downgrade the code to a version that pre-dates this change, after having previously picked it up, would not succeed if a reshare had already taken place (#5655).
  • SGX builds now use OpenSSL 3.1.1 inside the enclave by default (#5481).
  • CCF can now fetch SEV-SNP attestations from kernel 6.0 and above (#5848).
  • Added support for UVM endorsements signed with EC keys (#6231).

Bug Fixes

  • Improvements to the Raft implementation, to retain commit safety and liveness despite message loss (#6016).
  • Nodes are now more robust to unexpected traffic on node-to-node ports (#5889).
  • Lifted parser size limits on forwarded request from default values to more permissive ones. Note that the limits set out on the interface of the inbound node still apply (#5803).
  • Fixed an issue where a JS runtime limit could be hit out of user code execution, leading to an incorrectly constructed JS runtime or a crash (#5730).
  • Fix for JS execution behaviour when reusing interpreters. Storing KV handles on the global state may lead to unsafe accesses. Work around that by lazily requesting handles in the TypedKvMap for TypeScript apps.
  • Fix TLS bug that could cause TLS handshakes to fail (#5482)

5.0.0-rc2

Fixed

  • Restore inline implementation of two symbols, and keep a third symbol private (#6362)

5.0.0-rc1

Added

  • The cchost configuration file now includes an idle_connection_timeout option. This controls how long the node will keep idle connections (for user TLS sessions) before automatically closing them. This may be set to null to restore the previous behaviour, where idle connections are never closed. By default connections will be closed after 60s of idle time.
  • New endpoints GET /gov/service/javascript-modules and GET /gov/service/javascript-modules/{moduleName} to retrieve the raw JS code of the currently installed app. Note that the {moduleName} path parameter will need to be URL-encoded to escape any / characters (eg - /foo/bar.js should become %2Ffoo%2Fbar.js).
  • New gov API version 2024-07-01. This is near-identical to 2023-06-01-preview, but additionally offers the new javascript-modules endpoints.
  • Historical cache soft limit now is a node-specific startup parameter.

Changed

  • Set LTO on for both debug/release linkages to support linking against CCF libraries if the client code has been built in debug mode.

5.0.0-rc0

Added

  • More public namespaces have been moved under ::ccf
    • ::ds is now ccf::ds
    • ::siphash is now ccf::siphash
    • ::threading is now ccf::threading, and ccf/ds/thread_ids.h has moved to ccf/threading/thread_ids.h
    • ::consensus is now ccf::consensus
    • ::tls is now ccf::tls
    • ::http is now ccf::http
    • ::nonstd is now ccf::nonstd
    • ::crypto is now ccf::crypto
    • ::kv is now ccf::kv
    • ::logger is now ccf::logger
    • ::ccfapp is now ::ccf
  • The programmability sample app now demonstrates how applications can define their own extensions, creating bindings between C++ and JS state, and allowing JS endpoints to call functions implemented in C++.
  • Introduce DynamicJSEndpointRegistry::record_action_for_audit_v1 and DynamicJSEndpointRegistry::check_action_not_replayed_v1 to allow an application making use of the programmability feature to easily implement auditability, and protect users allowed to update the application against replay attacks (#6285).
  • Endpoints now support a ToBackup redirection strategy, for requests which should never be executed on a primary. These must also be read-only. These are configured similar to ToPrimary endpoints, with a to_backup object (specifying by-role or statically-addressed targets) in each node's configuration.
  • Introduced ccf::historical::read_only_adapter_v4 and ccf::historical::read_write_adapter_v4. Users are now capable of passing a custom error handler to the adapter to customise RPC responses for internal historical queries errors, which are listed in ccf::historical::HistoricalQueryErrorCode enum.

Changed

  • Updated Open Enclave to 0.19.7.

Deprecated

  • ccf::historical::adapter_v3 becomes deprecated in favour of _v4 version.

Removed

  • Removed the existing metrics endpoint and API (GET /api/metrics, get_metrics_v1). Stats for request execution can instead be gathered by overriding the EndpointRegistry::handle_event_request_completed() method.
  • Removed automatic msgpack support from JSON endpoint adapters, and related include/ccf/serdes.h file.

5.0.0-dev18

Added

  • Added TypeScript TypedKvSet and ccfapp.typedKv<K> to facilitate set handling from application code.
  • Added support for UVM endorsements signed with EC keys (#6231).
  • Updated Open Enclave to 0.19.6.

Removed

  • Removed unused openenclave.verifyOpenEnclaveEvidence API from JS/TS

Changed

  • Added token.iss claim validation to JWT authentication (#5809). Must-knows:
    • Supports both the OpenID requirements and the Entra specification of it.
    • All keys fetched after the upgrade will not work against tokens missing the 'iss' claim if the issuer has been specified in the .well-known/openid-configuration/.
    • Due to an internal schema change, networks that are in the process of upgrading to this version may see inconsistent authorization behaviour while the network contains nodes of different versions (depending which node executes the auto-refresh, any nodes on the other version will not use any newly provided keys). We recommend a full upgrade to this version, removing any nodes on prior versions, followed by a key and issuer refresh.
    • A future release will remove the old tables entirely. Until then, some redundant state will be retained in the ledger. This is tracked in #6222.

5.0.0-dev17

Added

  • Moved JS registry to public header ccf/js/registry.h. Apps should subclass ccf::js::DynamicJSEndpointRegistry to get similar behaviour to the existing JS Generic app.

5.0.0-dev16

Added

  • Reusable functionality for creating an in-enclave JS interpreter has been added to the public C++ API. Applications should subclass CustomJSEndpointRegistry to get similar behaviour to the existing JS Generic app.

5.0.0-dev15

Added

  • CCF now supports a mode where HTTP redirect responses are returned, rather than relying on internal forwarding. See docs for description of redirection behaviour and migration instructions.
  • Authentication policies can now be conjoined (AND) together, in addition to the previous disjoint (OR) behaviour. The new ccf::AllOfAuthnPolicy takes a collection of other policies, all of which must be true for this auth policy to pass. In JS, this can be configured in the app.json as "authn_policies": [{ "all_of": ["policy_a", "policy_b"] }].

Changed

  • proposalId is now passed to resolve(proposal, proposerId, votes, proposalId), allowing proposals to consider other pending proposals in their resolution process. (#5995)
  • The current state of an accepted proposal is written to the KV so that it can be accessed in the constitution's apply(proposal, proposalId) function (#6114).

5.0.0-dev14

Added

  • Added a ccfapp.checkedJson converter to the CCF TypeScript package, which will raise errors when given objects which cannot be roundtrip-converted through JSON (currently Map and Date). There is a slight cost to checking this on each instance during encode, so the behaviour is opt-in (not directly replacing ccfapp.json), but it is recommended that most tables update to use this converter.

Removed

  • The scurl.sh script has been removed. With #5137 removing support for HTTP signed requests, it is no longer needed.

5.0.0-dev13

Fixed

  • Improvements to the Raft implementation, to retain commit safety and liveness despite message loss (#6016).

Added

  • Added 2 new log lines which may be helpful diagnostics in production deployments, both including a new [rollback] tag.
    • [rollback] ... Dropping conflicting branch may be emitted after network partitions, and indicates that some Pending (non-committed) transactions have been lost. This is expected, but worth investigating if it occurs regularly - it is a sign of elections impacting service availability.
    • [rollback] ... Ignoring conflicting AppendEntries could also be emitted after a network partition, but should be reported to the CCF development team. It is a sign of an unexpected execution path, which could lead to loss of liveness (inability to advance commit).

5.0.0-dev12

Added

  • There is now a contains_globally_committed(k) method on kv::Set<K>, with the same semantics as get_globally_committed(k) on kv::Map<K, V> (#5928).

Changed

  • JS endpoints marked as "mode": "readonly" are prevented from writing to the KV. Attempting to call map.set(k, v), map.delete(k), or map.clear() on any KV table in such an endpoint will now result in an error being thrown (#5921).

Fixed

  • Nodes are now more robust to unexpected traffic on node-to-node ports (#5889).
  • Added a GET /node/backup endpoint, returning 200 when backup and 404 when not, for load balancers to use (#5789).

5.0.0-dev11

Removed

  • ccf::historical::adapter_v2 is removed, replaced by ccf::historical::adapter_v3 first introduced in 2.0.0.
  • ccf::EnclaveAttestationProvider has been removed. It is replaced by ccf::AttestationProvider
  • The attestation.environment.security_context_directory configuration entry and --snp-security-context-dir-var CLI option have been removed. SNP collateral must now be provided through the snp_security_policy_file, snp_uvm_endorsements_file and snp_endorsements_servers configuration values. See documentation for details and platform-specific configuration samples.

5.0.0-dev10

  • The url field in snp_endorsements_servers can now contain environment variables that will be resolved at startup, such as "$Fabric_NodeIPOrFQDN:2377" (#5862).
  • Add a new snp_security_policy_file configuration value under attestation, superseding the lookup from $UVM_SECURITY_CONTEXT_DIR. The value can contain environment variables, for example: "snp_security_policy_file": "$UVM_SECURITY_CONTEXT_DIR/security-policy-base64".
  • Add a new snp_uvm_endorsements_file configuration value under attestation, superseding the lookup from $UVM_SECURITY_CONTEXT_DIR. The value can contain environment variables, for example: "snp_uvm_endorsements_file": "$UVM_SECURITY_CONTEXT_DIR/reference-info-base64". This value can come from an untrusted location, like snp_security_policy_file and AMD endorsements (fetched from snp_endorsements_servers), because the CCF code contains pre-defined roots of trust.

5.0.0-dev9

  • snp_endorsements_servers now supports a THIM type, which is the recommended value when running in Confidential AKS preview.

5.0.0-dev8

  • ccf.crypto.generateEddsaKeyPair, pubEddsaPemToJwk and eddsaPemToJwk now support x25519 as well as curve25519 (#5846).
  • POST /recovery/members/{memberId}:recover is now authenticated by COSE Sign1, making it consistent with the other POST endpoints in governance, and avoiding a potential denial of service where un-authenticated and un-authorised clients could submit invalid shares repeatedly. The submit_recovery_share.sh script has been amended accordingly, and now takes a --member-id-privk and --member-id-cert (#5821).
  • CCF can now fetch SEV-SNP attestations from kernel 6.0 and above (#5848).

5.0.0-dev7

  • POST /recovery/members/{memberId}:recover is now authenticated by COSE Sign1, making it consistent with the other POST endpoints in governance, and avoiding a potential denial of service where un-authenticated and un-authorised clients could submit invalid shares repeatedly. The submit_recovery_share.sh script has been amended accordingly, and now takes a --member-id-privk and --member-id-cert (#5821).

5.0.0-dev6

  • Lifted parser size limits on forwarded request from default values to more permissive ones. Note that the limits set out on the interface of the inbound node still apply (#5803).
  • ccf.crypto.unwrapKey() has been added to the JS API (#5792).

5.0.0-dev5

  • In governance contexts, JS runtimes now only use runtime limits from the public:ccf.gov.js_runtime_options map if they are strictly higher than the defaults (#5730).
  • Fixed an issue where a JS runtime limit could be hit out of user code execution, leading to an incorrectly constructed JS runtime or a crash (#5730).
  • Added a GET /node/primary endpoint, returning 200 when primary and 404 when not, for load balancers to use (#5789).

5.0.0-dev4

  • Fix for JS execution behaviour when reusing interpreters. Storing KV handles on the global state may lead to unsafe accesses. Work around that by lazily requesting handles in the TypedKvMap for TypeScript apps.
  • On retirement, nodes that are primary now request that their most likely successor triggers and instant election, without waiting for a timeout. This speeds up some reconfigurations, particularly code updates since they result in all the nodes being replaced. (#5697)

5.0.0-dev3

  • Added a consensus.max_uncommitted_tx_count configuration option, which specifies the maximum number of transactions that can be pending on the primary. When that threshold is exceeded, a 503 Service Unavailable is temporarily returned on all but the /node/* paths (#5692).
  • A new versioned governance API is now available, with the api-version=2023-06-01-preview query parameter. This will fully replace the previous governance endpoints, which will be removed in a future release. A guide to aid in upgrading from the previous API is available here

5.0.0-dev2

  • Updated llhttp from 6.0.9 to 9.0.1.
  • Updated fmt library from 9.1.0 to 10.1.1.
  • Updated QCBOR from 1.1 to 1.2.
  • Updated nghttp2 from 1.51.0 to 1.55.1.
  • Converted SNP attestation UVM endorsements from integer to arbitrary string.
  • Updated Intel SGX PSW from 2.17 to 2.20 (#5616)
  • Path to the enclave file should now be passed as --enclave-file CLI argument to cchost, rather than enclave.file entry within configuration file. A potential SNP security context directory environment variable override, where desired, should now be passed as --snp-security-context-dir-var CLI argument to cchost, rather than attestation.environment.security_context_directory entry within configuration file. This is to ensure that these values are attested on Confidential Containers/SNP, even if the configuration itself is provided from un-attested storage, such as an external mount. The configuration entries are deprecated, and will be removed in a future release.
  • Added ccf.SnpAttestation.verifySnpAttestation() endpoint for TypeScript apps. (#5653)
  • Secret sharing used for ledger recovery now relies on a much simpler implementation that requires no external dependencies. Note that while the code still accepts shares generated by the old code for now, it only generates shares with the new implementation. As a result, a DR attempt that would downgrade the code to a version that pre-dates this change, after having previously picked it up, would not succeed if a reshare had already taken place (#5655).

5.0.0-dev1

  • Added support for reusing JS interpreters, persisting global state. See docs for more detail.

5.0.0-dev0

  • Add HMAC support to JS API. Call with ccf.crypto.sign({"name": "HMAC", "hash": "SHA-256"}, key, data).
  • Add /node/ready/app and /node/ready/gov endpoints for the use of load balancers wanting to check if a node is ready to accept application or governance transactions. See Operator RPC API for details.
  • SGX builds now use OpenSSL 3.1.1 inside the enclave by default (#5481).
  • JWT verifiers are now automatically cached, for increased performance (#5575).
  • GET /api/metrics now correctly returns templated endpoint paths (#5539).
  • Fix TLS bug that could cause TLS handshakes to fail (#5482).

4.0.7

  • Expose COSESign1 content for user_cose_sign1 authenticated endpoints in JavaScript/TypeScript apps (#5465).

4.0.6

  • Updated Open Enclave to 0.19.3.

4.0.5

  • Debug logging is now available in non-SGX builds by default, and controlled by a run-time CLI argument (--enclave-log-level). On SGX this remains a build-time decision (#5375).
  • Supporting intermediate cert chain included in TLS handshake, where previously only server leaf certificate was present (#5453).
  • Added getVersionOfPreviousWrite to TypeScript TypedKvMap interface (#5451).

4.0.4

  • Added TypeScript interfaces UserCOSESign1AuthnIdentity and MemberCOSESign1AuthnIdentity, to be used with user_cose_sign1 and member_cose_sign1 authentication policies.

4.0.3

  • User can now pass a --config-timeout option to cchost on startup. For example, a user wanting to start a cchost that may need to wait up 10 seconds for a valid config to appear under /cfg/path can invoke ./cchost --config-timeout 10s --config /path/cfg.
  • If a pid file path is configured, cchost will no longer start if a file is present at that path.

4.0.2

  • Added ccf::UserCOSESign1AuthnPolicy (C++) and user_cose_sign1 (JavaScript) authentication policies.

4.0.1

  • The set_js_runtime_options action now accepts return_exception_details and log_exception_details boolean options, which set the corresponding keys in the public:ccf.gov.js_runtime_options KV map. When enabled, a stack trace is respectively returned to the caller, and emitted to the log, on uncaught JS exceptions in application code.

Changed

  • For security reasons, OpenSSL >=1.1.1f must be first installed on the system (Ubuntu) before installing the CCF Debian package (#5227).

Added

  • Added ccf::historical::populate_service_endorsements to public C++ API, allowing custom historical endpoints to do the same work as adapters.

4.0.0

In order to upgrade an existing 3.x service to 4.x, CCF must be on the latest 3.x version (at least 3.0.10). For more information, see our documentation

Developer API

C++

  • When starting a host subprocess, applications may now pass data to its standard input. Additionally, the process' output is captured and logged by CCF (#5056).
  • Add new constructors to cryptography C++ API to generate EC/RSA/EdDSA keys from Json Web Key (#4876).
  • Added BaseEndpointRegistry::get_view_history_v1 function to get the view history since a given revision (#4580)
  • Renamed ccf::CodeDigest to ccf:pal::PlatformAttestationMeasurement and get_code_id() to get_measurement() (#5063).
  • ccf::RpcContext::set_response() has been renamed to ccf::RpcContext::set_response_json() (#4813).

JavaScript

  • Added logging of JS execution time for all requests. This can be disabled in confidential scenarios with the new ccf.enableMetricsLogging function in the JS API. After calling ccf.enableMetricsLogging(false), this logging will not be emitted.
  • Added ccf.enableUntrustedDateTime to JS API. After calling ccf.enableUntrustedDateTime(true), the Date global object will use the untrusted host time to retrieve the current time.
  • Add new ccf.crypto.jwkToPem, ccf.crypto.pubJwkToPem, ccf.crypto.rsaJwkToPem, ccf.crypto.pubRsaJwkToPem, ccf.crypto.eddsaJwkToPem, ccf.crypto.pubEddsaJwkToPem to JavaScript/TypesScript API to convert EC/RSA/EdDSA keys from PEM to Json Web Key (#4876).
  • ccf.crypto.sign() previously returned DER-encoded ECDSA signatures and now returns IEEE P1363 encoded signatures, aligning with the behavior of the Web Crypto API and ccf.crypto.verifySignature() (#4829).
  • Increased default NumHeapPages (heap size) for js_generic from 131072 (500MB) to 524288 (2GB).

Governance

  • The submit_recovery_share.sh script now takes a --cert argument.
  • Added missing ccf.gov.msg.type value encrypted_recovery_share to ccf_cose_sign1* scripts.
  • Proposals authenticated with COSE Sign1 must now contain a ccf.gov.msg.created_at header parameter, set to a positive integer number of seconds since epoch. This timestamp is used to detect potential proposal replay. The ccf_cose_sign1* scripts have been updated accordingly and require a --ccf-gov-msg-created_at.
  • The ccf Python package now includes a ccf_cose_sign1 CLI tool, to facilitate the creation of COSE Sign1 requests for governance purposes. It also includes ccf_cose_sign1_prepare and ccf_cose_sign1_finish CLI tools, to facilitate the creation of COSE Sign1 requests for governance purposes, signed with external key management systems such as AKV. See documentation for details.

Operations

  • ignore_first_sigterm config option. When set, will cause a node to ignore the first SIGTERM it receives, but the /node/state endpoint expose "stop_notice": true. A second SIGTERM will cause the process to shut down as normal. This can be useful in orchestration settings where nodes receive unsollicited signals that the operator wishes to react to.
  • Endorsement certificates for SEV-SNP attestation report can now be retrieved via an environment variable, as specified by attestation.environment.report_endorsements configuration entry (#4940).
  • Additional logging of historical query flow in UNSAFE builds.
  • enclave.type configuration entry now only supports Debug or Release. Trusted Execution Environment platform should be specified via new enclave.platform configuration entry (SGX, SNP or Virtual) (#4569).
  • consensus.type has been removed from cchost configuration.
  • Nodes running in confidential ACI (SEV-SNP) can now read the security context from a directory, as specified by attestation.environment.security_context_directory configuration entry (#5175).
  • SEV-SNP ACI: Remove support for reading security policy, report and UVM endorsements from environment variables. The environment.security_context_directory environment variable should be set instead (#5217).
  • Added a [gov] tag to logs emitted during governance operations. All logging from the constitution will have this tag added, and all error responses from /gov endpoints will now be logged with this tag.
  • Improved ledger durability when a node joins from an old snapshot (#5151).
  • Removed experimental 2tx reconfiguration mode, and the associated "reconfiguration_type" config option (#5179).

Client API

  • GET /gov/recovery_share is deprecated in favour of the unauthenticated GET /gov/encrypted_recovery_share/{member_id}.
  • New /node/index/strategies endpoint, which will list all indexing strategies currently installed alongside a description of how far each has progressed.
  • Added view_history and view_history_since query parameters to /app/commit endpoint for retrieving the full view history and the view history since a certain view (#4580)
  • /gov/members endpoint is deprecated. It is replaced by /gov/kv/members/certs, /gov/kv/members/encryption_public_keys, /gov/kv/members/info.
  • /gov/code endpoint is deprecated. It is replaced by /gov/kv/nodes/code_ids.
  • /gov/jwt_keys/all endpoint is deprecated. It is replaced by /gov/kv/jwt/public_signing_keys, /gov/kv/jwt/public_signing_key_issue, and /gov/kv/jwt/issuers
  • The built-in authentication policies for JWTs and certs will now enforce expiry times, based on the current time received from the host. JWTs must contain "nbf" and "exp" claims, and if those are outside the current time then the request will get an authentication error (#4786).
  • TCP_NODELAY is now set for all incoming and outgoing TCP connections (#4717).
  • Builtin governance tables now have endpoints for accessing their content directly from the KV, under /gov/kv. For instance, /gov/kv/constitution will read the current constitution.
  • Support for HTTP request signing has been removed (#5137). Governance requests must use COSE Sign1 signing instead, see documentation for details.

Dependencies

  • Updated Clang version requirement to >= 11 in cmake.
  • Updated Open Enclave to 0.19.0 final.
  • Upgraded t_cose from v1.1 to v1.1.1. v1.1.1 can optionally allow unknown critical header parameters in COSE_Sign1 envelopes which is desirable for CCF C++ applications.
  • Updated snmalloc to 0.6.0. This may result in a slight increase in the reported memory usage (~2MB), with improved latency for small memory allocations, especially in multi-threaded scenarios (#5165).
  • Update to clang-11 for SGX builds (#5165).

Bug Fixes

  • Historical query system will re-request entries if the host fails to provide them within a fixed time.
  • Node-to-node channels no longer check certificate expiry times. This previously caused "Peer certificate verification failed" error messages when node or service certs expired. (#4733)
  • node_data_json_file configuration option is now correctly applied in Start and Recover modes (#4761).
  • Session consistency is now provided even across elections. If session consistency would be broken, the inconsistent request will return an error and the TLS session will be terminated.
  • Fixed issue where invalid snapshots could be generated depending on the pattern of additions/removals of keys in a given key-value map (#4730).
  • Fix issue with large snapshots that may cause node crash on startup (join/recover) if configured stack size was too low (#4566).

4.0.0-rc2

Added

  • Nodes running in confidential ACI (SEV-SNP) can now read the security context from a directory, as specified by attestation.environment.security_context_directory configuration entry (#5175).

Changed

  • Updated Open Enclave to 0.19.0 (#5165).
  • Updated snmalloc to 0.6.0. This may result in a slight increase in the reported memory usage (~2MB), with improved latency for small memory allocations, especially in multi-threaded scenarios (#5165).
  • Update to clang-11 for SGX builds (#5165).

Removed

  • Support for HTTP request signing has been removed (#5137). Governance requests must use COSE Sign1 signing instead, see documentation for details.
  • Removed experimental 2tx reconfiguration mode, and the associated "reconfiguration_type" config option (#5179).

4.0.0-rc1

Changed

  • Added a [gov] tag to logs emitted during governance operations. All logging from the constitution will have this tag added, and all error responses from /gov endpoints will now be logged with this tag.
  • Improved ledger durability when a node joins from an old snapshot (#5151).

4.0.0-rc0

In order to upgrade an existing 3.x service to 4.x, CCF must be on the latest 3.x version (at least 3.0.10). For more information, see our documentation

Developer API

C++

  • When starting a host subprocess, applications may now pass data to its standard input. Additionally, the process' output is captured and logged by CCF (#5056).
  • Add new constructors to cryptography C++ API to generate EC/RSA/EdDSA keys from Json Web Key (#4876).
  • Added BaseEndpointRegistry::get_view_history_v1 function to get the view history since a given revision (#4580)
  • Renamed ccf::CodeDigest to ccf:pal::PlatformAttestationMeasurement and get_code_id() to get_measurement() (#5063).
  • ccf::RpcContext::set_response() has been renamed to ccf::RpcContext::set_response_json() (#4813).

JavaScript

  • Added logging of JS execution time for all requests. This can be disabled in confidential scenarios with the new ccf.enableMetricsLogging function in the JS API. After calling ccf.enableMetricsLogging(false), this logging will not be emitted.
  • Added ccf.enableUntrustedDateTime to JS API. After calling ccf.enableUntrustedDateTime(true), the Date global object will use the untrusted host time to retrieve the current time.
  • Add new ccf.crypto.jwkToPem, ccf.crypto.pubJwkToPem, ccf.crypto.rsaJwkToPem, ccf.crypto.pubRsaJwkToPem, ccf.crypto.eddsaJwkToPem, ccf.crypto.pubEddsaJwkToPem to JavaScript/TypesScript API to convert EC/RSA/EdDSA keys from PEM to Json Web Key (#4876).
  • ccf.crypto.sign() previously returned DER-encoded ECDSA signatures and now returns IEEE P1363 encoded signatures, aligning with the behavior of the Web Crypto API and ccf.crypto.verifySignature() (#4829).
  • Increased default NumHeapPages (heap size) for js_generic from 131072 (500MB) to 524288 (2GB).

Governance

  • The submit_recovery_share.sh script now takes a --cert argument.
  • Added missing ccf.gov.msg.type value encrypted_recovery_share to ccf_cose_sign1* scripts.
  • Proposals authenticated with COSE Sign1 must now contain a ccf.gov.msg.created_at header parameter, set to a positive integer number of seconds since epoch. This timestamp is used to detect potential proposal replay. The ccf_cose_sign1* scripts have been updated accordingly and require a --ccf-gov-msg-created_at.
  • The ccf Python package now includes a ccf_cose_sign1 CLI tool, to facilitate the creation of COSE Sign1 requests for governance purposes. It also includes ccf_cose_sign1_prepare and ccf_cose_sign1_finish CLI tools, to facilitate the creation of COSE Sign1 requests for governance purposes, signed with external key management systems such as AKV. See documentation for details.

Operations

  • ignore_first_sigterm config option. When set, will cause a node to ignore the first SIGTERM it receives, but the /node/state endpoint expose "stop_notice": true. A second SIGTERM will cause the process to shut down as normal. This can be useful in orchestration settings where nodes receive unsollicited signals that the operator wishes to react to.
  • Endorsement certificates for SEV-SNP attestation report can now be retrieved via an environment variable, as specified by attestation.environment.report_endorsements configuration entry (#4940).
  • Additional logging of historical query flow in UNSAFE builds.
  • enclave.type configuration entry now only supports Debug or Release. Trusted Execution Environment platform should be specified via new enclave.platform configuration entry (SGX, SNP or Virtual) (#4569).

Client API

  • GET /gov/recovery_share is deprecated in favour of the unauthenticated GET /gov/encrypted_recovery_share/{member_id}.
  • New /node/index/strategies endpoint, which will list all indexing strategies currently installed alongside a description of how far each has progressed.
  • Added view_history and view_history_since query parameters to /app/commit endpoint for retrieving the full view history and the view history since a certain view (#4580)
  • /gov/members endpoint is deprecated. It is replaced by /gov/kv/members/certs, /gov/kv/members/encryption_public_keys, /gov/kv/members/info.
  • /gov/code endpoint is deprecated. It is replaced by /gov/kv/nodes/code_ids.
  • /gov/jwt_keys/all endpoint is deprecated. It is replaced by /gov/kv/jwt/public_signing_keys, /gov/kv/jwt/public_signing_key_issue, and /gov/kv/jwt/issuers
  • The built-in authentication policies for JWTs and certs will now enforce expiry times, based on the current time received from the host. JWTs must contain "nbf" and "exp" claims, and if those are outside the current time then the request will get an authentication error (#4786).
  • TCP_NODELAY is now set for all incoming and outgoing TCP connections (#4717).
  • Builtin governance tables now have endpoints for accessing their content directly from the KV, under /gov/kv. For instance, /gov/kv/constitution will read the current constitution.

Dependencies

  • Updated Clang version requirement to >= 10 in cmake.
  • Upgraded OpenEnclave to 0.18.5.
  • Upgraded t_cose from v1.1 to v1.1.1. v1.1.1 can optionally allow unknown critical header parameters in COSE_Sign1 envelopes which is desirable for CCF C++ applications.

Bug Fixes

  • Historical query system will re-request entries if the host fails to provide them within a fixed time.
  • Node-to-node channels no longer check certificate expiry times. This previously caused "Peer certificate verification failed" error messages when node or service certs expired. (#4733)
  • node_data_json_file configuration option is now correctly applied in Start and Recover modes (#4761).
  • Session consistency is now provided even across elections. If session consistency would be broken, the inconsistent request will return an error and the TLS session will be terminated.
  • Fixed issue where invalid snapshots could be generated depending on the pattern of additions/removals of keys in a given key-value map (#4730).
  • Fix issue with large snapshots that may cause node crash on startup (join/recover) if configured stack size was too low (#4566).

4.0.0-dev6

Added

  • Added logging of JS execution time for all requests. This can be disabled in confidential scenarios with the new ccf.enableMetricsLogging function in the JS API. After calling ccf.enableMetricsLogging(false), this logging will not be emitted.

4.0.0-dev5

Changed

  • Additional logging of historical query flow in UNSAFE builds.
  • Historical query system will re-request entries if the host fails to provide them within a fixed time.
  • Renamed ccf::CodeDigest to ccf:pal::PlatformAttestationMeasurement and get_code_id() to get_measurement() (#5063).

Dependencies

  • Upgraded OpenEnclave to 0.18.5.
  • Upgraded t_cose from v1.1 to v1.1.1. v1.1.1 can optionally allow unknown critical header parameters in COSE_Sign1 envelopes which is desirable for CCF C++ applications.

Added

  • New /node/index/strategies endpoint, which will list all indexing strategies currently installed alongside a description of how far each has progressed.
  • When starting a host subprocess, applications may now pass data to its standard input. Additionally, the process' output is captured and logged by CCF (#5056).
  • ignore_first_sigterm config option. When set, will cause a node to ignore the first SIGTERM it receives, but the /node/state endpoint expose "stop_notice": true. A second SIGTERM will cause the process to shut down as normal. This can be useful in orchestration settings where nodes receive unsollicited signals that the operator wishes to react to.

4.0.0-dev4

Changed

  • /gov/members endpoint is deprecated. It is replaced by /gov/kv/members/certs, /gov/kv/members/encryption_public_keys, /gov/kv/members/info.
  • /gov/code endpoint is deprecated. It is replaced by /gov/kv/nodes/code_ids.
  • /gov/jwt_keys/all endpoint is deprecated. It is replaced by /gov/kv/jwt/public_signing_keys, /gov/kv/jwt/public_signing_key_issue, and /gov/kv/jwt/issuers
  • ccf::RpcContext::set_response() has been renamed to ccf::RpcContext::set_response_json() (#4813).
  • The built-in authentication policies for JWTs and certs will now enforce expiry times, based on the current time received from the host. JWTs must contain "nbf" and "exp" claims, and if those are outside the current time then the request will get an authentication error (#4786).
  • ccf.crypto.sign() previously returned DER-encoded ECDSA signatures and now returns IEEE P1363 encoded signatures, aligning with the behavior of the Web Crypto API and ccf.crypto.verifySignature() (#4829).
  • Proposals authenticated with COSE Sign1 must now contain a ccf.gov.msg.created_at header parameter, set to a positive integer number of seconds since epoch. This timestamp is used to detect potential proposal replay. The ccf_cose_sign1* scripts have been updated accordingly and require a --ccf-gov-msg-created_at.
  • Updated Clang version requirement to >= 10 in cmake.

Added

  • Added ccf.enableUntrustedDateTime to JS API. After calling ccf.enableUntrustedDateTime(true), the Date global object will use the untrusted host time to retrieve the current time.
  • Add new ccf.crypto.jwkToPem, ccf.crypto.pubJwkToPem, ccf.crypto.rsaJwkToPem, ccf.crypto.pubRsaJwkToPem, ccf.crypto.eddsaJwkToPem, ccf.crypto.pubEddsaJwkToPem to JavaScript/TypesScript API to convert EC/RSA/EdDSA keys from PEM to Json Web Key (#4876).
  • Add new constructors to cryptography C++ API to generate EC/RSA/EdDSA keys from Json Web Key (#4876).
  • Endorsement certificates for SEV-SNP attestation report can now be retrieved via an environment variable, as specified by attestation.environment.report_endorsements configuration entry (#4940).

4.0.0-dev3

Fixed

  • Node-to-node channels no longer check certificate expiry times. This previously caused "Peer certificate verification failed" error messages when node or service certs expired. (#4733)
  • node_data_json_file configuration option is now correctly applied in Start and Recover modes (#4761).

Changed

  • Increased default NumHeapPages (heap size) for js_generic from 131072 (500MB) to 524288 (2GB).
  • TCP_NODELAY is now set for all incoming and outgoing TCP connections (#4717).

4.0.0-dev2

Added

  • The ccf Python package now includes a ccf_cose_sign1 CLI tool, to facilitate the creation of COSE Sign1 requests for governance purposes. It also includes ccf_cose_sign1_prepare and ccf_cose_sign1_finish CLI tools, to facilitate the creation of COSE Sign1 requests for governance purposes, signed with external key management systems such as AKV. See documentation for details.
  • Builtin governance tables now have endpoints for accessing their content directly from the KV, under /gov/kv. For instance, /gov/kv/constitution will read the current constitution.

Fixed

  • Session consistency is now provided even across elections. If session consistency would be broken, the inconsistent request will return an error and the TLS session will be terminated.
  • Fixed issue where invalid snapshots could be generated depending on the pattern of additions/removals of keys in a given key-value map (#4730).

4.0.0-dev0

Added

  • Added view_history and view_history_since query parameters to /app/commit endpoint for retrieving the full view history and the view history since a certain view (#4580)
  • Added BaseEndpointRegistry::get_view_history_v1 function to get the view history since a given revision (#4580)

Changed

  • enclave.type configuration entry now only supports Debug or Release. Trusted Execution Environment platform should be specified via new enclave.platform configuration entry (SGX, SNP or Virtual) (#4569).

Fixed

  • Fix issue with large snapshots that may cause node crash on startup (join/recover) if configured stack size was too low (#4566).

3.0.0-rc2

Dependencies

  • Upgraded OpenEnclave to 0.18.4.

Added

  • Added new ccf.crypto.eddsaPemToJwk, ccf.crypto.pubEddsaPemToJwk to JavaScript/TypesScript API to convert EdDSA keys from PEM to JWK (#4524).

Changed

3.0.0-rc1

Added

  • sandbox.sh now accepts a --consensus-update-timeout-ms to modify the consensus.message_timeout value in each node's configuration. This can be used to alter multi-node commit latency.
  • Add ccf.crypto.sign() API in the JavaScript runtime (#4454).

Changed

  • CCF is now a separate CMake project and Debian package per platform (sgx, snp and virtual), rather than the same project and package with a decorated version, to prevent accidental misuse and narrow down dependencies. (#4421).
    • C++ applications should find the appropriate CCF package in CMake with find_package("ccf_<platform>" REQUIRED).
    • CCF Debian packages are now installed at /opt/ccf_<platform> rather than /opt/ccf.
  • We now support QuickJS runtime caps such as max_heap_bytes, max_stack_bytes and max_execution_time_ms. These can be set via a governance proposal. They can also be fetched via the GET /node/js_metrics endpoint (#4396).

3.0.0-rc0

Developer API

C++

  • Removed deprecated set_execute_outside_consensus() API (#3886, #3673).
  • Application code should now use the CCF_APP_* macros rather than LOG_*_FMT (eg - CCF_APP_INFO replacing LOG_INFO_FMT). The new macros will add an [app] tag to all lines so they can be easily filtered from framework code (#4024).
  • The previous logging macros (LOG_INFO_FMT, LOG_DEBUG_FMT etc) have been deprecated, and should no longer be used by application code. Replace with the CCF_APP_* equivalent.
  • Added a new method get_decoded_request_path_params that returns a map of decoded path parameters (#4126).
  • New crypto::hmac API (#4204).
  • The ccf::RpcContext now contains functionality for storing user data with set_user_data and retrieving it with get_user_data (#4291).
  • There are now make_endpoint_with_local_commit_handler and make_read_only_endpoint_with_local_commit_handler functions to install endpoints with post local-commit logic (#4296).
  • ccf::historical::adapter, ccf::historical::adapter_v1, ccf::historical::is_tx_committed and ccf::historical::is_tx_committed_v1 have been removed. Application code should upgrade to ccf::historical::adapter_v3 and ccf::historical::is_tx_committed_v2.
  • ccf::EnclaveAttestationProvider is deprecated and will be removed in a future release. It should be replaced by ccf::AttestationProvider.
  • The functions starts_with, ends_with, remove_prefix, and remove_suffix, and the type remove_cvref have been removed from nonstd::. The C++20 equivalents should be used instead.

JavaScript

  • Add ccf.generateEcdsaKeyPair API in the JavaScript runtime (#4271).
  • Add secp256k1 support to ccf.crypto.generateEcdsaKeyPair() and ccf.crypto.verifySignature() (#4347).
  • Add ccf.crypto.generateEddsaKeyPair() API with Curve25519 support in the JavaScript runtime (#4391).
  • Add new ccf.crypto.pemToJwk, ccf.crypto.pubPemToJwk, ccf.crypto.rsaPemToJwk, ccf.crypto.pubRsaPemToJwk to JavaScript/TypesScript API to convert EC/RSA keys from PEM to JWK (#4359).

Governance

  • set_user action in sample constitutions correctly handles user_data (#4229).
  • Governance endpoints now support COSE Sign1 input, as well as signed HTTP requests (#4392).

Operations

  • The node-to-node interface configuration now supports a published_address to enable networks with nodes running in different (virtual) subnets (#3867).
  • Primary node now automatically steps down as backup (in the same view) if it has not heard back from a majority of backup nodes for an election timeout (#3685).
  • New nodes automatically shutdown if the target service certificate is misconfigured (#3895).
  • New per-interface configuration entries (network.rpc_interfaces.http_configuration) are added to let operators cap the maximum size of body, header value size and number of headers in client HTTP requests. The client session is automatically closed if the HTTP request exceeds one of these limits (#3941).
  • Added new read_only_directory snapshots directory node configuration so that committed snapshots can be shared between nodes (#3973).
  • Fixed issue with recovery of large ledger entries (#3986).
  • New GET /node/network/removable_nodes and DELETE /node/network/nodes/{node_id} exposed to allow operator to decide which nodes can be safely shut down after retirement, and clear their state from the Key-Value Store.
  • Fixed issue where two primary nodes could be elected if an election occurred while a reconfiguration transaction was still pending (#4018).
  • New snpinfo.sh script (#4196).
  • New "attestation" section in node JSON configuration to specify remote endpoint required to retrieve the endorsement certificates for SEV-SNP attestation report (#4277, #4302).

Release artefacts

  • ccf_unsafe is now a separate project and package, rather than the same project and package with a decorated version, to prevent accidental misuse.
  • Release assets now include variants per TEE platform: ccf_sgx_<version>_amd64.deb, ccf_snp_<version>_amd64.deb and ccf_virtual_<version>_amd64.deb.
  • Docker images now include variants per TEE platform, identified via image tag: :<version>-sgx, :<version>-snp and :<version>-virtual.

Auditor

  • Node and service PEM certificates no longer contain a trailing null byte (#3885).

Client API

  • Added a GET /node/service/previous_identity endpoint, which can be used during a recovery to look up the identity of the service before the catastrophic failure (#3880).
  • GET /node/version now contains an unsafe flag reflecting the status of the build.
  • Added new recovery_count field to GET /node/network endpoint to track the number of disaster recovery procedures undergone by the service (#3982).
  • Added new service_data_json_file configuration entry to cchost to point to free-form JSON file to set arbitrary data to service (#3997).
  • Added new current_service_create_txid field to GET /node/network endpoint to indicate TxID at which current service was created (#3996).
  • Experimental support for HTTP/2 (#4010).
  • Generated OpenAPI now describes whether each endpoint is forwarded (#3935).
  • When running with curve-id set to secp256r1, we now correctly support temporary ECDH keys on curve secp256r1 for TLS 1.2 clients.
  • Application-defined endpoints are now accessible with both /app prefix and un-prefixed, e.g. GET /app/log/private and GET /log/private (#4147).

Dependencies

  • Updated PSW in images to 2.16.100.
  • Upgraded Open Enclave to 0.18.1 (#4023).

Documentation

  • The "Node Output" page has been relabelled as "Troubleshooting" in the documentation and CLI commands for troubleshooting have been added to it.

3.0.0-dev7

Added

  • Added new ccf.crypto.pemToJwk, ccf.crypto.pubPemToJwk, ccf.crypto.rsaPemToJwk, ccf.crypto.pubRsaPemToJwk to JavaScript/TypesScript API to convert EC/RSA keys from PEM to JWK (#4359).

Changed

  • JavaScript crypto API (e.g. generateAesKey and wrapKey) are now included as part of the ccf.crypto package (#4372).

3.0.0-dev6

Added

  • Experimental ccf::MemberCOSESign1AuthnPolicy (#3875)
  • Add secp256k1 support to ccf.crypto.generateEcdsaKeyPair() and ccf.crypto.verifySignature() (#4347).

Deprecated

  • ccf::EnclaveAttestationProvider is deprecated and will be removed in a future release. It should be replaced by ccf::AttestationProvider

3.0.0-dev5

Added

  • Added a new proposal action set_js_runtime_options that accepts max_heap_bytes and max_stack_bytes for QuickJS runtime.
  • Experimental support for AMD SEV-SNP nodes (#4106, #4235)
  • New "attestation" section in node JSON configuration to specify remote endpoint required to retrieve the endorsement certificates for SEV-SNP attestation report (#4277, #4302).
  • The ccf::RpcContext now contains functionality for storing user data with set_user_data and retrieving it with get_user_data (#4291).
  • There are now make_endpoint_with_local_commit_handler and make_read_only_endpoint_with_local_commit_handler functions to install endpoints with post local-commit logic (#4296).

Changed

  • The endpoint GET /node/js_metrics now also returns the QuickJS runtime memory options.

Fixed

  • Also install *.inc files for third-party dependencies (#4266).
  • Add ccf.generateEcdsaKeyPair API in the JavaScript runtime (#4271).

Removed

  • ccf::historical::adapter, ccf::historical::adapter_v1, ccf::historical::is_tx_committed and ccf::historical::is_tx_committed_v1 have been removed. Application code should upgrade to ccf::historical::adapter_v3 and ccf::historical::is_tx_committed_v2.

3.0.0-dev4

Fixed

  • set_user action in sample constitutions correctly handles user_data (#4229).

Removed

  • Snapshots generated by 1.x services can no longer be used to join from or recover a new service, i.e. 1.x services should first upgrade to 2.x before upgrading to 3.x when making use of existing snapshots (#4255).

3.0.0-dev3

Added

  • New snpinfo.sh script (#4196).
  • New crypto::hmac API (#4204).

Changed

  • Application-defined endpoints are now accessible with both /app prefix and un-prefixed, e.g. GET /app/log/private and GET /log/private (#4147).
  • The method EndpointRegistry::get_metrics_for_endpoint(const EndpointDefinitionPtr&) has been replaced with EndpointRegistry::get_metrics_for_endpoint(const std::string& method, const std::string& verb).

3.0.0-dev2

Dependencies

  • Upgraded OpenEnclave to 0.18.2 (#4132).

Added

  • New GET /node/network/removable_nodes and DELETE /node/network/nodes/{node_id} exposed to allow operator to decide which nodes can be safely shut down after retirement, and clear their state from the Key-Value Store.
  • Added a new method get_decoded_request_path_params that returns a map of decoded path parameters (#4126)

Changed

  • Calling remove(K) on a KV handle no longer returns a bool indicating if the key was previously present. This can be simulated by calling has(K) beforehand. This avoids introducing a read-dependency with every call to remove().

Fixed

  • Fixed issue where two primary nodes could be elected if an election occurred while a reconfiguration transaction was still pending (#4018).
  • When running with --curve-id secp256r1, we now correctly support temporary ECDH keys on curve secp256r1 for TLS 1.2 clients.

Deprecated

  • The previous logging macros (LOG_INFO_FMT, LOG_DEBUG_FMT etc) have been deprecated, and should no longer be used by application code. Replace with the CCF_APP_* equivalent.

3.0.0-dev1

Added

  • /node/version now contains an unsafe flag reflecting the status of the build.
  • New per-interface configuration entries (network.rpc_interfaces.http_configuration) are added to let operators cap the maximum size of body, header value size and number of headers in client HTTP requests. The client session is automatically closed if the HTTP request exceeds one of these limits (#3941).
  • Added new recovery_count field to GET /node/network endpoint to track the number of disaster recovery procedures undergone by the service (#3982).
  • Added new service_data_json_file configuration entry to cchost to point to free-form JSON file to set arbitrary data to service (#3997).
  • Added new current_service_create_txid field to GET /node/network endpoint to indicate TxID at which current service was created (#3996).
  • Added new read_only_directory snapshots directory node configuration so that committed snapshots can be shared between nodes (#3973).
  • Experimental support for HTTP/2 (#4010).

Changed

  • Generated OpenAPI now describes whether each endpoint is forwarded (#3935).
  • Application code should now use the CCF_APP_* macros rather than LOG_*_FMT (eg - CCF_APP_INFO replacing LOG_INFO_FMT). The new macros will add an [app] tag to all lines so they can be easily filtered from framework code (#4024).

Fixed

  • Fixed issue with recovery of large ledger entries (#3986).

Documentation

  • The "Node Output" page has been relabelled as "Troubleshooting" in the documentation and CLI commands for troubleshooting have been added to it.

Dependencies

  • Upgraded Open Enclave to 0.18.1 (#4023).

3.0.0-dev0

Added

  • The node-to-node interface configuration now supports a published_address to enable networks with nodes running in different (virtual) subnets (#3867).
  • Added a GET /node/service/previous_identity endpoint, which can be used during a recovery to look up the identity of the service before the catastrophic failure (#3880).
  • Added an automatic certificate management environment (ACME) client to automatically manage TLS certificates that are globally endorsed by an external authority, e.g. Let's Encrypt (#3877).

Changed

  • Primary node now automatically steps down as backup (in the same view) if it has not heard back from a majority of backup nodes for an election timeout (#3685).
  • Node and service PEM certificates no longer contain a trailing null byte (#3885).
  • New nodes automatically shutdown if the target service certificate is misconfigured (#3895).
  • Updated PSW in images to 2.16.100.
  • ccf_unsafe is now a separate project and package, rather than the same project and package with a decorated version, to prevent accidental misuse.

Removed

  • Removed deprecated set_execute_outside_consensus() API (#3886, #3673).

2.0.0

See documentation for code upgrade 1.x to 2.0 to upgrade an existing 1.x CCF service to 2.0

Developer API

C++

  • CCF is now built with Clang 10. It is strongly recommended that C++ applications upgrade to Clang 10 as well.

  • Raised the minimum supported CMake version for building CCF to 3.16 (#2946).

  • Removed mbedtls as cryptography and TLS library.

  • The CCF public API is now under include/ccf, and all application includes of framework code should use only these files.

  • Private headers have been moved to ccf/include/ccf/_private so they cannot be accidentally included from existing paths. Any applications relying on private headers should remove this dependency, or raise an issue to request the dependency be moved to the public API. In a future release private headers will be removed entirely from the installed package.

  • The enclave:: namespace has been removed, and all types which were under it are now under ccf::. This will affect any apps using enclave::RpcContext, which should be replaced with ccf::RpcContext (#3664).

  • The kv::Store type is no longer visible to application code, and is replaced by a simpler kv::ReadOnlyStore. This is the interface given to historical queries to access historical state and enforces read-only access, without exposing internal implementation details of the store. This should have no impact on JS apps, but C++ apps will need to replace calls to store->current_txid() with calls to store->get_txid(), and store->create_tx() to store->create_read_only_tx().

  • The C++ types used to define public governance tables are now exposed in public headers. Any C++ applications reading these tables should update their include paths (ie - #include "service/tables/nodes.h" => #include "ccf/service/tables/nodes.h") (#3608).

  • TxReceipt::describe() has been replaced with ccf::describe_receipt_v2(). Note that the previous JSON format is still available, but must be retrieved as a JSON object from describe_receipt_v1(). Includes of the private node/tx_receipt.h from C++ applications should be removed (#3610).

  • The entry point for creation of C++ apps is now make_user_endpoints(). The old entry point get_rpc_handler() has been removed (#3562). For an example of the necessary change, see this diff of the logging sample app (#3562).

  • Added get_untrusted_host_time_v1 API. This can be used to retrieve a timestamp during endpoint execution, accurate to within a few milliseconds. Note that this timestamp comes directly from the host so is not trusted, and should not be used to make sensitive decisions within a transaction (#2550).

  • Added get_quotes_for_all_trusted_nodes_v1 API. This returns the ID and quote for all nodes which are currently trusted and participating in the service, for live audit (#2511).

  • Added get_metrics_v1 API to BaseEndpointRegistry for applications that do not make use of builtins and want to version or customise metrics output.

  • Added set_claims_digest() API to RpcContext, see documentation on how to use it to attach application-defined claims to transaction receipts.

  • Added indexing system to speed up historical queries (#3280, #3444).

  • Removed get_node_state() from AbstractNodeContext. The local node's ID is still available to endpoints as get_node_id(), and other subsystems which are app-visible can be fetched directly (#3552).

  • Receipts now come with service endorsements of previous service identities after recoveries (#3679). See verify_receipt in e2e_logging.py for an example of how to verify the resulting certificate chain. This functionality is introduced in ccf::historical::adapter_v3.

  • ccf::historical::adapter_v2, and its successor ccf::historical::adapter_v3 now return 404, with either TransactionPendingOrUnknown or TransactionInvalid, rather than 400 when a user performs a historical query for a transaction id that is not committed.

  • ccf::historical::AbstractStateCache::drop_requests() renamed to drop_cached_states() (#3187).

Key-Value Store

  • Added kv::Value and kv::Set, as a more error-proof alternative to kv::Maps which had a single key or meaningless values (#2599).
  • Added foreach_key and foreach_value to C++ KV API, to iterate without deserializing both entries when only one is used (#2918).

JavaScript

  • Added JavaScript bytecode caching to avoid repeated compilation overhead. See the documentation for more information (#2643).

  • Added ccf.crypto.verifySignature() for verifying digital signatures to the JavaScript API (#2661).

  • Added experimental JavaScript API ccf.host.triggerSubprocess() (#2461).

  • ccf.crypto.verifySignature() previously required DER-encoded ECDSA signatures and now requires IEEE P1363 encoded signatures, aligning with the behavior of the Web Crypto API (#2735).

  • ccf.historical.getStateRange / ccf.historical.dropCachedStates JavaScript APIs to manually retrieve historical state in endpoints declared as "mode": "readonly" (#3033).

  • JavaScript endpoints with "mode": "historical" now expose the historical KV at ccf.historicalState.kv while ccf.kv always refers to the current KV state. Applications relying on the old behaviour should make their code forward-compatible before upgrading to 2.x with const kv = ccf.historicalState.kv || ccf.kv.

  • Receipts accessible through JavaScript no longer contain the redundant root hash field. Applications should be changed to not rely on this field anymore before upgrading to 2.x.

  • Add request details with additional URL components to JS + TS API: request.url, request.route, request.method, request.hostname (#3498).

Governance

  • Updated actions.js constitution fragment to record service-endorsed node certificate on the transition_node_to_trusted action. The constitution must be updated using the existing set_constitution proposal (#2844).
  • The existing transition_node_to_trusted proposal action now requires a new valid_from argument (and optional validity_period_days, which defaults to the value of maximum_node_certificate_validity_days).
  • The proposal_generator has been removed from the ccf Python package. The majority of proposals can be trivially constructed in existing client tooling, without needing to invoke Python. This also introduces parity between the default constitution and custom constitution actions - all should be constructed and called from the same governance client code. Some jinja templates are included in samples/templates for constructing careful ballots from existing proposals.
  • A new governance action trigger_ledger_chunk to request the creation of a ledger chunk at the next signature (#3519).
  • A new governance action trigger_snapshot to request the creation of a snapshot at the next signature (#3544).
  • Configurations and proposals now accept more date/time formats, including the Python-default ISO 8601 format (#3739).
  • The transition_service_to_open governance proposal now requires the service identity as an argument to ensure the correct service is started. During recovery, it further requires the previous service identity to ensure the right service is recovered (#3624).

Operations

cchost Configuration

  • Breaking change: Configuration for CCF node is now a JSON configuration file passed in to cchost via --config /path/to/config/file/ CLI argument. Existing CLI arguments have been removed. The migrate_1_x_config.py script (included in ccf Python package) should be used to migrate existing .ini configuration files to .json format (#3209).
  • Added support for listening on multiple interfaces for incoming client RPCs, with individual session caps (#2628).
  • The per-node session cap behaviour has changed. The network.rpc_interfaces.<interface_name>.max_open_sessions_soft is now a soft cap on the number of sessions. Beyond this, new sessions will receive a HTTP 503 error immediately after completing the TLS handshake. The existing hard cap (where sessions are closed immediately, before the TLS handshake) is still available, under the new argument network.rpc_interfaces.<interface_name>.max_open_sessions_hard (#2583).
  • Snapshot files now include receipt of evidence transaction. Nodes can now join or recover a service from a standalone snapshot file. 2.x nodes can still make use of snapshots created by a 1.x node, as long as the ledger suffix containing the proof of evidence is also specified at start-up (#2998).
  • If no node_certificate.subject_alt_names is specified at node start-up, the node certificate Subject Alternative Name extension now defaults to the value of published_address of the first RPC interface (#2902).
  • Primary node now also reports time at which the ack from each backup node was last received (GET /node/consensus endpoint). This can be used by operators to detect one-way partitions between the primary and backup nodes (#3769).
  • Added new GET /node/self_signed_certificate endpoint to retrieve the self-signed certificate of the target node (#3767).
  • New GET /gov/members endpoint which returns details of all members from the KV (#3615).
  • The new endorsement configuration entry lets operators set the desired TLS certificate endorsement, either service-endorsed or node-endorsed (self-signed), for each network RPC interface of a node, defaulting to service-endorsed (#2875).

Certificate(s) Validity Period

  • Nodes certificates validity period is no longer hardcoded and must instead be set by operators and renewed by members (#2924):

    • The new node_certificate.initial_validity_days (defaults to 1 day) configuration entry lets operators set the initial validity period for the node certificate (valid from the current system time).
    • The new command.start.service_configuration.maximum_node_certificate_validity_days (defaults to 365 days) configuration entry sets the maximum validity period allowed for node certificates.
    • The new set_node_certificate_validity proposal action allows members to renew a node certificate (or set_all_nodes_certificate_validity equivalent action to renew all trusted nodes certificates).

  • Service certificate validity period is no longer hardcoded and must instead be set by operators and renewed by members (#3363):

    • The new service_certificate_initial_validity_days (defaults to 1 day) configuration entry lets operators set the initial validity period for the service certificate (valid from the current system time).
    • The new maximum_service_certificate_validity_days (defaults to 365 days) configuration entry sets the maximum validity period allowed for service certificate.
    • The new set_service_certificate_validity proposal action allows members to renew the service certificate.

Misc

  • The service certificate output by first node default name is now service_cert.pem rather than networkcert.pem (#3363).

  • Log more detailed errors on early startup (#3116).

  • Format of node output RPC and node-to-node addresses files is now JSON (#3300).

  • Joining nodes now present service-endorsed certificate in client TLS sessions after they have observed their own addition to the store, rather than as soon as they have joined the service. Operators should monitor the initial progress of a new node using its self-signed certificate as TLS session certificate authority (#2844).

  • Slow ledger IO operations will now be logged at level FAIL. The threshold over which logging will activate can be adjusted by the slow_io_logging_threshold configuration entry to cchost (#3067).

  • Added a new client_connection_timeout configuration entry to specify the maximum time a node should wait before re-establishing failed client connections. This should be set to a significantly lower value than consensus.election_timeout (#2618).

  • Nodes code digests are now extracted and cached at network join time in public:ccf.gov.nodes.info, and the GET /node/quotes and GET /node/quotes/self endpoints will use this cached value whenever possible (#2651).

  • DNS resolution of client connections is now asynchronous (#3140).

  • The curve-id selected for the identity of joining nodes no longer needs to match that of the network (#2525).

  • Removed long-deprecated --domain argument from cchost. Node certificate Subject Alternative Names should be passed in via existing node_certificate.subject_alt_names configuration entry (#2798).

  • Added experimental support for 2-transaction reconfiguration with CFT consensus, see documentation. Note that mixing 1tx and 2tx nodes in the same network is unsupported and unsafe at this stage (#3097).

  • Aside from regular release packages, CCF now also provides unsafe packages with verbose logging, helpful for troubleshooting. The extent of the logging in these builds make them fundamentally UNSAFE to use for production purposes, hence the name.

  • Nodes no longer crash at start-up if the ledger in the read-only ledger directories (ledger.read_only_directories) is ahead of the ledger in the main ledger directory (ledger.directory) (#3597).

  • Nodes now have a free-form node_data field, to match users and members. This can be set when the node is launched, or modified by governance. It is intended to store correlation IDs describing the node's deployment, such as a VM name or Pod identifier (#3662).

  • New GET /node/consensus endpoint now also returns primary node ID and current view (#3666).

  • HTTP parsing errors are now recorded per-interface and returned by GET /node/metrics (#3671).

  • Failed recovery procedures no longer block subsequent recoveries: .recovery ledger files are now created while the recovery is in progress and ignored or deleted by nodes on startup (#3563).

  • Corrupted or incomplete ledger files are now recovered gracefully, until the last valid entry (#3585).

Fixed

  • Fixed issue with ledger inconsistency when starting a new joiner node without a snapshot but with an existing ledger prefix (#3064).
  • Fixed issue with join nodes which could get stuck if an election was triggered while catching up (#3169).
  • Nodes joining must have a snapshot at least as recent as the primary's (#3573).

Release artefacts

  • cchost can now run both SGX and virtual enclave libraries. cchost.virtual is no longer needed, and has been removed (#3476).
  • CCF Docker images are now available through Azure Container Registry rather than Docker Hub (#3839, #3821).
    • The ccfmsrc.azurecr.io/ccf-sgx-app-run image is now available at ccfmsrc.azurecr.io/public/ccf/app/run:<tag>-sgx.
    • The ccfmsrc.azurecr.io/ccf-sgx-app-dev image is now available at ccfmsrc.azurecr.io/public/ccf/app/dev:<tag>-sgx.
    • New ccfmsrc.azurecr.io/public/ccf/app/run-js JavaScript application runtime image (including libjs_generic application under /usr/lib/ccf) (#3845).

Auditor

  • Receipts now include the endorsed certificate of the node, as well as its node id, for convenience (#2991).
  • Retired nodes are now removed from the store/ledger as soon as their retirement is committed (#3409).
  • Service-endorsed node certificates are now recorded in a new public:ccf.gov.nodes.endorsed_certificates table, while the existing cert field in the public:ccf.gov.nodes.info table is now deprecated (#2844).
  • New split_ledger.py utility to split existing ledger files (#3129).
  • Python ccf.read_ledger module now accepts custom formatting rules for the key and value based on the key-value store table name (#2791).
  • Ledger entries now contain a commit_evidence_digest, as well as a claims_digest, which can be set with set_claims_digest(). The digest of the write set was previously the per-transaction leaf in the Merkle Tree, but is now combined with the digest of the commit evidence and the user claims. Receipt verification instructions have been amended accordingly. The presence of commit_evidence in receipts serves two purposes: giving the user access to the TxID without having to parse the write set, and proving that a transaction has been committed by the service. Transactions are flushed to disk eagerly by the primary to keep in-enclave memory use to a minimum, so the existence of a ledger suffix is not on its own indicative of its commit status. The digest of the commit evidence is in the ledger to allow audit and recovery, but only the disclosure of the commit evidence indicates that a transaction has been committed by the service
  • Add --insecure-skip-verification to ledger_viz utility, to allow visualisation of unverified ledger chunks (#3618).
  • Add --split-services to ledger_viz utility, to easily find out at which TxID new services were created (#3621).
  • Python ccf.read_ledger and ccf.ledger_viz tools now accept paths to individual ledger chunks, to avoid parsing the entire ledger.

Client API

  • Added support for TLS 1.3 (now used by default).

  • Added GET /gov/jwt_keys/all endpoint (#2519).

  • Added new operator RPC GET /node/js_metrics returning the JavaScript bytecode size and whether the bytecode is used (#2643).

  • Added a new GET /node/metrics endpoint which includes the count of active and peak concurrent sessions handled by the node (#2596).

  • Added endpoint to obtain service configuration via GET /node/service/configuration (#3251).

  • Added QuickJS version to RPC GET /node/version (#2643).

  • Added a GET /node/jwt_metrics endpoint to monitor attempts and successes of key refresh for each issuer. See documentation on how to use it.

  • Schema of GET /network/nodes/{node_id} and GET /network/nodes endpoints has been modified to include all RPC interfaces (#3300).

  • Improved performance for lookup of path-templated endpoints (#2918).

  • CCF now responds to HTTP requests that could not be parsed with a 400 response including error details (#2652).

  • Node RPC interfaces do not transition anymore from node-endorsed to service-endorsed TLS certificates but are fixed to a single configured type. While a given endorsement is not available yet (typically at start-up for service-endorsed certificates) the interface rejects TLS sessions instead of defaulting to a node-endorsed certificate (#2875).

  • Websockets endpoints are no longer supported. Usage is insufficient to justify ongoing maintenance.

  • The ccf Python package no longer provides utilities to issue requests to a running CCF service. This is because CCF supports widely-used client-server protocols (TLS, HTTP) that should already be provided by libraries for all programming languages. The ccf Python package can still be used to audit the ledger and snapshot files (#3386).

Dependencies

  • Upgraded Open Enclave to 0.17.7 (#3815).

Misc Fixes

  • When using the sandbox.sh script, always wait for /app frontend to be open on all nodes before marking the service as open (#3779).
  • Snapshot generation no longer causes a node crash if the snapshot is larger than the ring buffer message size (memory.max_msg_size). Instead, the generation of the large snapshot is skipped (#3603).

2.0.0-rc9

Fixed

  • Fixed an issue where new node started without a snapshot would be able to join from a node that started with a snapshot (#3573).
  • Fixed consensus issue where a node would grant its vote even though it already knew about the current primary (#3810).
  • Fixed issue with JSON configuration for cchost where extra fields were silently ignored rather than being rejected at startup (#3816).

Changed

  • Upgraded Open Enclave to 0.17.7 (#3815).
  • CCF Docker images are now available through Azure Container Registry rather than Docker Hub (#3821).
    • The ccfciteam/ccf-app-run image is now available at ccfmsrc.azurecr.io/ccf-sgx-app-run.
    • The ccfciteam/ccf-app-ci image is now available at ccfmsrc.azurecr.io/ccf-sgx-app-dev.
  • Added support for ciphers 'ECDHE-RSA-AES256-GCM-SHA384' and 'ECDHE-RSA-AES128-GCM-SHA256' when using TLS 1.2 (#3822).

2.0.0-rc8

Fixed

  • When using the sandbox.sh script, always wait for /app frontend to be open on all nodes before marking the service as open (#3779).

Changed

  • Every leaf in the Merkle Tree, and every receipt now includes a claims digest (#3606).

Added

  • Primary node now also reports time at which the ack from each backup node was last received (GET /node/consensus endpoint). This can be used by operators to detect one-way partitions between the primary and backup nodes (#3769).
  • Current receipt format is now exposed to C++ applications as ccf::Receipt, retrieved from describe_receipt_v2. Note that the previous JSON format is still available, but must be retrieved as a JSON object from describe_receipt_v1.

2.0.0-rc7

Fixed

  • Fixed issue with incorrect node and service certificate validity period when starting node in non-GMT timezone (#3732).
  • Fixed issue with self-signed node certificates that are now renewed when the set_node_certificate_validity proposal is applied (#3767).

Changed

  • Configurations and proposals now accept more date/time formats, including the Python-default ISO 8601 format (#3739).

Added

  • Added new GET /node/self_signed_certificate endpoint to retrieve the self-signed certificate of the target node (#3767).

2.0.0-rc6

Changed

  • host_processes_interface.h is now a public header, accessible under ccf/node/host_processes_interface.h.

2.0.0-rc5

Changed

  • Nodes now have a free-form node_data field, to match users and members. This can be set when the node is launched, or modified by governance. It is intended to store correlation IDs describing the node's deployment, such as a VM name or Pod identifier (#3662).
  • New GET /node/consensus endpoint now also returns primary node ID and current view (#3666).
  • The enclave:: namespace has been removed, and all types which were under it are now under ccf::. This will affect any apps using enclave::RpcContext, which should be replaced with ccf::RpcContext (#3664).
  • HTTP parsing errors are now recorded per-interface and returned by GET /node/metrics (#3671).
  • The kv::Store type is no longer visible to application code, and is replaced by a simpler kv::ReadOnlyStore. This is the interface given to historical queries to access historical state and enforces read-only access, without exposing internal implementation details of the store. This should have no impact on JS apps, but C++ apps will need to replace calls to store->current_txid() with calls to store->get_txid(), and store->create_tx() to store->create_read_only_tx().
  • Receipts now come with service endorsements of previous service identities after recoveries (#3679). See verify_receipt in e2e_logging.py for an example of how to verify the resulting certificate chain. This functionality is introduced in ccf::historical::adapter_v3.
  • Private headers have been moved to ccf/include/ccf/_private so they cannot be accidentally included from existing paths. Any applications relying on private headers should remove this dependence, or raise an issue to request the dependency be moved to the public API. In a future release private headers will be removed entirely from the installed package.

2.0.0-rc4

Added

  • Aside from regular release packages, CCF now also provides unsafe packages with verbose logging, helpful for troubleshooting. The extent of the logging in these builds make them fundamentally UNSAFE to use for production purposes, hence the name.

Changed

  • The transition_service_to_open governance proposal now requires the service identity as an argument to ensure the correct service is started. During recovery, it further requires the previous service identity to ensure the right service is recovered (#3624).

2.0.0-rc3

Fixed

  • Snapshot generation no longer causes a node crash if the snapshot is larger than the ring buffer message size (memory.max_msg_size). Instead, the generation of the large snapshot is skipped (#3603).

Changed

  • The C++ types used to define public governance tables are now exposed in public headers. Any C++ applications reading these tables should update their include paths (ie - #include "service/tables/nodes.h" => #include "ccf/service/tables/nodes.h") (#3608).
  • TxReceipt::describe() has been replaced with ccf::describe_receipt(). Includes of the private node/tx_receipt.h from C++ applications should be removed (#3610).
  • Python ccf.read_ledger and ccf.ledger_viz tools now accept paths to individual ledger chunks, to avoid parsing the entire ledger.

Added

  • New GET /gov/members endpoint which returns details of all members from the KV (#3615).
  • Add --insecure-skip-verification to ledger_viz utility, to allow visualisation of unverified ledger chunks (#3618).
  • Add --split-services to ledger_viz utility, to easily find out at which TxID new services were created (#3621).

2.0.0-rc2

Changed

  • The entry point for creation of C++ apps is now make_user_endpoints(). The old entry point get_rpc_handler() has been removed (#3562). For an example of the necessary change, see this diff of the logging sample app (#3562).
  • Failed recovery procedures no longer block subsequent recoveries: .recovery ledger files are now created while the recovery is in progress and ignored or deleted by nodes on startup (#3563).
  • Corrupted or incomplete ledger files are now recovered gracefully, until the last valid entry (#3585).
  • The CCF public API is now under include/ccf, and all application includes of framework code should use only these files.

Removed

  • get_node_state() is removed from AbstractNodeContext. The local node's ID is still available to endpoints as get_node_id(), and other subsystems which are app-visible can be fetched directly (#3552).

Fixed

  • Nodes no longer crash at start-up if the ledger in the read-only ledger directories (ledger.read_only_directories) is ahead of the ledger in the main ledger directory (ledger.directory) (#3597).

2.0.0-rc1

Added

  • The new endorsement configuration entry lets operators set the desired TLS certificate endorsement, either service-endorsed or node-endorsed (self-signed), for each network RPC interface of a node, defaulting to service-endorsed (#2875).
  • A new governance action trigger_ledger_chunk to request the creation of a ledger chunk at the next signature (#3519).
  • A new governance action trigger_snapshot to request the creation of a snapshot at the next signature (#3544).

Changed

  • Node RPC interfaces do not transition anymore from node-endorsed to service-endorsed TLS certificates but are fixed to a single configured type. While a given endorsement is not available yet (typically at start-up for service-endorsed certificates) the interface rejects TLS sessions instead of defaulting to a node-endorsed certificate (#2875).
  • Add request details with additional URL components to JS + TS API: request.url, request.route, request.method, request.hostname (#3498).
  • cchost can now run both SGX and virtual enclave libraries. cchost.virtual is no longer needed, and has been removed (#3476).

Dependencies

  • Upgraded Open Enclave to 0.17.6.

2.0.0-rc0

See documentation for code upgrade 1.x to 2.0 to upgrade an existing 1.x CCF service to 2.0

Developer API

C++

  • CCF is now built with Clang 10. It is strongly recommended that C++ applications upgrade to Clang 10 as well.

  • Raised the minimum supported CMake version for building CCF to 3.16 (#2946).

  • Removed mbedtls as cryptography and TLS library.

  • Added get_untrusted_host_time_v1 API. This can be used to retrieve a timestamp during endpoint execution, accurate to within a few milliseconds. Note that this timestamp comes directly from the host so is not trusted, and should not be used to make sensitive decisions within a transaction (#2550).

  • Added get_quotes_for_all_trusted_nodes_v1 API. This returns the ID and quote for all nodes which are currently trusted and participating in the service, for live audit (#2511).

  • Added get_metrics_v1 API to BaseEndpointRegistry for applications that do not make use of builtins and want to version or customise metrics output.

  • Added set_claims_digest() API to RpcContext, see documentation on how to use it to attach application-defined claims to transaction receipts.

  • Added indexing system to speed up historical queries (#3280, #3444).

  • ccf::historical::adapter_v2 now returns 404, with either TransactionPendingOrUnknown or TransactionInvalid, rather than 400 when a user performs a historical query for a transaction id that is not committed.

  • ccf::historical::AbstractStateCache::drop_requests() renamed to drop_cached_states() (#3187).

  • get_state_at() now returns receipts for signature transactions (#2785), see documentation for details.

Key-Value Store

  • Added kv::Value and kv::Set, as a more error-proof alternative to kv::Maps which had a single key or meaningless values (#2599).
  • Added foreach_key and foreach_value to C++ KV API, to iterate without deserializing both entries when only one is used (#2918).

JavaScript

  • Added JavaScript bytecode caching to avoid repeated compilation overhead. See the documentation for more information (#2643).

  • Added ccf.crypto.verifySignature() for verifying digital signatures to the JavaScript API (#2661).

  • Added experimental JavaScript API ccf.host.triggerSubprocess() (#2461).

  • ccf.crypto.verifySignature() previously required DER-encoded ECDSA signatures and now requires IEEE P1363 encoded signatures, aligning with the behavior of the Web Crypto API (#2735).

  • ccf.historical.getStateRange / ccf.historical.dropCachedStates JavaScript APIs to manually retrieve historical state in endpoints declared as "mode": "readonly" (#3033).

  • JavaScript endpoints with "mode": "historical" now expose the historical KV at ccf.historicalState.kv while ccf.kv always refers to the current KV state. Applications relying on the old behaviour should make their code forward-compatible before upgrading to 2.x with const kv = ccf.historicalState.kv || ccf.kv.

  • Receipts accessible through JavaScript no longer contain the redundant root hash field. Applications should be changed to not rely on this field anymore before upgrading to 2.x.

Governance

  • Updated actions.js constitution fragment to record service-endorsed node certificate on the transition_node_to_trusted action. The constitution must be updated using the existing set_constitution proposal (#2844).
  • The existing transition_node_to_trusted proposal action now requires a new valid_from argument (and optional validity_period_days, which defaults to the value of maximum_node_certificate_validity_days).
  • The proposal_generator has been removed from the ccf Python package. The majority of proposals can be trivially constructed in existing client tooling, without needing to invoke Python. This also introduces parity between the default constitution and custom constitution actions - all should be constructed and called from the same governance client code. Some jinja templates are included in samples/templates for constructing careful ballots from existing proposals.

Operations

cchost Configuration

  • Breaking change: Configuration for CCF node is now a JSON configuration file passed in to cchost via --config /path/to/config/file/ CLI argument. Existing CLI arguments have been removed. The migrate_1_x_config.py script (included in ccf Python package) should be used to migrate existing .ini configuration files to .json format (#3209).
  • Added support for listening on multiple interfaces for incoming client RPCs, with individual session caps (#2628).
  • The per-node session cap behaviour has changed. The network.rpc_interfaces.<interface_name>.max_open_sessions_soft is now a soft cap on the number of sessions. Beyond this, new sessions will receive a HTTP 503 error immediately after completing the TLS handshake. The existing hard cap (where sessions are closed immediately, before the TLS handshake) is still available, under the new argument network.rpc_interfaces.<interface_name>.max_open_sessions_hard (#2583).
  • Snapshot files now include receipt of evidence transaction. Nodes can now join or recover a service from a standalone snapshot file. 2.x nodes can still make use of snapshots created by a 1.x node, as long as the ledger suffix containing the proof of evidence is also specified at start-up (#2998).
  • If no node_certificate.subject_alt_names is specified at node start-up, the node certificate Subject Alternative Name extension now defaults to the value of published_address of the first RPC interface (#2902).

Certificate(s) Validity Period

  • Nodes certificates validity period is no longer hardcoded and must instead be set by operators and renewed by members (#2924):

    • The new node_certificate.initial_validity_days (defaults to 1 day) configuration entry lets operators set the initial validity period for the node certificate (valid from the current system time).
    • The new command.start.service_configuration.maximum_node_certificate_validity_days (defaults to 365 days) configuration entry sets the maximum validity period allowed for node certificates.
    • The new set_node_certificate_validity proposal action allows members to renew a node certificate (or set_all_nodes_certificate_validity equivalent action to renew all trusted nodes certificates).

  • Service certificate validity period is no longer hardcoded and must instead be set by operators and renewed by members (#3363):

    • The new service_certificate_initial_validity_days (defaults to 1 day) configuration entry lets operators set the initial validity period for the service certificate (valid from the current system time).
    • The new maximum_service_certificate_validity_days (defaults to 365 days) configuration entry sets the maximum validity period allowed for service certificate.
    • The new set_service_certificate_validity proposal action allows members to renew the service certificate.

Misc

  • The service certificate output by first node default name is now service_cert.pem rather than networkcert.pem (#3363).

  • Log more detailed errors on early startup (#3116).

  • Format of node output RPC and node-to-node addresses files is now JSON (#3300).

  • Joining nodes now present service-endorsed certificate in client TLS sessions after they have observed their own addition to the store, rather than as soon as they have joined the service. Operators should monitor the initial progress of a new node using its self-signed certificate as TLS session certificate authority (#2844).

  • Slow ledger IO operations will now be logged at level FAIL. The threshold over which logging will activate can be adjusted by the slow_io_logging_threshold configuration entry to cchost (#3067).

  • Added a new client_connection_timeout configuration entry to specify the maximum time a node should wait before re-establishing failed client connections. This should be set to a significantly lower value than consensus.election_timeout (#2618).

  • Nodes code digests are now extracted and cached at network join time in public:ccf.gov.nodes.info, and the GET /node/quotes and GET /node/quotes/self endpoints will use this cached value whenever possible (#2651).

  • DNS resolution of client connections is now asynchronous (#3140).

  • The curve-id selected for the identity of joining nodes no longer needs to match that of the network (#2525).

  • Removed long-deprecated --domain argument from cchost. Node certificate Subject Alternative Names should be passed in via existing node_certificate.subject_alt_names configuration entry (#2798).

  • Added experimental support for 2-transaction reconfiguration with CFT consensus, see documentation. Note that mixing 1tx and 2tx nodes in the same network is unsupported and unsafe at this stage (#3097).

Fixed

  • Fixed issue with ledger inconsistency when starting a new joiner node without a snapshot but with an existing ledger prefix (#3064).
  • Fixed issue with join nodes which could get stuck if an election was triggered while catching up (#3169).

Auditor

  • Receipts now include the endorsed certificate of the node, as well as its node id, for convenience (#2991).
  • Retired nodes are now removed from the store/ledger as soon as their retirement is committed (#3409).
  • Service-endorsed node certificates are now recorded in a new public:ccf.gov.nodes.endorsed_certificates table, while the existing cert field in the public:ccf.gov.nodes.info table is now deprecated (#2844).
  • New split_ledger.py utility to split existing ledger files (#3129).
  • Python ccf.read_ledger module now accepts custom formatting rules for the key and value based on the key-value store table name (#2791).
  • Ledger entries now contain a commit_evidence_digest, as well as an optional claims_digest when set_claims_digest() is used. The digest of the write set was previously the per-transaction leaf in the Merkle Tree, but is now combined with the digest of the commit evidence and optionally the user claims when present. Receipt verification instructions have been amended accordingly. The presence of commit_evidence in receipts serves two purposes: giving the user access to the TxID without having to parse the write set, and proving that a transaction has been committed by the service. Transactions are flushed to disk eagerly by the primary to keep in-enclave memory use to a minimum, so the existence of a ledger suffix is not on its own indicative of its commit status. The digest of the commit evidence is in the ledger to allow audit and recovery, but only the disclosure of the commit evidence indicates that a transaction has been committed by the service

Client API

  • Added support for TLS 1.3 (now used by default).

  • Added GET /gov/jwt_keys/all endpoint (#2519).

  • Added new operator RPC GET /node/js_metrics returning the JavaScript bytecode size and whether the bytecode is used (#2643).

  • Added a new GET /node/metrics endpoint which includes the count of active and peak concurrent sessions handled by the node (#2596).

  • Added endpoint to obtain service configuration via GET /node/service/configuration (#3251).

  • Added QuickJS version to RPC GET /node/version (#2643).

  • Added a GET /node/jwt_metrics endpoint to monitor attempts and successes of key refresh for each issuer. See documentation on how to use it.

  • Schema of GET /network/nodes/{node_id} and GET /network/nodes endpoints has been modified to include all RPC interfaces (#3300).

  • Improved performance for lookup of path-templated endpoints (#2918).

  • CCF now responds to HTTP requests that could not be parsed with a 400 response including error details (#2652).

  • Websockets endpoints are no longer supported. Usage is insufficient to justify ongoing maintenance.

  • The ccf Python package no longer provides utilities to issue requests to a running CCF service. This is because CCF supports widely-used client-server protocols (TLS, HTTP) that should already be provided by libraries for all programming languages. The ccf Python package can still be used to audit the ledger and snapshot files (#3386).

Dependencies

  • Upgraded Open Enclave to 0.17.5.

2.0.0-dev8

Added

  • Added set_claims_digest() API to RpcContext, see documentation on how to use it to attach application-defined claims to transaction receipts.
  • Added a GET /jwt_metrics endpoint to monitor attempts and successes of key refresh for each issuer. See documentation on how to use it.

Changed

  • Service certificate validity period is no longer hardcoded and can instead be set by operators and renewed by members (#3363):
    • The new service_certificate_initial_validity_days (defaults to 1 day) configuration entry lets operators set the initial validity period for the service certificate (valid from the current system time).
    • The new maximum_service_certificate_validity_days (defaults to 365 days) configuration entry sets the maximum validity period allowed for service certificate.
    • The new set_service_certificate_validity proposal action allows members to renew the service certificate.
  • Service certificate output by first node default name is now service_cert.pem rather than networkcert.pem (#3363).
  • Retired nodes are now removed from the store/ledger as soon as their retirement is committed (#3409).

Removed

  • The ccf Python package no longer provides utilities to issue requests to a running CCF service. This is because CCF supports widely-used client-server protocols (TLS, HTTP) that should already be provided by libraries for all programming languages. The ccf Python package can still be used to audit the ledger and snapshot files (#3386).
  • The proposal_generator has been removed from the ccf Python package. The majority of proposals can be trivially constructed in existing client tooling, without needing to invoke Python. This also introduces parity between the default constitution and custom constitution actions - all should be constructed and called from the same governance client code. Some jinja templates are included in samples/templates for constructing careful ballots from existing proposals.

2.0.0-dev7

Added

  • Added endpoint to obtain service configuration via /node/service/configuration (#3251)

Changed

  • Breaking change: Configuration for CCF node is now a JSON configuration file passed in to cchost via --config /path/to/config/file/ CLI argument. Existing CLI arguments have been removed. The migrate_1_x_config.py script (included in ccf Python package) should be used to migrate existing .ini configuration files to .json format (#3209).
  • Format of node output RPC and node-to-node addresses files is now JSON (#3300).
  • Schema of GET /network/nodes/{node_id} and GET /network/nodes endpoints has been modified to include all RPC interfaces (#3300).

Renamed

  • ccf::historical::AbstractStateCache::drop_requests() renamed to drop_cached_states() (#3187).

Dependency

  • Upgrade OpenEnclave from 0.17.2 to 0.17.5

2.0.0-dev6

Added

  • Added experimental support for 2-transaction reconfiguration with CFT consensus (#3097), see documentation. Note that mixing 1tx and 2tx nodes in the same network is unsupported and unsafe at this stage.

Changed

  • DNS resolution of client connections is now asynchronous.

Fixed

  • Fixed issue with join nodes which could get stuck if an election was triggered while catching up (#3169).

2.0.0-dev5

Added

  • Receipts now include the endorsed certificate of the node, as well as its node id, for convenience (#2991).
  • get_metrics_v1 API to BaseEndpointRegistry for applications that do not make use of builtins and want to version or customise metrics output.
  • Slow ledger IO operations will now be logged at level FAIL. The threshold over which logging will activate can be adjusted by the --io-logging-threshold CLI argument to cchost (#3067).
  • Snapshot files now include receipt of evidence transaction. Nodes can now join or recover a service from a standalone snapshot file. 2.x nodes can still make use of snapshots created by a 1.x node, as long as the ledger suffix containing the proof of evidence is also specified at start-up (#2998).
  • Nodes certificates validity period is no longer hardcoded and can instead be set by operators and renewed by members (#2924):
    • The new --initial-node-cert-validity-days (defaults to 1 day) CLI argument to cchost lets operators set the initial validity period for the node certificate (valid from the current system time).
    • The new --max-allowed-node-cert-validity-days (defaults to 365 days) CLI argument to cchost sets the maximum validity period allowed for node certificates.
    • The new set_node_certificate_validity proposal action allows members to renew a node certificate (or set_all_nodes_certificate_validity equivalent action to renew all trusted nodes certificates).
    • The existing transition_node_to_trusted proposal action now requires a new valid_from argument (and optional validity_period_days, which defaults to the value of ``--max-allowed-node-cert-validity-days`).
  • ccf.historical.getStateRange / ccf.historical.dropCachedStates JavaScript APIs to manually retrieve historical state in endpoints declared as "mode": "readonly" (#3033).
  • Log more detailed errors on early startup (#3116).
  • New split_ledger.py utility to split existing ledger files (#3129).

Changed

  • JavaScript endpoints with "mode": "historical" now expose the historical KV at ccf.historicalState.kv while ccf.kv always refers to the current KV state. Applications relying on the old behaviour should make their code forward-compatible before upgrading to 2.x with const kv = ccf.historicalState.kv || ccf.kv.

Removed

  • Receipts accessible through JavaScript no longer contain the redundant root hash field. Applications should be changed to not rely on this field anymore before upgrading to 2.x.

Fixed

  • Fixed issue with ledger inconsistency when starting a new joiner node without a snapshot but with an existing ledger prefix (#3064).

2.0.0-dev4

Added

  • Added foreach_key and foreach_value to C++ KV API, to iterate without deserializing both entries when only one is used (#2918).
  • ccf::historical::adapter_v2 now returns 404, with either TransactionPendingOrUnknown or TransactionInvalid, rather than 400 when a user performs a historical query for a transaction id that is not committed.

Changed

  • Service-endorsed node certificates are now recorded in a new public:ccf.gov.nodes.endorsed_certificates table, while the existing cert field in the public:ccf.gov.nodes.info table is now deprecated (#2844).
  • Joining nodes now present service-endorsed certificate in client TLS sessions after they have observed their own addition to the store, rather than as soon as they have joined the service. Operators should monitor the initial progress of a new node using its self-signed certificate as TLS session certificate authority (#2844).
  • Updated actions.js constitution fragment to record service-endorsed node certificate on the transition_node_to_trusted action. The constitution should be updated using the existing set_constitution proposal (#2844).
  • Improved performance for lookup of path-templated endpoints (#2918).
  • Raised the minimum supported CMake version for building CCF to 3.16 (#2946).

Dependency

  • Upgrade OpenEnclave from 0.17.1 to 0.17.2 (#2992)

2.0.0-dev3

Added

  • Added support for listening on multiple interfaces for incoming client RPCs, with individual session caps (#2628).

Changed

  • Upgrade OpenEnclave from 0.17.0 to 0.17.1.
  • get_state_at() now returns receipts for signature transactions (#2785), see documentation for details.
  • Upgrade playbooks and base CI image to Ubuntu 20.04. CCF is now primarily developed and tested against Ubuntu 20.04.
  • Python ccf.read_ledger module now accepts custom formatting rules for the key and value based on the key-value store table name (#2791).
  • CCF is now built with Clang 10. It is recommended that C++ applications upgrade to Clang 10 as well.
  • Internal /gov/jwt_keys/refresh endpoint has been moved to /node/jwt_keys/refresh (#2885).
  • If no --san is specified at node start-up, the node certificate Subject Alternative Name extension now defaults to the value of --public-rpc-address (#2902).

Removed

  • Remove long-deprecated --domain argument from cchost. Node certificate Subject Alternative Names should be passed in via existing --san argument (#2798).
  • Removed Forum sample app.

2.0.0-dev2

Changed

  • ccf.crypto.verifySignature() previously required DER-encoded ECDSA signatures and now requires IEEE P1363 encoded signatures, aligning with the behavior of the Web Crypto API (#2735).
  • Upgrade OpenEnclave from 0.16.1 to 0.17.0.

Added

  • Nodes code digests are now extracted and cached at network join time in public:ccf.gov.nodes.info, and the /node/quotes and /node/quotes/self endpoints will use this cached value whenever possible (#2651).

Removed

  • Websockets endpoints are no longer supported. Usage is insufficient to justify ongoing maintenance.

Bugfix

  • Fixed incorrect transaction view returned in x-ms-ccf-transaction-id HTTP response header after primary change (i.e. new view) (#2755).

2.0.0-dev1

Added

  • Added a new --client-connection-timeout-ms command line argument to cchost to specify the maximum time a node should wait before re-establishing failed client connections. This should be set to a significantly lower value than --raft-election-timeout-ms (#2618).
  • Add kv::Value and kv::Set, as a more error-proof alternative to kv::Maps which had a single key or meaningless values (#2599).
  • Added JavaScript bytecode caching to avoid repeated compilation overhead. See the documentation for more information (#2643).
  • Added new operator RPC /node/js_metrics returning the JavaScript bytecode size and whether the bytecode is used (#2643).
  • Added QuickJS version to RPC /node/version (#2643).
  • Added GET /gov/jwt_keys/all endpoint (#2519).
  • Added ccf.crypto.verifySignature() for verifying digital signatures to the JavaScript API (#2661).

Changed

  • CCF now responds to HTTP requests that could not be parsed with a 400 response including error details (#2652).

2.0.0-dev0

Added

  • Added get_untrusted_host_time_v1 API. This can be used to retrieve a timestamp during endpoint execution, accurate to within a few milliseconds. Note that this timestamp comes directly from the host so is not trusted, and should not be used to make sensitive decisions within a transaction (#2550).
  • Added get_quotes_for_all_trusted_nodes_v1 API. This returns the ID and quote for all nodes which are currently trusted and participating in the service, for live audit (#2511).
  • Added node start-up check for cchost and enclave compatibility, which should both always be from the same release for a single node (#2532).
  • Added a new /node/version endpoint to return the CCF version of a node (#2582).
  • Added a new /node/metrics endpoint which includes the count of active and peak concurrent sessions handled by the node (#2596).
  • Added experimental JavaScript API ccf.host.triggerSubprocess() (#2461).

Changed

  • The curve-id selected for the identity of joining nodes no longer needs to match that of the network (#2525).
  • The per-node session cap behaviour has changed. The --max-open-sessions is now a soft cap on the number of sessions. Beyond this, new sessions will receive a HTTP 503 error immediately after completing the TLS handshake. The existing hard cap (where sessions are closed immediately, before the TLS handshake) is still available, under the new argument --max-open-sessions-hard (#2583).
  • Requests with a url-encoded query string are now forwarded correctly from backups to the primary (#2587).
  • Signed requests with a url-encoded query string are now handled correctly rather than rejected (#2592).
  • Fixed consistency issue between ledger files on different nodes when snapshotting is active (#2607).

Dependency

  • Upgrade OpenEnclave from 0.15.0 to 0.16.1 (#2609)

1.0.4

Changed

  • CCF now responds to HTTP requests that could not be parsed with a 400 response including error details (#2652).

1.0.3

Dependency

  • Upgrade OpenEnclave from 0.15.0 to 0.16.1 (#2609)

1.0.2

Bugfix

  • Fixed consistency issue between ledger files on different nodes when snapshotting is active (#2607).

1.0.1

Bugfix

  • Requests with a url-encoded query string are now forwarded correctly from backups to the primary (#2587).
  • Signed requests with a url-encoded query string are now handled correctly rather than rejected (#2592).

1.0.0

The Confidential Consortium Framework CCF is an open-source framework for building a new category of secure, highly available, and performant applications that focus on multi-party compute and data.

This is the first long term support release for CCF. The 1.0 branch will only receive security and critical bug fixes, please see our release policy for more detail.

Active development will continue on the main branch, and regular development snapshots including new features will continue to be published.

Browse our documentation to get started with CCF, or open a discussion on GitHub if you have any questions.

1.0.0-rc3

Changed

  • Rename Store::commit_version() to the more accurate Store::compacted_version() (#1355).

1.0.0-rc2

Changed

  • Adjust release pipeline to cope with GitHub renaming debian packages containing tildes.

1.0.0-rc1

Changed

  • By default, CCF is now installed under /opt/ccf rather than /opt/ccf-x.y.z.

0.99.4

Fixed

  • Fixed use of --curve-id argument to cchost, which can now start a network with both node and service identities using curve secp256r1 (#2516).

0.99.3

Added

  • kv::MapHandle::size() can be used to get the number of entries in a given map.
  • kv::MapHandle::clear() can be used to remove all entries from a map.

0.99.2

Changed

  • The default constitution no longer contains set_service_principal or remove_service_principal since they are not used by the core framework. Instead any apps which wish to use these tables should add them to their own constitution. A sample implementation is available, and used in the CI tests.
  • Proposal status now includes a final_votes and vote_failures map, recording the outcome of each vote per member. failure_reason and failure_trace have been consolidated into a single failure object, which is also used for vote_failures.

0.99.1

Added

  • The service certificate is now returned as part of the /node/network/ endpoint response (#2442).

Changed

  • kv::Map is now an alias to kv::JsonSerialisedMap, which means all existing applications using kv::Maps will now require DECLARE_JSON... macros for custom key and value types. msgpack-c is no longer available to apps and MSGPACK_DEFINE macros should be removed. Note that this change may affect throughput of existing applications, in which case an app-defined serialiser (or kv::RawCopySerialisedMap) should be used (#2449).
  • /node/state endpoint also returns the seqno at which a node was started (i.e. seqno of the snapshot a node started from or 0 otherwise) (#2422).

Removed

  • /gov/query and /gov/read governance endpoints are removed (#2442).
  • Lua governance is removed. JS_GOVERNANCE env var is no longer necessary, and JS constitution is the only governance script which must be provided and will be used by the service. --gov-script can no longer be passed to cchost or sandbox.sh.

0.99.0

This is a bridging release to simplify the upgrade to 1.0. It includes the new JS constitution, but also supports the existing Lua governance so that users can upgrade in 2 steps - first implementing all of the changes below with their existing Lua governance, then upgrading to the JS governance. Lua governance will be removed in CCF 1.0. See temporary docs for help with transitioning from Lua to JS.

The 1.0 release will require minimal changes from this release.

Added

  • A new read_ledger.py Python command line utility was added to parse and display the content of a ledger directory.
  • ccf-app npm package to help with developing JavaScript and TypeScript CCF apps. See docs for further details (#2331).

Changed

  • Retired members are now deleted from the store, instead of being marked as Retired (#1401).
  • retire_member proposal has been renamed to remove_member and is now idempotent (i.e. succeeds even if the member was already removed) (#1401).
  • accept_recovery and open_network proposals have been merged into a single idempotent transition_service_to_open proposal (#1791).
  • The /tx endpoint now takes a single transaction_id query parameter. For example, rather than calling /node/tx?view=2&seqno=42, call /node/tx?transaction_id=2.42.
  • The /commit endpoint now returns a response with a single transaction_id rather than separate view and seqno fields.
  • UserRpcFrontend has been removed, and the return value of get_rpc_handler which apps should construct is now simply a ccf::RpcFrontend.
  • There is now a distinction between public and private headers. The public headers under include/ccf are those we expect apps to use, and others are implementation details which may change/be deprecated/be hidden in future. Most apps should now be including "ccf/app_interface.h" and "ccf/common_endpoint_registry.h".
  • Various endpoint-related types have moved under the ccf::endpoints namespace. Apps will need to rename these types where they are not using auto, for instance to ccf::endpoints::EndpointContext and ccf::endpoints::ForwardingRequired.
  • Ledger entry frames are no longer serialised with msgpack (#2343).
  • In JavaScript apps, the field caller.jwt.key_issuer in the Request object has been renamed caller.jwt.keyIssuer (#2362).
  • The proposals set_module, remove_module and set_js_app have been removed and deploy_js_app renamed to set_js_app (#2391).

0.19.3

Changed

  • The status filter passed to /node/network/nodes now takes the correct CamelCased values (#2238).

0.19.2

Added

  • New get_user_data_v1 and get_member_data_v1 C++ API calls have been added to retrieve the data associated with users/members. The user/member data is no longer included in the AuthnIdentity caller struct (#2301).
  • New get_user_cert_v1 and get_member_cert_v1 C++ API calls have been added to retrieve the PEM certificate of the users/members. The user/member certificate is no longer included in the AuthnIdentity caller struct (#2301).

Changed

  • String values in query parameters no longer need to be quoted. For instance, you should now call /network/nodes?host=127.0.0.1 rather than /network/nodes?host="127.0.0.1" (#2309).
  • Schema documentation for query parameters should now be added with add_query_parameter, rather than set_auto_schema. The In type of set_auto_schema should only be used to describe the request body (#2309).
  • json_adapter will no longer try to convert query parameters to a JSON object. The JSON passed as an argument to these handlers will now be populated only by the request body. The query string should be parsed separately, and http::parse_query(s) is added as a starting point. This means strings in query parameters no longer need to be quoted (#2309).
  • Enum values returned by built-in REST API endpoints are now PascalCase. Lua governance scripts that use enum values need to be updated as well, for example, "ACTIVE" becomes "Active" for member info. The same applies when using the /gov/query endpoint (#2152).
  • Most service tables (e.g. for nodes and signatures) are now serialised as JSON instead of msgpack. Some tables (e.g. user and member certificates) are serialised as raw bytes for performance reasons (#2301).
  • The users and members tables have been split into public:ccf.gov.users.certs/public:ccf.gov.users.info and public:ccf.gov.members.certs/public:ccf.gov.members.encryption_public_keys/public:ccf.gov.members.info respectively (#2301).
  • TypeScript interface/class names have been renamed to PascalCase (#2325).

0.19.1

Added

  • Historical point query support has been added to JavaScript endpoints (#2285).
  • RSA key generation JavaScript endpoint (#2293).

Changed

  • "readonly" has been replaced by "mode" in app.json in JavaScript apps (#2285).

0.19.0

Changed

  • x-ccf-tx-view and x-ccf-tx-seqno response headers have been removed, and replaced with x-ms-ccf-transaction-id. This includes both original fields, separated by a single .. Historical queries using ccf::historical::adapter should also pass a single combined x-ms-ccf-transaction-id header (#2257).
  • Node unique identifier is now the hex-encoded string of the SHA-256 digest of the node's DER-encoded identity public key, which is also used as the node's quote report data. The sandbox.sh script still uses incrementing IDs to keep track of nodes and for their respective directories (#2241).
  • Members and users unique identifier is now the hex-encoded string of the SHA-256 digest of their DER-encoded identity certificate (i.e. fingerprint), which has to be specified as the keyId field for signed HTTP requests (#2279).
  • The receipt interface has changed, /app/receipt?commit=23 is replaced by /app/receipt?transaction_id=2.23. Receipt fetching is now implemented as a historical query, which means that the first reponse(s) may be 202 with a Retry-After header. Receipts are now structured JSON, as opposed to a flat byte sequence, and /app/receipt/verify has been removed in favour of an offline verification sample.
  • ccfapp::get_rpc_handler() now takes a reference to a ccf::AbstractNodeContext rather than ccf::AbstractNodeState. The node state can be obtained from the context via get_node_state().

Removed

  • get_receipt_for_seqno_v1 has been removed. Handlers wanting to return receipts must now use the historical API, and can obtain a receipt via ccf::historical::StatePtr. See the historical query with receipt sample for reference.
  • caller_id endpoint has been removed. Members and users can now compute their unique identifier without interacting with CCF (#2279).
  • public:ccf.internal.members.certs_der, public:ccf.internal.users.certs_der, public:ccf.internal.members.digests and public:ccf.internal.users.digests KV tables have been removed (#2279).
  • view_change_in_progress field in network/status response has been removed (#2288).

0.18.5

Changed

  • Historical query system now supports range queries.

0.18.4

Changed

  • Governance proposals can be submitted successfully against secondaries (#2247)
  • set_ca_cert/remove_ca_cert proposals have been renamed set_ca_cert_bundle/remove_ca_cert_bundle and now also accept a bundle of certificates encoded as concatenated PEM string (#2221). The ca_cert_name parameter to the set_jwt_issuer proposal has been renamed to ca_cert_bundle_name.

Added

  • Support for multiple key wrapping algorithms for C++ and JavaScript applications (#2246)

0.18.3

Changed

  • Fixed format of notBefore and notAfter in node and network certificates (#2243).
  • CCF now depends on Open Enclave 0.14.

0.18.2

Added

  • Support for historical queries after ledger rekey and service recovery (#2200).

Changed

  • CCF now supports OpenSSL for many crypto tasks like hashing, signing, and signature verification (#2123).
  • In progress ledger files no longer cause a node to crash when they are committed (#2209).

0.18.1

Changed

  • "id" field in state endpoint response has been renamed to "node_id" (#2150).
  • user_id endpoint is renamed caller_id (#2142).
  • Nodes' quotes format updated to Open Enclave's SGX_ECDSA. Quote endorsements are also stored in CCF and can be retrieved via the quotes/self and quotes endpoints (#2161).
  • get_quote_for_this_node_v1() takes a QuoteInfo structure (containing the format, raw quote and corresponding endorsements) as out parameter instead of the distinct format and raw quote as two out paramters (#2161).
  • Several internal tables are renamed (#2166).
  • /node/network/nodes correctly returns all nodes if no filter is specified (#2188).

0.18.0

Changed

  • endpoint_metrics is renamed api/metrics and now returns an array of objects instead of nested path/method objects (#2068).
  • Governance proposal ids are now digests of the proposal and store state observed during their creation, hex-encoded as strings. This makes votes entirely specific to an instance of a proposal without having to include a nonce. (#2104, #2135).
  • quote endpoint has been renamed to quotes/self (#2149).
  • TxViews have been renamed to MapHandles, to clearly distinguish them from consensus views. Calls to tx.get_view must be replaced with tx.rw.
  • tx.rw does not support retrieving multiple views in a single call. Instead of auto [view1, view2] = tx.get_view(map1, map2);, you must write auto handle1 = tx.rw(map1); auto handle2 = tx.rw(map2);.

Added

  • Added get_version_of_previous_write(const K& k) to MapHandle. If this entry was written to by a previous transaction, this returns the version at which that transaction was applied. See docs for more details.

Removed

  • The x-ccf-global-commit header is no longer sent with responses (#1586, #2144). This was a hint of global commit progress, but was known to be imprecise and unrelated to the executed transaction. Instead, clients should call /commit to monitor commit progress or /tx for a specific transaction.

0.17.2

Fixed

  • Fixed incorrect ledger chunking on backup nodes when snapshotting is enabled (#2110).

0.17.1

Changed

  • JS endpoints now list their auth policies by name, similar to C++ endpoints. The fields require_client_identity, require_client_signature, and require_jwt_authentication are removed, and should be replaced by authn_policies. For example, the previous default "require_client_identity": true should be replaced with "authn_policies": ["user_cert"], an endpoint which would like to handle a JWT but will also accept unauthenticated requests would be "authn_policies": ["jwt", "no_auth"], and a fully unauthenticated endpoint would be "authn_policies": []. See docs for further detail.

0.17.0

Added

  • Versioned APIs for common CCF functionality: get_status_for_txid_v1, get_last_committed_txid_v1, generate_openapi_document_v1, get_receipt_for_seqno_v1, get_quote_for_this_node_v1. We will aim to support these function signatures long-term, and provide similar functionality with incremental version bumps when this is no longer possible. In particular, this enables building an app which does not expose the default endpoints but instead exposes similar functionality through its own API.

Changed

  • /network, /network_info, /node/ids, /primary_info have been restructured into /network, /network/nodes, /network/nodes/{id}, /network/nodes/self, /network/nodes/primary while also changing the response schemas (#1954).
  • /ack responds with HTTP status 204 now instead of 200 and true as body (#2088).
  • /recovery_share has new request and response schemas (#2089).

0.16.3

Changed

  • To avoid accidentally unauthenticated endpoints, a vector of authentication policies must now be specified at construction (as a new argument to make_endpoint) rather than by calling add_authentication. The value ccf::no_auth_required must be used to explicitly indicate an unauthenticated endpoint.
  • All /gov endpoints accept signature authentication alone correctly, regardless of session authentication.
  • ccf.CCFClient now allows separate session_auth and signing_auth to be passed as construction time. ccf.CCFClient.call() no longer takes a signed argument, clients with a signing_auth always sign. Similarly, the disable_session_auth constructor argument is removed, the same effect can be achieved by setting session_auth to None.

0.16.2

Changed

  • Snapshots are generated by default on the current primary node, every 10,000 committed transaction (#2029).
  • Node information exposed in the API now correctly reports the public port when it differs from the local one. (#2001)
  • All /gov endpoints accept signature authentication again. Read-only /gov endpoints had been incorrectly changed in 0.16.1 to accept session certification authentication only (#2033).

0.16.1

Added

  • C++ endpoints can be omitted from OpenAPI with set_openapi_hidden(true) (#2008).
  • JS endpoints can be omitted from OpenAPI if the "openapi_hidden" field in app.json is true (#2008).

Changed

  • Error responses of built-in endpoints are now JSON and follow the OData schema (#1919).
  • Code ids are now deleted rather than marked as RETIRED. ACTIVE is replaced with the more precise ALLOWED_TO_JOIN (#1996).
  • Authentication policies can be specified per-endpoint with add_authentication. Sample policies are implemented which check for a user TLS handshake, a member TLS handshake, a user HTTP signature, a member HTTP signature, and a valid JWT. This allows multiple policies per-endpoints, and decouples auth from frontends - apps can define member-only endpoints (#2010).
  • By default, if no authentication policy is specified, endpoints are now unauthenticated and accessible to anyone (previously the default was user TLS handshakes, where the new default is equivalent to set_require_client_identity(false)).
  • CCF now depends on Open Enclave 0.13.

Removed

  • The methods Endpoint::set_require_client_signature, Endpoint::set_require_client_identity and Endpoint::set_require_jwt_authentication are removed, and should be replaced by calls to add_authentication. For unauthenticated endpoints, either add no policies, or add the built-in empty_auth policy which accepts all requests.
    • .set_require_client_signature(true) must be replaced with .add_authentication(user_signature_auth_policy)
    • .set_require_client_identity(true) must be replaced with .add_authentication(user_cert_auth_policy)
    • .set_require_jwt_authentication(true) must be replaced with .add_authentication(jwt_auth_policy)

0.16.0

Added

  • CLI options are printed on every node launch (#1923).
  • JS logging sample app is included in CCF package (#1932).
  • C++ apps can be built using cmake's find_package(ccf REQUIRED) (see cmake sample) (#1947).

Changed

  • JWT signing keys are auto-refreshed immediately when adding a new issuer instead of waiting until the next auto-refresh event is due (#1978).
  • Snapshots are only committed when proof of snapshot evidence is committed (#1972).
  • Snapshot evidence must be validated before joining/recovering from snapshot (see doc) (#1925).

Fixed

  • Ledger index is recovered correctly even if --ledger-dir directory is empty (#1953).
  • Memory leak fixes (#1957, #1959, #1974, #1982).
  • Consensus fixes (#1977, #1981).
  • Enclave schedules messages in a fairer way (#1991).

Security

  • Hostname of TLS certificate is checked when auto-refreshing JWT signing keys (#1934).
  • Evercrypt update to 0.3.0 (#1967).

0.15.2

Added

  • JWT key auto-refresh (#1908), can be enabled by providing "auto_refresh": true and "ca_cert_name": "..." in set_jwt_issuer proposal.
    • Auto-refresh is currently only supported for providers following the OpenID Connect standard where keys are published under the /.well-known/openid-configuration path of the issuer URL.
    • ca_cert_name refers to a certificate stored with a set_ca_cert proposal and is used to validate the TLS connection to the provider endpoint.
  • JWT signature validation (#1912), can be enabled with the require_jwt_authentication endpoint property.

Changed

  • Members can no longer vote multiple times on governance proposals (#1743).
  • update_ca_cert proposal has been replaced by set_ca_cert/remove_ca_cert (#1917).

Deprecated

  • set_js_app proposal and --js-app-script argument are deprecated, and should be replaced by deploy_js_app and --js-app-bundle. See #1895 for an example of converting from the old style (JS embedded in a Lua script) to the new style (app bundle described by app.json).

Removed

  • kv::Store::create is removed.
  • luageneric is removed.

0.15.1

Added

Changed

  • /gov/ack/update_state_digest and /gov/ack now only return/accept a hex string (#1873).
  • /node/quote schema update (#1885).
  • AFT consensus improvements (#1880, #1881).

0.15.0

Added

  • Support for non-recovery members: only members with an associated public encryption key are handed recovery shares (#1866).
  • AFT consensus verify entry validity (#1864).
  • JWT validation in forum sample app (#1867).
  • JavaScript endpoints OpenAPI definition is now included in /api (#1874).

Changed

  • The keyId field in the Authorization header must now be set to the hex-encoded SHA-256 digest of the corresponding member certificate encoded in PEM format. The scurl.sh script and Python client have been modified accordingly. scurl.sh can be run with DISABLE_CLIENT_AUTH=1 (equivalent disable_client_auth=False argument to Python client) to issue signed requests without session-level client authentication (#1870).
  • Governance endpoints no longer require session-level client authentication matching a member identity, the request signature now serves as authentication. The purpose of this change is to facilitate member key storage in systems such as HSMs (#1870).
  • Support for hs2019 scheme for HTTP signatures (#1872).
    • ecdsa-sha256 scheme will be deprecated in the next release.

0.14.3

Added

  • Added support for storing JWT public signing keys (#1834).
    • The new proposals set_jwt_issuer, remove_jwt_issuer, and set_jwt_public_signing_keys can be generated with the latest version of the ccf Python package.
    • sandbox.sh has a new --jwt-issuer <json-path> argument to easily bootstrap with an initial set of signing keys using the set_jwt_issuer proposal.
    • See tests/npm-app/src/endpoints/jwt.ts for validating tokens received in the Authorization HTTP header in TypeScript.
    • Includes special support for SGX-attested signing keys as used in MAA.

Changed

  • CCF now depends on Open Enclave 0.12 (#1830).
  • /app/user_id now takes {"cert": user_cert_as_pem_string} rather than {"cert": user_cert_as_der_list_of_bytes} (#278).
  • Members' recovery shares are now encrypted using RSA-OAEP-256 (#1841). This has the following implications:
    • Network's encryption key is no longer output by the first node of a CCF service is no longer required to decrypt recovery shares.
    • The latest version of the submit_recovery_share.sh script should be used.
    • The latest version of the proposal_generator.py should be used (please upgrade the ccf Python package).
  • submit_recovery_share.sh script's --rpc-address argument has been removed. The node's address (e.g. https://127.0.0.1:8000) should be used directly as the first argument instead (#1841).
  • The constitution's pass function now takes an extra argument: proposer_id, which contains the member_id of the member who submitted the proposal. To adjust for this change, replace tables, calls, votes = ... with tables, calls, votes, proposer_id = ... at the beginning of the pass definition.
  • Bundled votes (ie. the ballot entry in POST /proposals) have been removed. Votes can either happen explicitly via POST /proposals/{proposal_id}/votes, or the constitution may choose to pass a proposal without separate votes by examining its contents and its proposer, as illustrated in the operating member constitution sample. The --vote-against flag in proposal_generator.py, has also been removed as a consequence.

Fixed

  • Added tools.cmake to the install, which ccf_app.cmake depends on and was missing from the previous release.

Deprecated

  • kv::Store::create is deprecated, and will be removed in a future release. It is no longer necessary to create a kv::Map from a Store, it can be constructed locally (kv::Map<K, V> my_map("my_map_name");) or accessed purely by name (auto view = tx.get_view<K, V>("my_map_name");) (#1847).

0.14.2

Changed

  • The start_test_network.sh script has been replaced by sandbox.sh. Users wishing to override the default network config (a single node on '127.0.0.1:8000') must now explictly specify if they should be started locally (eg. -n 'local://127.4.4.5:7000') or on remote machine via password-less ssh (eg. -n 'ssh://10.0.0.1:6000').
  • node/quote endpoint now returns a single JSON object containing the node's quote (#1761).
  • Calling foreach on a TxView now iterates over the entries which previously existed, ignoring any modifications made by the functor while iterating.
  • JS: ccf.kv.<map>.get(key) returns undefined instead of throwing an exception if key does not exist.
  • JS: ccf.kv.<map>.delete(key) returns false instead of throwing an exception if key does not exist, and true instead of undefined otherwise.
  • JS: ccf.kv.<map>.set(key, val) returns the map object instead of undefined.

0.14.1

Added

  • /node/memory endpoint exposing the maximum configured heap size, peak and current used sizes.

Changed

  • Public tables in the KV must now indicate this in their name (with a public: prefix), and internal tables have been renamed. Any governance or auditing scripts which operate over internal tables must use the new names (eg - ccf.members is now public:ccf.gov.members).
  • --member-info on cchost can now take a third, optional file path to a JSON file containing additional member data (#1712).

Removed

  • /api/schema endpoints are removed, as the same information is now available in the OpenAPI document at /api.

Deprecated

  • Passing the SecurityDomain when creating a KV map is deprecated, and will be removed in a future release. This should be encoded in the table's name, with a public: prefix for public tables.

0.14.0

Added

  • Nodes can recover rapidly from a snapshot, rather than needing to reprocess an entire ledger (#1656)
  • Python client code wraps creation and replacement of an entire JS app bundle in a single operation (#1651)
  • Snapshots are only usable when the corresponding evidence is committed (#1668).
  • JSON data associated to each consortium member to facilitate flexible member roles (#1657).

Changed

  • /api endpoints return an OpenAPI document rather than a custom response (#1612, #1664)
  • Python ledger types can process individual chunks as well as entire ledger (#1644)
  • POST recovery_share/submit endpoint is renamed to POST recovery_share (#1660).

Fixed

  • Elections will not allow transactions which were reported as globally committed to be rolled back (#1641)

Deprecated

  • lua_generic app is deprecated and will be removed in a future release. Please migrate old Lua apps to JS

0.13.4

Changed

  • Fixed infinite memory growth issue (#1639)
  • Step CLI updated to 0.15.2 (#1636)

0.13.3

Added

  • Sample TypeScript application (#1614, #1596)

Changed

  • Handlers can implement custom authorisation headers (#1203, #1563)
  • Reduced CPU usage when nodes are idle (#1625, #1626)
  • Upgrade to Open Enclave 0.11 (#1620, #1624)
  • Snapshots now include view history, so nodes resuming from snapshots can accurately serve transaction status requests (#1616)
  • Request is now passed as an argument to JavaScript handlers (#1604), which can return arbitrary content types (#1575)
  • Quote RPC now returns an error when the quote cannot be found (#1594)
  • Upgraded third party dependencies (#1589, #1588, #1576, #1572, #1573, #1570, #1569)
  • Consensus types renamed from raft and pbft to cft and bft (#1591)

Removed

  • Notification server (#1582)

0.13.2

Added

  • retire_node_code proposal (#1558)
  • Ability to update a collection of JS modules in a single proposal (#1557)

0.13.1

Fixed

  • Handle setting multiple subject alternative names correctly in node certificate (#1552)
  • Fix host memory check on startup ecall (#1553)

0.13.0

Added

  • Experimental

    • New CCF nodes can now join from a snapshot (#1500, #1532)
    • New KV maps can now be created dynamically in a transaction (#1507, #1528)

  • CLI

    • Subject Name and Subject Alternative Names for the node certificates can now be passed to cchost using the --sn and --san CLI switches (#1537)
    • Signature and ledger splitting flags have been renamed more accurately (#1534)

  • Governance

    • user_data can be set at user creation, as well as later (#1488)

  • Javascript

    • js_generic endpoints are now modules with a single default call. Their dependencies can be stored in a separate table and loaded with import. (#1469, #1472, #1481, #1484)

Fixed

  • Retiring the primary from a network is now correctly handled (#1522)

Deprecated

  • CLI
    • --domain=... is superseded by --san=dNSName:... and will be removed in a future release

Removed

  • API
    • Removed redirection from legacy frontend names (members -> gov, nodes -> node, users -> app) (#1543)
    • Removed old install() API, replaced by make_endpoint() in 0.11.1 (#1541)

0.12.2

Fixed

  • Fix published containers

0.12.1

Changed

  • Release tarball replaced by a .deb

Fixed

  • Fix LVI build for applications using CCF (#1466)

0.12.0

Added

  • Tooling
    • New Python proposal and vote generator (#1370). See docs.
    • New CCF tools Python package for client, ledger parsing and member proposal/vote generation (#1429, #1435). See docs.
  • HTTP endpoints
    • Templated URI for HTTP endpoints (#1384, #1393).
    • New remove_user proposal (#1379).
    • New node endpoints: /node/state and /node/is_primary (#1387, #1439)
    • New metrics endpoint (#1422).

Changed

  • Tooling
    • Updated version of Open Enclave (0.10) (#1424). Users should use the Intel PSW tested with Open Enclave 0.10, see Open Enclave releases notes: https://github.com/openenclave/openenclave/releases/tag/v0.10.0 for more details.
    • CCF releases no longer include a build of Open Enclave, instead the upstream binary release should be used. Playbooks and containers have been updated accordingly (#1437).
    • CCF is now built with LVI mitigations (#1427). CCF should now be built with a new LVI-enabled toolchain, available via CCF playbooks and containers.
    • Updated version of snmalloc (#1391).
  • HTTP endpoints
    • Pass PEM certificates rather than byte-arrays (#1374).
    • Member /ack schema (#1395).
    • Authorisation HTTP request header now accepts unquoted values (#1411).
    • Fix double opening of /app on backups after recovery (#1445).
  • Other
    • Merkle tree deserialisation fix (#1363).
    • Improve resilience of node-to-node channels (#1371).
    • First Raft election no longer fails (#1392).
    • Fix message leak (#1442).

Removed

  • mkSign endpoint (#1398).

0.11.7

Changed

  1. Fix a bug that could cause signatures not to be recorded on transactions hitting conflicts (#1346)
  2. Fix a bug that could allow transactions to be executed by members before a recovered network was fully opened (#1347)
  3. Improve error reporting on transactions with invalid signatures (#1356)

Added

  1. All format and linting checks are now covered by scripts/ci-checks.sh (#1359)
  2. node/code RPC returns all code versions and their status (#1351)

0.11.4

Changed

  • Add clang-format to the application CI container, to facilitate application development (#1340)
  • Websocket handlers are now distinct, and can be defined by passing ws::Verb::WEBSOCKET as a verb to make_endpoint() (#1333)
  • Custom KV serialisation is documented

Fixed

  • Fix application runtime container, which had been missing a dependency in the previous release (#1340)

0.11.1

Added

  • CLI tool for managing recovery shares (#1295). usage
  • New standard endpoint node/ids for retrieving node ID from IP address (#1319).
  • Support for read-only transactions. Use tx.get_read_only_view to retrieve read-only views, and install with make_read_only_endpoint if all operations are read-only.
  • Support for distinct handlers on the same URI. Each installed handler/endpoint is now associated with a single HTTP method, so you can install different operations on POST /foo and GET /foo.

Changed

  • The frontend names, used as a prefix on all URIs, have been changed. Calls to /members/... or /users/... should be replaced with /gov/... and /app/... respectively. The old paths will return HTTP redirects in this release, but may return 404 in a future release (#1325).
  • App-handler installation API has changed. install(URI, FN, READWRITE) should be replaced with make_endpoint(URI, VERB, FN).install(). Existing apps should compile with deprecation warnings in this release, but the old API will be removed in a future release. See this diff of logging.cpp for an example of the required changes.
  • Improved quickstart documentation (#1298, #1316).
  • Member ACKs are required, even when the service is opening (#1318).
  • The naming scheme for releases has changed to be more consistent. The tags will now be in the form ccf-X.Y.Z.

0.11

Changed

  • KV reorganisation to enable app-defined serialisation (#1179, #1216, #1234)

kv.h has been split into multiple headers so apps may need to add includes for kv/store.h and kv/tx.h. The typedefs ccf::Store and ccf::Tx have been removed; apps should now use kv::Store and kv::Tx.

CCF now deals internally only with serialised data in its tables, mapping byte-vectors to byte-vectors. By default all tables will convert their keys and values to msgpack, using the existing macros for user-defined types. Apps may define custom serialisers for their own types - see kv/serialise_entry_json.h for an example.

  • Fixed issues that affected the accuracy of tx status reporting (#1157, #1150)
  • All RPCs and external APIs now use view and seqno to describe the components of a transaction ID, regardless of the specific consensus implementation selected (#1187, #1227)
  • Improved resiliency of recovery process (#1051)
  • foreach early-exit semantics are now consistent (#1222)
  • Third party dependency updates (#1144, #1148, #1149, #1151, #1155, #1255)
  • All logging output now goes to stdout, and can be configured to be either JSON or plain text (#1258) doc
  • Initial support for historical query handlers (#1207) sample
  • Implement the equivalent of "log rolling" for the ledger (#1135) doc
  • Internal RPCs renamed to follow more traditional REST conventions (#968) doc

Added

  • Support for floating point types in default KV serialiser (#1174)
  • The start_test_network.sh script now supports recovering an old network with the --recover flag (#1095) doc
  • Application CI and runtime containers are now available (#1178)
    1. ccfciteam/ccf-app-ci:0.11 is recommended to build CCF applications
    2. ccfciteam/ccf-app-run:0.11 is recommended to run CCF nodes, for example in k8s
  • Initial websockets support (#629) sample

Removed

  • ccf::Store and ccf::Tx typdefs, in favour of kv::Store and kv::Tx.

0.10

Added

  • Brand new versioned documentation: https://microsoft.github.io/CCF.
  • New /tx endpoint to check that a transaction is committed (#1111). See docs.
  • Disaster recovery is now performed with members key shares (#1101). See docs.
  • Open Enclave install is included in CCF install (#1125).
  • New sgxinfo.sh script (#1081).
  • New --transaction-rate flag to performance client (#1071).

Changed

  • CCF now uses Open Enclave 0.9 (#1098).
  • cchost's --enclave-type is release by default (#1083).
  • keygenerator.sh's --gen-key-share option renamed to --gen-enc-key to generate member encryption key (#1101).
  • Enhanced view change support for PBFT (#1085, #1087, #1092).
  • JavaScript demo logging app is now more generic (#1110).
  • Updated method to retrieve time in enclave from host (#1100).
  • Correct use of Everycrypt hashing (#1098).
  • Maximum number of active members is 255 (#1107).
  • Python infra: handle proposals correctly with single member (#1079).
  • Dependencies updates (#1080, #1082).

Removed

  • cchost no longer outputs a sealed secrets file to be used for recovery (#1101).

0.9.3

Added

  1. Install artifacts include virtual build (#1072)
  2. add_enclave_library_c is exposed in ccp_app.cmake (#1073)

0.9.2

Added

  • Handlers can decide if transaction writes are applied independently from error status (#1054)
  • Scenario Perf Client is now part of the CCF install to facilitate performance tests (#1058)

Changed

  • Handle writes when host is reconnecting (#1038)
  • Member tables are no longer whitelisted for raw_puts (#1041)
  • Projects including CCF's CMake files now use the same build type default (#1057)

0.9.1

Added

0.9

This pre-release improves support for handling HTTP requests.

Added

  • Key shares will be accepted after multiple disaster recovery operations (#992).
  • HTTP response headers and status can be set directly from handler (#921, #977).
  • Handlers can be restricted to accept only specific HTTP verbs (#966).
  • Handlers can accept requests without a matching client cert (#962).
  • PBFT messages are authenticated by each receiving node (#947).
  • snmalloc can be used as allocator (#943, #990).
  • Performance optimisations (#946, #971).
  • Install improvements (#983, #986).

Changed

  • HTTP request and responses no longer need to contain JSON-RPC objects (#930, #977).
  • Files and binaries have been renamed to use a consistent lower_snake_case (#989). Most app includes should be unaffected, but users of the luageneric app should now look for lua_generic.
  • Threading support relies on fixes from a recent build of OE (#990). Existing machines should re-run the ansible playbooks to install the current dependencies.
  • Consensus is chosen at run-time, rather than build-time (#922).
  • API for installing handlers has changed (#960). See the logging app or documentation for the current style.
  • Several standard endpoints are now GET-only, and must be passed a URL query (ie GET /users/getCommit?id=42).

0.8.2

Changed

  • CCF install can now be installed anywhere (#950).
  • PBFT messages are now authenticated (#947).
  • Miscellaneous performance improvements (#946).

0.8.1

Added

  • PBFT timers can be set fromcchost CLI (#929). See docs.
  • Nodes output their PID in a cchost.pid file on start-up (#927).
  • (Experimental) Members can retrieve their decrypted recovery shares via getEncryptedRecoveryShare and submit the decrypted share via submitRecoveryShare (#932).

Changed

  • App handlers should set HTTP response fields instead of custom error codes (#921). See docs.
  • Single build for Raft and PBFT consensuses (#922, #929, #935).
  • Members' proposals are forever rejected if they fail to execute (#930).
  • Original consortium members can ACK (#933).
  • PBFT performance improvements (#940, #942).
  • PBFT ledger private tables are now encrypted (#939).

0.8

This pre-release enables experimental support for running CCF with the PBFT consensus protocol. In providing an experimental release of CCF with PBFT we hope to get feedback from early adopters.

Added

  • Experimental PBFT support docs
  • Increased threading support docs (#831, #838)
  • Governance proposals can now be rejected, which allows constitutions to implement veto power (#854)
  • Support for non JSON-RPC payloads (#852)
  • RPC to get the OE report (containing the SGX quote) of a specific node (#907)

Changed

  • Compatibility with msgpack 1.0.0
  • Members now need to provide two public keys, an identity to sign their proposals and votes as before, and public key with which their recovery key share will be encrypted. --member_cert cli argument replaced with --member-info when starting up a network to allow this docs
  • Member status is now a string, eg. "ACTIVE" rather than an integer (#827)
  • User apps have access to standard user-cert lookup (#906)
  • get_rpc_handler() now returns UserRpcFrontend instead of RpcHandler docs (#908)
  • All governance RPC's must now be signed (#911)
  • Test infra stores keys and certificates (e.g. networkcert.pem, user0_privk.pem) in new workspace/<test_label>_common/ folder (#892)

Removed

  • FramedTCP support

0.7.1

Added

  • Installed Python infrastructure can now be used to launch test networks of external builds (#809)
  • Initial threading support, Raft nodes now execute transactions on multiple worker threads (#773, #822)

0.7

This pre-release enables experimental support for Javascript as a CCF runtime, and switches the default transport to HTTP. FramedTCP is still supported in this release (-DFTCP=ON) but is deprecated and will be dropped in the next release.

Changed

  • Fixed node deadlock that could occur under heavy load (#628)
  • Fixed vulnerability to possible replay attack (#419)
  • CCF has an installable bundle (#742)
  • HTTP is the default frame format (#744)

Added

  • Added support for re-keying the ledger (#50)
  • Added QuickJS runtime and sample Javascript app (#668)

Deprecated

  • FramedTCP support. Please use the ccf_FTCP.tar.gz release bundle or build CCF with -DFTCP=ON if you require FTCP support.

0.6

This pre-release enables support for HTTP in CCF

Changed

  • Quote format in getQuotes changed from string to vector of bytes (#566)
  • Improved error reporting and logging (#572, #577, #620)
  • Node certificates endorsed by the network (#581)
  • The keygenerator.sh scripts replaces the keygenerator CLI utility to generate member and user identities.

Added

  • HTTP endpoint support when built with -DHTTP=ON, see https://microsoft.github.io/CCF/users/client.html for details.
  • [Only when building with -DHTTP=ON] The new scurl.sh script can be used to issue signed HTTP requests to CCF (e.g. for member votes). The script takes the same arguments as curl.
  • listMethods RPC for luageneric app (#570)
  • getReceipt/verifyReceipt RPCs (#567)
  • Support for app-defined ACLs (#590)

Binaries for cchost and libluagenericenc.so are attached to this release. Note that libluagenericenc.so should be signed before being deployed by CCF (see https://microsoft.github.io/CCF/developers/build_app.html#standalone-signing).

0.5

This pre-release fixes minor issues and clarifies some of cchost command line options.

Removed

Changed

  • The members and users certificates can now be registered by the consortium using clients that are not the memberclient CLI (e.g. using the tests/infra/jsonrpc.py module) (#550).
  • Fix for Raft consensus to truncate the ledger whenever a rollback occurs and use commit_idx instead of last_idx in many places because of signatures (#503).
  • Join protocol over HTTP fix (#550).
  • Clearer error messages for when untrusted users/members issue transactions to CCF (#530).
  • devcontainer.json now points to right Dockerfile (#543).
  • cchost --raft-election-timeout CLI option default now set to 5000 ms (#559).
  • Better descriptions for cchost command line options (e.g. --raft-election-timeout) (#559).

The cchost, libluagenericenc.so, keygenerator and memberclient are also attached to this release to start a CCF network with lua application. Note that libluagenericenc.so should be signed before being deployed by CCF (see https://microsoft.github.io/CCF/developers/build_app.html#standalone-signing).

0.4

In this preview release, it is possible to run CCF with the PBFT consensus algorithm, albeit with significant limitations.

The evercrypt submodule has been removed, the code is instead imported, to make release tarballs easier to use.

0.3

This pre-release implements the genesis model described in the TR, with a distinct service opening phase. See https://microsoft.github.io/CCF/start_network.html for details.

Some discrepancies with the TR remain, and are being tracked under https://github.com/microsoft/CCF/milestone/2

0.2

Initial pre-release