Cardano DB Sync 13.1.0.2

Cardano DB Sync 13.1.0.2 is a maintenance release addressing a fix in ledger internal data structures.

There is also a new postgres requirement for users that don't use docker-compose and have to manually config postgres:

It is advised to increase the maintenance_work_mem from Postgres config to 0.5GB - 1GB (default is 64MB).
Also use the default (2) or higher max_parallel_maintenance_workers (this parameter was introduced in postgres 11)

This release is compatible with Cardano Node 1.35.5.

Upgrade instructions

13.1.0.2 is fully compatible with 13.1.0.0, so resyncing or migrating is not necessary.
In order to upgrade from 13.0.x to 13.1.0.2 resyncing is not necessary and requires no special flags from the user. DBSync will automatically spawn a fixing procedure, which fixes old values related to plutus data. After that schema migrations will run on top of the existing db.
Full upgrade details can be found here.

The state snapshot files for this release can be found here:

Network	Link to snapshot
Mainnet	https://update-cardano-mainnet.iohk.io/cardano-db-sync/index.html#13.1/

Documentation for restoring from these files available at state-snapshot.md.

Requires ghc-8.10.x to build.

Static binaries download

Known issues

• When --disable-ledger flag is enabled, the --state-dir option still needs to be provided, even though db-sync will completely ignore it.
• Unhandled DCertGenesis certificate #290

Documentation

Cardano DB Sync Overview
Building and Running the Cardano DB Sync Node
Schema Management
Validation
State snapshot
Smash

Platforms

• Linux 64-bit
• Docker

Sign off

Role	Approval
Technical Lead	✔️
QA Engineer	✔️
Ops	✔️
Release Engineer	✔️

Cardano DB Sync 13.1.0.0

This release delivers a number of enhancements and fixes, see below for details.

It also introduces a new way to enumerate releases, based on how it affects the db, details can be found here.

There is also a new postgres requirement for users that don't use docker-compose and have to manually config postgres:

It is advised to increase the maintenance_work_mem from Postgres config to 0.5GB - 1GB (default is 64MB).
Also use the default (2) or higher max_parallel_maintenance_workers (this parameter was introduced in postgres 11)

This release is compatible with Cardano Node 1.35.4.

Upgrade instructions

In order to upgrade from 13.0.x to 13.1.0.0 resyncing is not necessary and requires no special flags from the user. DBSync will automatically spawn a fixing procedure, which fixes old values related to plutus data. After that schema migrations will run on top of the existing db.
Full upgrade details can be found here.

The state snapshot files for this release can be found here:

Network	Link to snapshot
Mainnet	https://update-cardano-mainnet.iohk.io/cardano-db-sync/index.html#13.1/

Documentation for restoring from these files available at state-snapshot.md.

Requires ghc-8.10.x to build.

Static binaries download

Added

• Added a new reverse_index table to speed up rollbacks
• Added experimental flags skip-plutus-data-fix, only-plutus-data-fix, force-indexes to cardano-db-sync
• Added experimental flags force-indexes, mock-fix for cardano-db-tool run-migrations command
• Added new flag POSTGRES_ARGS (wih reccomended default values) to docker-compose.yml for customization of database settings

Improved

• Avoids rollbacks on restarts, making them way faster [#1190]
• Syncing speed is increased
• Allows to migrate from previous 13.x releases without resync [#1172]
• Delays the creation of most indexes while syncing. They are created when syncing is almost complete [#1293]
• Reworked the way that rollback works, using reverse indexes.
• Snapshot creation no longer rollbacks to the ledger snapshot file, so it's faster

Removed

• Removed many unique keys that were never used [#1087]
• Removed all foreign keys [#1082]
• Columns stake_address.tx_id, cost_model.block_id are removed

Resolved

• Creates a procedure that fixes old values related to plutus data [#1214] as the first step of migration [#1278]
• Fixed Stake Pool metadata fetch error exceeded 512 bytes [#1270]
• Bump iohk-nix to fetch the correct config for preview and preprod respin

Known issues

• When --disable-ledger flag is enabled, the --state-dir option still needs to be provided, even though db-sync will completely ignore it.
• Unhandled DCertGenesis certificate #290

Documentation

Cardano DB Sync Overview
Building and Running the Cardano DB Sync Node
Schema Management
Validation
State snapshot
Smash

Platforms

• Linux 64-bit
• Docker

Sign off

Role	Approval
Technical Lead	✔️
QA Engineer	✔️
Ops	✔️
Release Engineer	✔️

Cardano DB Sync 13.0.5

This release delivers a number of enhancements and fixes, as well as support for the new Preview and Pre-production networks.

This release is compatible with Cardano Node 1.35.3.

⚠️ Note that this release requires the database to be dropped and recreated if upgrading from a release prior to 13.0.0.

The state snapshot files for this release can be found here:

Network	Link to snapshot
Mainnet	https://update-cardano-mainnet.iohk.io/cardano-db-sync/index.html#13/

Documentation for restoring from these files available at state-snapshot.md.

Requires ghc-8.10.x to build.

Hydra binaries download

Extra Arguments image

For users wishing to use the new flags to --disable-ledger or --disable-cache use the 13.0.5-extra-args docker image, which is otherwise identical to the 13.0.5 image.

Added

• Support new networks - preview, preprod

Improved

• Docker documentation
• Made disable options like disable-ledger easily available through docker

Resolved

• Fixed an issue where StakeAddress cache was not cleaned up properly and could cause crashes [#1222]
• Fixed an issue where fees for txs with phase 2 failure which din't include the total_collateral fields appeared to be 0 [#1242]
• Fixed an issue where Datum were reserialised and could be inserted in a different CBOR format [#1214]

Known issues

• When --disable-ledger flag is enabled, the --state-dir option still needs to be provided, even though db-sync will completely ignore it.
• Unhandled DCertGenesis certificate #290

Documentation

Cardano DB Sync Overview
Building and Running the Cardano DB Sync Node
Schema Management
Validation
State snapshot
Smash

Platforms

• Linux 64-bit
• Docker

Sign off

Role	Approval
Technical Lead	✔️
QA Engineer	✔️
Ops	⏳
Release Manager	✔️

Cardano DB Sync 13.0.4

This release includes the following:

• A fix for a bug that manifests itself if db-sync is running in the Babbage era and shuts down, it has to re-sync from an Alonzo snapshot, or from Genesis if it doesn't exist.
• Integrates the fix of the obsolete check in the new 'Praos' protocol. (#3891)
• Integrates the fix for the misused minfee function in ledger. (#2938)

This release is compatible with Cardano Node 1.35.3.

⚠️ Note that this release requires the database to be dropped and recreated if upgrading from a release prior to 13.0.0.

The state snapshot files for this release can be found here:

Network	Link to snapshot
Mainnet	https://update-cardano-mainnet.iohk.io/cardano-db-sync/index.html#13/

Documentation for restoring from these files available at state-snapshot.md.

Requires ghc-8.10.x to build.

Hydra binaries download

Resolved

• Ensure Babbage TxOut decoder can't fail due to malformed Ptr. This bug manifests itself if db-sync is running in the Babbage era and shuts down, it has to re-sync from an Alonzo snapshot, or from Genesis if it doesn't exist. (#1181)

Known issues

• When --disable-ledger flag is enabled, the --state-dir option still needs to be provided, even though db-sync will completely ignore it.
• Datum round-trip corrupts datum bytes #1214
• Unhandled DCertGenesis certificate #290

Documentation

Cardano DB Sync Overview
Building and Running the Cardano DB Sync Node
Schema Management
Validation
The Plug-in System
State snapshot
Smash

Platforms

• Linux 64-bit
• Docker

Sign off

Role	Approval
Technical Lead	✔️
QA Engineer	✔️
Ops	⏳
Release Manager	✔️

Cardano DB Sync 13.0.0

This release introduces support for Vasil.

This release is compatible with Cardano Node 1.35.0.

⚠️ Note that this release requires the database to be dropped and recreated.

The state snapshot files for this release can be found here:

Network	Link to snapshot
Testnet	https://updates-cardano-testnet.s3.amazonaws.com/cardano-db-sync/index.html#13/
Mainnet	https://update-cardano-mainnet.iohk.io/cardano-db-sync/index.html#13/

Documentation for restoring from these files available at state-snapshot.md.

Requires ghc-8.10.x to build.

Hydra binaries download

Added

• Added --disable-ledger flag, which significantly reduces memory usage. Read more at doc/configuration.md.
• Added --disable-cache flag, which slightly reduces memory usage. Read more at doc/configuration.md.
• Handle PlutusV1 and PlutusV2 scripts as separate script types, removing old Plutus type.
• Store CBOR serialized Datum and Redeemer via bytes field.
• HF integration. Schema changes for new Babbage fields. (#1081).
• Used pulsing rewards coming from new ledger events. Also incremental stake slices.
• Added connection pools for smash, instead of opening a connection for each request. Configurable with --pool for admins.
• Added docker healthchecks for postgres and cardano-node.
• Allow to add additional migrations (indexes and views) (#1044).
• Added multiple unit tests.
• Store requiredSigners (transaction extra key witnesses).
• Outputs, inputs and multi asset outputs are stored in batches grouped by blocks, instead of one by one.
• Log cabal version, git hash and command line options on startup (#1166).

Improved

• Reduction in memory usage.
• Reduction in sync-time.
• Renamed --no-epoch-table flag to disable-epoch. Read more at doc/configuration.md.
• Improved logging. It includes some performance statistics.
• Extended the caches.
• Reduced the blocks it takes for prometheus block height to be updated.
• pool_owner now references pool_update instead of pool_hash and tx (#986).

Removed

• BulkOperations which affected memory usage and rollbacks are removed.
• Removed "no-store" from metadata caching for smash (#1075)
• Removed plugin System, merges 3 packages cardano-db-sync-extended, cardano-sync and cardano-db-sync into the last.

Resolved

• Fix unique keys of reserves, treasury, pool_update, pool_retire, stake_registration, stake_deregistration, delegation (#1051).
• Fix smash server error "Pool is retired" for pools that don't exist. (#997)
• Fix duplicate instant rewards (#981).
• Fix handling of StakeRefPtr (#1024).
• Fix parameter_proposal.max_block_size (#965).
• Fixes on testnets that fork directly to Shelley (#953).

Known issues

• When --disable-ledger flag is enabled, the --state-dir option still needs to be provided, even though db-sync will completely ignore it.
• Unhandled DCertGenesis certificate #290

Documentation

Cardano DB Sync Overview
Building and Running the Cardano DB Sync Node
Schema Management
Validation
The Plug-in System
State snapshot
Smash

Platforms

• Linux 64-bit
• Docker

Sign off

Role	Approval
Technical Lead	✔️
QA Engineer	✔️
Ops	✔️
Release Manager	✔️

Cardano DB Sync 12.0.2

This is a minor release to fix the issues detailed below.

This release is compatible with Cardano Node 1.31.0 and Cardano Node 1.33.0.

The state snapshot files for this release can be found here:

Network	Link to snapshot
Testnet	https://updates-cardano-testnet.s3.amazonaws.com/cardano-db-sync/index.html#12/
Mainnet	https://update-cardano-mainnet.iohk.io/cardano-db-sync/index.html#12/

Documentation for restoring from these files available at state-snapshot.md.

Requires ghc-8.10.x to build.

Hydra binaries download

Resolved

• Fix metadata hash in served JSON. (2c3396c)
• Fix PoolOfflineFetchError URL entry (#697)

Documentation

Cardano DB Sync Overview
Building and Running the Cardano DB Sync Node
Schema Management
Validation
The Plug-in System
State snapshot
Smash

Platforms

• Linux 64-bit
• Docker

Sign off

Role	Approval
Technical Lead	✔️
QA Engineer	✔️
Ops	⏳
Release Manager	✔️

Cardano DB Sync 12.0.0

This release is compatible with Cardano Node 1.31.0.

⚠️ Note that this release requires the database to be dropped and recreated.

The state snapshot files for this release can be found here:

Network	Link to snapshot
Testnet	https://updates-cardano-testnet.s3.amazonaws.com/cardano-db-sync/index.html#12/
Mainnet	https://update-cardano-mainnet.iohk.io/cardano-db-sync/index.html#12/

Documentation for restoring from these files available at state-snapshot.md.

Requires ghc-8.10.x to build.

Added

• Include json and (raw) bytes fields to script table.
• Add cost_model, datum and redeemer tables.
• Add a multi_asset table with policy, name and asset fingerprint columns (#868).
• Ported SMASH version (its documentation), which is a standalone SMASH ported to DB sync 12, under trial-run and production monitoring to verify performance. Therefore, SMASH users are advised to stay on their current standalone version and NOT update to this ported SMASH version. Updates on the ported version will follow in the following DB sync releases.

Improved

• Update rewardtype enum (used in reward table) to include a pool deposit refund type.
• Update cost_model* fields of param_proposal and epoch_param tables to reference cost_model
  table.
• Unify SQL types of epoch_stake.epoch_no and epoch.no (#811).
• Handle the empty list case in insertManyUncheckedUnique (#869).
• Update system requirements (#951).

Removed

• Drop the policy and name columns of ma_tx_mint and ma_tx_out tables, replacing those
  columns with a reference to the multi_asset table.
• Stop supporting test configs which don't initiate Shelley properly.

Resolved

• Fix handling of transactions which have contracts which fail second stage validation (#883).
• Fix missing and inconsistent rewards issues (#791, #796, #805. #882, #826, #918, #921, #923, #939,
  #947).
• Fix the git SHA hash in the version subcommand (#844).

Documentation

Cardano DB Sync Overview
Building and Running the Cardano DB Sync Node
Schema Management
Validation
The Plug-in System
State snapshot
Smash

Platforms

• Linux 64-bit
• Docker

Sign off

Role	Approval
Technical Lead	✔️
QA Engineer	✔️
Ops	✔️
Release Manager	✔️

Cardano DB Sync 11.0.4

This release is compatible with Cardano Node 1.30.1.

⚠️ This release has known problems with rewards (see the known issues section). Work is continuing to deliver a fix for this issue as soon as possible.

The state snapshot files for this release can be found here:

Network	Link to snapshot
Testnet	https://updates-cardano-testnet.s3.amazonaws.com/cardano-db-sync/index.html#11/
Mainnet	https://update-cardano-mainnet.iohk.io/cardano-db-sync/index.html#11/

Documentation for restoring from these files available at state-snapshot.md.

Requires ghc-8.10.x to build.

Resolved

• Fix race condition on insertion of pool offline data or error response (#806, #823, #858).

Known issues

• Unhandled DCertGenesis certificate #290
• Missing staking rewards #683

Documentation

Cardano DB Sync Overview
Building and Running the Cardano DB Sync Node
Schema Management
Validation
The Plug-in System
State snapshot

Platforms

• Linux 64-bit
• Docker

Sign off

Role	Approval
Technical Lead	✔️
QA Engineer	✔️
Ops	⏳
Release Manager	✔️

Cardano DB Sync 11.0.3

This release is compatible with Cardano Node 1.30.1.

⚠️ This release has known problems with rewards (see the known issues section). Work is continuing to deliver a fix for this issue as soon as possible.

The state snapshot files for this release can be found here:

Network	Link to snapshot
Testnet	https://updates-cardano-testnet.s3.amazonaws.com/cardano-db-sync/index.html#11/
Mainnet	https://update-cardano-mainnet.iohk.io/cardano-db-sync/index.html#11/

Documentation for restoring from these files available at state-snapshot.md.

Requires ghc-8.10.x to build.

Updated

• Use same dependencies as 1.30.1 of cardano-node.

Resolved

• Fix docker issue (#810).
• Fix schema documentation typo (#799).
• Fix db-sync makes a rollback after using RESTORE_SNAPSHOT (#784)

Known issues

• Race condition on insertion of pool offline data or error response #806, #823, #831.
• Unhandled DCertGenesis certificate #290
• Missing staking rewards #683

Documentation

Cardano DB Sync Overview
Building and Running the Cardano DB Sync Node
Schema Management
Validation
The Plug-in System
State snapshot

Platforms

• Linux 64-bit
• Docker

Sign off

Role	Approval
Technical Lead	✔️
QA Engineer	✔️
Ops	⏳
Release Manager	✔️

Cardano DB Sync 11.0.0

This release introduces support for Alonzo.
It is compatible with Cardano Node 1.29.0.
Note that this release requires the database to be dropped and recreated.

⚠️ This release has known problems with rewards (see the known issues section). Work is continuing to deliver a fix for this issue as soon as possible.

The state snapshot files for this release can be found here:

Network	Link to snapshot
Testnet	https://updates-cardano-testnet.s3.amazonaws.com/cardano-db-sync/index.html#11/
Mainnet	https://update-cardano-mainnet.iohk.io/cardano-db-sync/index.html#11/

Documentation for restoring from these files available at state-snapshot.md.

Requires ghc-8.10.x to build.

Added

• Introduces Alonzo support.
• Database changes as per cardano-db changelog.
• Add run time validation that the on-disk schema matches what the app expects (#472).
• Add partial validation of Reward table.
• Add Alonzo (smart contracts) related Redeemer and Script tables.
• Add an epochNo columm to StakeRegistration/Deregistration tables.
• Add instantaneous rewards to the Reward table. This adds two new reward sources (treasury and reserves to the existing leader and member types) and makes the poolId NULLable (treasury and reserves rewards are not associated with a pool).
• For the EpochSyncTime table change the seconds column from double to Word64 and make it non-NULLable.
• For the Reward table, drop the epochNo column and relace it with earnedEpoch and spendableEpoch
• Add opCertCounter column to Block table.
• Add dataHash field to the TxOut table.
• Add a validation to ensure the sum of the AdaPots values per epoch is a constant (#718). - cardano-db-tool
• Add reporting functionalities (balance, teansactions, reward history, latest rewards). - cardano-db-tool

Improved

• Extend ProtocolParams and EpochParams tables for Alonzo.

Removed

• Drop OrphanedRewrd table (because it was incomplete can cannot be fixed).

Resolved

• Fix utxo_view defined in schema (#744).

Known issues

• Unhandled DCertGenesis certificate #290
• Missing staking rewards #683
• db-sync makes a rollback after using RESTORE_SNAPSHOT #784
  • Please use the below workaround for this issue:
    Modify docker-compose‧yml to add the following elements:

services:
  cardano-db-sync:
    working_dir: /var/lib/cdbsync
    volumes:
      - db-sync-tmp:/tmp

volumes:
  db-sync-tmp:

Documentation

Cardano DB Sync Overview
Building and Running the Cardano DB Sync Node
Schema Management
Validation
The Plug-in System
State snapshot

Platforms

• Linux 64-bit
• Docker

Sign off

Role	Approval
Technical Lead	✔️
QA Engineer	✔️
Ops	⏳
Release Manager	✔️