Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI attempt 2 #156

Merged
merged 18 commits into from
Mar 17, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 52 additions & 11 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,25 +20,23 @@ jobs:
fetch-depth: 0 # the check script below needs the whole history

- name: Setup Nix
uses: cachix/install-nix-action@v17
uses: cachix/install-nix-action@v20
with:
nix_path: nixpkgs=channel:nixos-unstable
extra_nix_config: |
accept-flake-config = true
extra_nix_config: accept-flake-config = true

- name: Run checks
run: nix develop -c ./scripts/check.sh

build:
build-repo:
runs-on: ubuntu-latest

steps:
- name: Setup Nix
uses: cachix/install-nix-action@v17
uses: cachix/install-nix-action@v20
with:
nix_path: nixpkgs=channel:nixos-unstable
extra_nix_config: |
accept-flake-config = true
extra_nix_config: accept-flake-config = true

- name: Checkout main
uses: actions/checkout@v3
Expand All @@ -64,8 +62,10 @@ jobs:
fi

- name: Build repository (main)
# We don't need the metadata here, since we're just
# using this to compare the generated index
run: |
nix develop -c foliage build -j 0
nix develop -c foliage build -j 0
mv _repo _repo-main

- name: Checkout tip commit
Expand All @@ -84,7 +84,7 @@ jobs:

# Do this before the check, useful to have the artifact in case the
# check fails!
- name: Upload artifact
- name: Upload built repository
uses: actions/upload-artifact@v3
with:
name: built-repo
Expand All @@ -100,11 +100,52 @@ jobs:
echo "then you may need to update the timestamps in your new packages to be newer than those in main."
./scripts/check-archive-extension.sh _repo-main/01-index.tar _repo/01-index.tar

build-packages:
runs-on: ubuntu-latest
needs:
- build-repo

steps:
- name: Install Nix
uses: cachix/install-nix-action@v20
with:
nix_path: nixpkgs=channel:nixos-unstable
extra_nix_config: accept-flake-config = true

- name: Setup nixbuild.net
uses: nixbuild/nixbuild-action@v16
with:
nixbuild_token: ${{secrets.NIXBUILD_TOKEN}}

- uses: actions/checkout@v3

- name: Download built repository
uses: actions/download-artifact@v3
with:
name: built-repo
path: _repo

- name: Build checks
# The > is the "YAML folded string" marker, which replaces
andreabedini marked this conversation as resolved.
Show resolved Hide resolved
# newlines with spaces, since the usual bash idiom of \
# doesn't work for some reason
#
# See https://github.com/nixbuild/feedback/issues/14 for
# why some of these options are here
run: >
nix flake check
-L
--override-input CHaP path:_repo
--eval-store auto
--store ssh-ng://eu.nixbuild.net
--max-jobs 2
--builders ""

deploy-check:
runs-on: ubuntu-latest
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
needs:
- build
- build-repo

steps:
- uses: actions/checkout@v3
Expand Down Expand Up @@ -135,7 +176,7 @@ jobs:
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
needs:
- check
- build
- build-repo
- deploy-check

concurrency:
Expand Down
28 changes: 24 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@ Ideally, include the conditions under which we can deprecate it, e.g. "can depre
## How to test changes to CHaP

Sometimes it is useful to test in advance how a new package or a cabal file
revision affects things
revision affects things.

The first steps are always the same, you need a built version of your modified
CHaP locally:
Expand All @@ -230,7 +230,7 @@ CHaP locally:
For the rest of this section we will assume the built repository is in
`/home/user/cardano-haskell-packages/_repo`.

### ... in isolation
### ... by building packages with `cabal`

You can test a locally built CHaP with a small test project consisting of just a
`cabal.project` file:
Expand Down Expand Up @@ -259,9 +259,26 @@ Then you can build whatever package version you want with `cabal`:
$ cabal build cardano-prelude
```

You can troubleshoot a failed build plan using the cabal flags `--constraint`, `--allow-newer- and `--allow-older`. Once you have obtained a working build plan, you should revise you cabal file with appropriate constraints.
You can troubleshoot a failed build plan using the cabal flags `--constraint`, `--allow-newer` and `--allow-older`. Once you have obtained a working build plan, you should revise you cabal file with appropriate constraints.

### ... against haskell.nix projects
### ... by building packages with Nix

You can build packages from CHaP using Nix like this:

```
nix build
--override-input CHaP /home/user/cardano-haskell-packages/_repo
.#haskellPackages.x86_64-linux.ghc8107.plutus-core."1.1.0.0"
```

This will build all the components of that package version that CHaP cares about, namely libraries and executables (test suites and benchmarks are not built).

We need to use `--override-input` because the CHaP flake relies on a built repository.
By default it points to a built repository on the main CHaP `repo` branch.
But if you have just produced your own built repository (see above) then you want to
use that instead, and `--override-input` will let you do that.

### ... by testing against a haskell.nix project

If you want to test a locally built CHaP against a project that uses CHaP
via haskell.nix, you can build the project while overriding CHaP
Expand Down Expand Up @@ -310,6 +327,9 @@ The CI for CHaP does the following things:
Along with requiring linear history, this ensures that package repository that we build is always an extension of the previous one.
- Builds the package repository from the metadata using `foliage`.
- Deploys the package repository to the `repo` branch, along with some static web content.
- Builds a small set of packages using the newly built repository, to flush out any build issues.
- We build with all the major GHC versions we expect to be in use.
- At the moment we don't build all the packages in the repository, only the latest versions of a fixed set.

## Creating a repository like CHaP

Expand Down
4 changes: 4 additions & 0 deletions _sources/cardano-crypto-class/2.0.0.1/meta.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
timestamp = 2022-10-17T00:00:00Z
github = { repo = "input-output-hk/cardano-base", rev = "4ef911eb7a4d628c938312ff4c92e7c0bb8c83a1" }
subdir = 'cardano-crypto-class'

[[revisions]]
number = 1
timestamp = 2023-03-15T16:30:48Z
139 changes: 139 additions & 0 deletions _sources/cardano-crypto-class/2.0.0.1/revisions/1.cabal
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
cabal-version: 2.2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to make it explicit the diff is

❯ diff -u _sources/cardano-crypto-class/2.0.0.1/revisions/1.cabal <(curl https://input-output-hk.github.io/cardano-haskell-packages/index/cardano-crypto-class/2.0.0.1/cardano-crypto-class.cabal)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3650  100  3650    0     0   189k      0 --:--:-- --:--:-- --:--:--  198k
--- _sources/cardano-crypto-class/2.0.0.1/revisions/1.cabal	2023-03-17 08:08:37.621314747 +0800
+++ /dev/fd/63	2023-03-17 08:22:12.796969362 +0800
@@ -39,6 +39,9 @@
     -Wincomplete-uni-patterns -Wpartial-fields -Wredundant-constraints
     -Wunused-packages
 
+  if !flag(development)
+    ghc-options: -Werror
+
 library
   import:            base, project-config
   hs-source-dirs:    src

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is confusing

flag development
  description: Disable `-Werror`
  default:     False
  manual:      True

if !flag(development)
  ghc-options: -Werror

I would expect to enable -Werror in development, and that development is by default false. Maybe it's me :D

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They're going to take it out in future, fortunately.

name: cardano-crypto-class
version: 2.0.0.1
synopsis:
Type classes abstracting over cryptography primitives for Cardano

description:
Type classes abstracting over cryptography primitives for Cardano

license: Apache-2.0
license-files:
LICENSE
NOTICE

author: IOHK
maintainer: [email protected]
copyright: 2019-2021 IOHK
category: Currency
build-type: Simple
extra-source-files: README.md

flag development
description: Disable `-Werror`
default: False
manual: True

flag secp256k1-support
description: Enable support for functions from libsecp256k1. Requires
a recent libsecp256k1 with support for Schnorr signatures.
default: True
manual: True

common base { build-depends: base >= 4.14 && < 4.17 }

common project-config
default-language: Haskell2010
ghc-options:
-Wall -Wcompat -Wincomplete-record-updates
-Wincomplete-uni-patterns -Wpartial-fields -Wredundant-constraints
-Wunused-packages

library
import: base, project-config
hs-source-dirs: src
exposed-modules:
Cardano.Crypto.DSIGN
Cardano.Crypto.DSIGN.Class
Cardano.Crypto.DSIGN.Ed25519
Cardano.Crypto.DSIGN.Ed448
Cardano.Crypto.DSIGN.Mock
Cardano.Crypto.DSIGN.NeverUsed
Cardano.Crypto.Hash
Cardano.Crypto.Hash.Blake2b
Cardano.Crypto.Hash.Class
Cardano.Crypto.Hash.Keccak256
Cardano.Crypto.Hash.NeverUsed
Cardano.Crypto.Hash.SHA256
Cardano.Crypto.Hash.SHA3_256
Cardano.Crypto.Hash.Short
Cardano.Crypto.Init
Cardano.Crypto.KES
Cardano.Crypto.KES.Class
Cardano.Crypto.KES.CompactSingle
Cardano.Crypto.KES.CompactSum
Cardano.Crypto.KES.Mock
Cardano.Crypto.KES.NeverUsed
Cardano.Crypto.KES.Simple
Cardano.Crypto.KES.Single
Cardano.Crypto.KES.Sum
Cardano.Crypto.Libsodium
Cardano.Crypto.Libsodium.C
Cardano.Crypto.Libsodium.Constants
Cardano.Crypto.Libsodium.Hash
Cardano.Crypto.Libsodium.Init
Cardano.Crypto.Libsodium.Memory
Cardano.Crypto.Libsodium.Memory.Internal
Cardano.Crypto.Libsodium.MLockedBytes
Cardano.Crypto.Libsodium.MLockedBytes.Internal
Cardano.Crypto.Libsodium.UnsafeC
Cardano.Crypto.PinnedSizedBytes
Cardano.Crypto.Seed
Cardano.Crypto.Util
Cardano.Crypto.VRF
Cardano.Crypto.VRF.Class
Cardano.Crypto.VRF.Mock
Cardano.Crypto.VRF.NeverUsed
Cardano.Crypto.VRF.Simple
Cardano.Foreign

other-modules:
Cardano.Crypto.PackedBytes

build-depends:
, aeson
, base
, base16-bytestring >=1
, bytestring
, cardano-binary
, cardano-strict-containers
, cryptonite
, deepseq
, heapwords
, memory
, nothunks
, primitive
, serialise
, template-haskell
, th-compat
, text
, transformers
, vector

if impl(ghc < 9.0.0)
build-depends:
integer-gmp

pkgconfig-depends: libsodium -any

if flag(secp256k1-support)
exposed-modules:
Cardano.Crypto.DSIGN.EcdsaSecp256k1
Cardano.Crypto.DSIGN.SchnorrSecp256k1
Cardano.Crypto.SECP256K1.Constants
Cardano.Crypto.SECP256K1.C
pkgconfig-depends: libsecp256k1 -any
cpp-options: -DSECP256K1_ENABLED

test-suite test-memory-example
import: base, project-config
type: exitcode-stdio-1.0
hs-source-dirs: memory-example
main-is: Main.hs
build-depends:
, base
, bytestring
, cardano-crypto-class

if (os(linux) || os(osx))
build-depends: unix
56 changes: 0 additions & 56 deletions builder/default.nix

This file was deleted.

6 changes: 3 additions & 3 deletions flake.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading