Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CommandError / permission denied error when running deb-bootstrap-builder (on Singularity 2) #67

Closed
mslw opened this issue Jul 11, 2022 · 3 comments
Closed

CommandError / permission denied error when running deb-bootstrap-builder (on Singularity 2) #67

mslw opened this issue Jul 11, 2022 · 3 comments
Labels
bug Something isn't working

Comments

@mslw
Copy link
Contributor

mslw commented Jul 11, 2022 •
edited
Loading

I was running

datalad deb-configure-builder -d builder dockerbase=debian:bullseye  # all ok
datalad deb-bootstrap-builder -d builder

The second command prompted for sudo password (*within run command and printed all the regular datalad run & singularity outputs, ending with:

Singularity container built: envs/singularity-amd64.sif
Cleaning up...
[INFO   ] == Command exit (modification check follows) ===== 
run(ok): /home/mszczepanik/Documents/hackathon-datalad-debian/mydistrib/builder (dataset) [sudo singularity build --force envs/sing...]
[WARNING] Received an exception CommandError(CommandError: 'git -c diff.ignoreSubmodules=none annex add --json --json-error-messages -c annex.dotfiles=true -- envs/singularity-amd64.sif' failed with exitcode 1 under /home/mszczepanik/Documents/hackathon-datalad-debian/mydistrib/builder [info keys: stdout_json]
>  envs/singularity-amd64.sif: setFileMode: permission denied (Operation not permitted) [err: 'add: 1 failed']). Canceling not-yet running jobs and waiting for completion of running. You can force earlier forceful exit by Ctrl-C. 
[INFO   ] Canceled 0 out of 0 jobs. 0 left running. 
Total: 0.00 datasets [00:00, ? datasets/s]CommandError: 'git -c diff.ignoreSubmodules=none annex add --json --json-error-messages -c annex.dotfiles=true -- envs/singularity-amd64.sif' failed with exitcode 1 under /home/mszczepanik/Documents/hackathon-datalad-debian/mydistrib/builder [info keys: stdout_json]
>  envs/singularity-amd64.sif: setFileMode: permission denied (Operation not permitted)
add: 1 failed

The singularity-amd64.sif file ends up created, but untracked by DataLad. It's owned by root:

ls -lh builder/envs/        
-rw-r--r-- 1 mszczepanik mszczepanik   46 Jul 11 12:35 README.md
-rwxr-xr-x 1 root        root        201M Jul 11 12:38 singularity-amd64.sif

I'm on Singularity 2.6.1-dist, DataLad 0.17 and git-annex 10.20220322-1

Full WTF output ```

WTF

configuration <SENSITIVE, report disabled by configuration>

credentials

  • keyring:
    • active_backends:
      • SecretService Keyring
      • PlaintextKeyring with no encyption v.1.0 at /home/mszczepanik/.local/share/python_keyring/keyring_pass.cfg
    • config_file: /home/mszczepanik/.config/python_keyring/keyringrc.cfg
    • data_root: /home/mszczepanik/.local/share/python_keyring

datalad

  • version: 0.17.0

dataset

  • branches:
    • git-annex@7247bd6
    • main@035d21a
  • id: 863397f4-3252-4592-aa75-c22c1c4b29fc
  • metadata: <SENSITIVE, report disabled by configuration>
  • path: /home/mszczepanik/Documents/hackathon-datalad-debian/mydistrib
  • repo: AnnexRepo

dependencies

  • annexremote: 1.6.0
  • boto: 2.49.0
  • cmd:7z: 16.02
  • cmd:annex: 10.20220322-1~ndall+1
  • cmd:bundled-git: 2.30.2
  • cmd:git: 2.30.2
  • cmd:ssh: 8.4p1
  • cmd:system-git: 2.30.2
  • cmd:system-ssh: 8.4p1
  • humanize: 4.2.3
  • iso8601: 1.0.2
  • keyring: 23.6.0
  • keyrings.alt: 4.1.0
  • msgpack: 1.0.4
  • platformdirs: 2.5.2
  • requests: 2.28.1

environment

  • LANG: en_US.UTF-8
  • LANGUAGE: en_US.UTF-8
  • LC_ALL: en_US.UTF-8
  • LC_CTYPE: en_US.UTF-8
  • PATH: /home/mszczepanik/.virtualenvs/datalad-debian-dev/bin:/home/mszczepanik/.dotfiles/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin:/usr/local/games:/usr/games:/home/mszczepanik/.local/bin:/home/mszczepanik/.local/share/gem/ruby/2.7.0/bin

extensions

  • container:
    • description: Containerized environments
    • entrypoints:
      • datalad_container.containers_add.ContainersAdd:
        • class: ContainersAdd
        • load_error: None
        • module: datalad_container.containers_add
        • names:
          • containers-add
          • containers_add
      • datalad_container.containers_list.ContainersList:
        • class: ContainersList
        • load_error: None
        • module: datalad_container.containers_list
        • names:
          • containers-list
          • containers_list
      • datalad_container.containers_remove.ContainersRemove:
        • class: ContainersRemove
        • load_error: None
        • module: datalad_container.containers_remove
        • names:
          • containers-remove
          • containers_remove
      • datalad_container.containers_run.ContainersRun:
        • class: ContainersRun
        • load_error: None
        • module: datalad_container.containers_run
        • names:
          • containers-run
          • containers_run
    • load_error: None
    • module: datalad_container
    • version: 1.1.6
  • datalad_debian:
    • description: DataLad for working with Debian packages
    • entrypoints:
      • datalad_debian.bootstrap_builder.BootstrapBuilder:
        • class: BootstrapBuilder
        • load_error: None
        • module: datalad_debian.bootstrap_builder
        • names:
          • deb-bootstrap-builder
          • deb_bootstrap_builder
      • datalad_debian.build_package.BuildPackage:
        • class: BuildPackage
        • load_error: None
        • module: datalad_debian.build_package
        • names:
          • deb-build-package
          • deb_build_package
      • datalad_debian.configure_builder.ConfigureBuilder:
        • class: ConfigureBuilder
        • load_error: None
        • module: datalad_debian.configure_builder
        • names:
          • deb-configure-builder
          • deb_configure_builder
      • datalad_debian.new_distribution.NewDistribution:
        • class: NewDistribution
        • load_error: None
        • module: datalad_debian.new_distribution
        • names:
          • deb-new-distribution
          • deb_new_distribution
      • datalad_debian.new_package.NewPackage:
        • class: NewPackage
        • load_error: None
        • module: datalad_debian.new_package
        • names:
          • deb-new-package
          • deb_new_package
    • load_error: None
    • module: datalad_debian
    • version: 0+untagged.56.g73e9d09

git-annex

  • build flags:
    • Assistant
    • Webapp
    • Pairing
    • Inotify
    • DBus
    • DesktopNotify
    • TorrentParser
    • MagicMime
    • Feeds
    • Testsuite
    • S3
    • WebDAV
  • dependency versions:
    • aws-0.22
    • bloomfilter-2.0.1.0
    • cryptonite-0.26
    • DAV-1.3.4
    • feed-1.3.0.1
    • ghc-8.8.4
    • http-client-0.6.4.1
    • persistent-sqlite-2.10.6.2
    • torrent-10000.1.1
    • uuid-1.3.13
    • yesod-1.6.1.0
  • key/value backends:
    • SHA256E
    • SHA256
    • SHA512E
    • SHA512
    • SHA224E
    • SHA224
    • SHA384E
    • SHA384
    • SHA3_256E
    • SHA3_256
    • SHA3_512E
    • SHA3_512
    • SHA3_224E
    • SHA3_224
    • SHA3_384E
    • SHA3_384
    • SKEIN256E
    • SKEIN256
    • SKEIN512E
    • SKEIN512
    • BLAKE2B256E
    • BLAKE2B256
    • BLAKE2B512E
    • BLAKE2B512
    • BLAKE2B160E
    • BLAKE2B160
    • BLAKE2B224E
    • BLAKE2B224
    • BLAKE2B384E
    • BLAKE2B384
    • BLAKE2BP512E
    • BLAKE2BP512
    • BLAKE2S256E
    • BLAKE2S256
    • BLAKE2S160E
    • BLAKE2S160
    • BLAKE2S224E
    • BLAKE2S224
    • BLAKE2SP256E
    • BLAKE2SP256
    • BLAKE2SP224E
    • BLAKE2SP224
    • SHA1E
    • SHA1
    • MD5E
    • MD5
    • WORM
    • URL
    • X*
  • local repository version: 8
  • operating system: linux x86_64
  • remote types:
    • git
    • gcrypt
    • p2p
    • S3
    • bup
    • directory
    • rsync
    • web
    • bittorrent
    • webdav
    • adb
    • tahoe
    • glacier
    • ddar
    • git-lfs
    • httpalso
    • borg
    • hook
    • external
  • supported repository versions:
    • 8
    • 9
    • 10
  • upgrade supported from repository versions:
    • 0
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
  • version: 10.20220322-1~ndall+1

location

  • path: /home/mszczepanik/Documents/hackathon-datalad-debian/mydistrib
  • type: dataset

metadata_extractors

  • annex:
    • distribution: datalad 0.17.0
    • load_error: None
    • module: datalad.metadata.extractors.annex
  • audio:
    • distribution: datalad 0.17.0
    • load_error: ModuleNotFoundError(No module named 'mutagen')
    • module: datalad.metadata.extractors.audio
  • datacite:
    • distribution: datalad 0.17.0
    • load_error: None
    • module: datalad.metadata.extractors.datacite
  • datalad_core:
    • distribution: datalad 0.17.0
    • load_error: None
    • module: datalad.metadata.extractors.datalad_core
  • datalad_rfc822:
    • distribution: datalad 0.17.0
    • load_error: None
    • module: datalad.metadata.extractors.datalad_rfc822
  • exif:
    • distribution: datalad 0.17.0
    • load_error: ModuleNotFoundError(No module named 'exifread')
    • module: datalad.metadata.extractors.exif
  • frictionless_datapackage:
    • distribution: datalad 0.17.0
    • load_error: None
    • module: datalad.metadata.extractors.frictionless_datapackage
  • image:
    • distribution: datalad 0.17.0
    • load_error: ModuleNotFoundError(No module named 'PIL')
    • module: datalad.metadata.extractors.image
  • xmp:
    • distribution: datalad 0.17.0
    • load_error: ModuleNotFoundError(No module named 'libxmp')
    • module: datalad.metadata.extractors.xmp

metadata_indexers

python

  • implementation: CPython
  • version: 3.9.2

system

  • distribution: debian/11/bullseye
  • encoding:
    • default: utf-8
    • filesystem: utf-8
    • locale.prefered: UTF-8
  • filesystem:
    • CWD:
      • path: /home/mszczepanik/Documents/hackathon-datalad-debian/mydistrib
    • HOME:
      • path: /home/mszczepanik
    • TMP:
      • path: /tmp
  • max_path_length: 318
  • name: Linux
  • release: 5.10.0-15-amd64
  • type: posix
  • version: Sketch for building Debian source packages in datalad-tracked containers #1 SMP Debian 5.10.120-1 (2022-06-09)
    Hint: install psutil to get filesystem information
</details>
@mslw mslw added the bug Something isn't working label Jul 11, 2022
@mih
Copy link
Contributor

mih commented Jul 11, 2022

Possibly an indication that we need singularity v3. If so, this issues should be closed with some kind of documentation on that, somewhere.

From my POV there is little point in supporting singularity v2.

@mslw
Copy link
Contributor Author

mslw commented Jul 11, 2022

I can confirm that updating to Singularity 3 (specifically 3.10.0) solved this issue. All commands return with ok result.

ls -lh builder/envs 
total 8.0K
-rw-r--r-- 1 mszczepanik mszczepanik  46 Jul 11 13:19 README.md
lrwxrwxrwx 1 mszczepanik mszczepanik 135 Jul 11 13:21 singularity-amd64.sif -> ../.git/annex/objects/V9/Kp/MD5E-s210337792--ba90081997dbd7244e08401e150a438f.sif/MD5E-s210337792--ba90081997dbd7244e08401e150a438f.sif

FTR I'm on Debian 11.4 (bullseye) and I installed Singularity from their GitHub release page (`singularity-ce_3.10.0-focal_amd64.deb)

Do you think that a short section "requirements" with a comment (saying that Singularity 3 is required) in ReST docs and / or the README file would be enough?

@mslw mslw changed the title CommandError / permission denied error when running deb-bootstrap-builder CommandError / permission denied error when running deb-bootstrap-builder (on Singularity 2) Jul 11, 2022
@mslw mslw mentioned this issue Jul 11, 2022
Open
@mslw
Copy link
Contributor Author

mslw commented Jul 11, 2022

Now superseded by an issue related to documentation.

@mslw mslw closed this as completed Jul 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mih @mslw