Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

s2i build --as-dockerfile not using local .s2i/[assemble, run] when builder image present #1124

Closed
cuppett opened this issue Aug 7, 2023 · 4 comments
Closed

s2i build --as-dockerfile not using local .s2i/[assemble, run] when builder image present #1124

cuppett opened this issue Aug 7, 2023 · 4 comments
Assignees
@coreydaley

Comments

@cuppett
Copy link
Member

cuppett commented Aug 7, 2023

When running s2i build with both local script files and a valid builder image, I'm not seeing the resulting
dockerfile having the correct RUN and CMD statements. However, when pointing to an invalid builder image
the expected set of statements and other elements (in the debug log) appear present in the container build file.

Version: 1.3.2 (Fedora 38) and 1.3.8

Reproduction:

(This repository contains both .s2i/bin/run and .s2i/bin/assemble

$ git clone https://github.com/cuppett/laravel-s2i.git
$ cd laravel-s2i
$ s2i build . registry.access.redhat.com/ubi8/php-80 --as-dockerfile Containerfile.invalid --loglevel=5

I0807 07:26:59.242962  100612 build.go:52] Running S2I version "v1.3.8"
I0807 07:26:59.243069  100612 util.go:70] Getting docker credentials for registry.access.redhat.com/ubi8/php-80
I0807 07:27:00.156576  100612 build.go:170] 
Source:				.
Output Image Tag:		
Environment:			
Labels:				
Incremental Build:		disabled
Remove Old Build:		disabled
Builder Pull Policy:		if-not-present
Previous Image Pull Policy:	if-not-present
Quiet:				disabled
Layered Build:			disabled
Docker Endpoint:		unix:///run/user/1000/podman/podman.sock
Docker Pull Config:		/home/scuppett/.docker/config.json
Docker Pull User:		

I0807 07:27:00.156769  100612 scm.go:20] DownloadForSource .
I0807 07:27:00.157914  100612 clone.go:37] Downloading "." ...
I0807 07:27:00.157922  100612 clone.go:41] Cloning sources into "upload/src"
I0807 07:27:00.198650  100612 clone.go:57] Checked out "HEAD"
I0807 07:27:00.213746  100612 clone.go:63] Updated submodules for "HEAD"
I0807 07:27:00.219627  100612 install.go:269] Using "assemble" installed from "<source-dir>/.s2i/bin/assemble"
I0807 07:27:00.219660  100612 install.go:269] Using "run" installed from "<source-dir>/.s2i/bin/run"
I0807 07:27:00.219680  100612 install.go:269] Using "save-artifacts" installed from "image:///usr/libexec/s2i/save-artifacts"
I0807 07:27:00.219698  100612 ignore.go:84] .s2iignore lists a file spec of .dockerignore 
I0807 07:27:00.219703  100612 ignore.go:116] using globspec upload/src/.dockerignore 
I0807 07:27:00.219711  100612 ignore.go:122] Glob result upload/src/.dockerignore 
I0807 07:27:00.219715  100612 ignore.go:84] .s2iignore lists a file spec of .env* 
I0807 07:27:00.219720  100612 ignore.go:116] using globspec upload/src/.env* 
I0807 07:27:00.219751  100612 ignore.go:122] Glob result upload/src/.env.example 
I0807 07:27:00.219756  100612 ignore.go:84] .s2iignore lists a file spec of .git 
I0807 07:27:00.219761  100612 ignore.go:116] using globspec upload/src/.git 
I0807 07:27:00.219768  100612 ignore.go:122] Glob result upload/src/.git 
I0807 07:27:00.219772  100612 ignore.go:84] .s2iignore lists a file spec of .idea 
I0807 07:27:00.219776  100612 ignore.go:116] using globspec upload/src/.idea 
I0807 07:27:00.219785  100612 ignore.go:84] .s2iignore lists a file spec of .s2iignore 
I0807 07:27:00.219789  100612 ignore.go:116] using globspec upload/src/.s2iignore 
I0807 07:27:00.219796  100612 ignore.go:122] Glob result upload/src/.s2iignore 
I0807 07:27:00.219800  100612 ignore.go:84] .s2iignore lists a file spec of Dockerfile.s2i-fake 
I0807 07:27:00.219804  100612 ignore.go:116] using globspec upload/src/Dockerfile.s2i-fake 
I0807 07:27:00.219811  100612 ignore.go:84] .s2iignore lists a file spec of LICENSE 
I0807 07:27:00.219816  100612 ignore.go:116] using globspec upload/src/LICENSE 
I0807 07:27:00.219823  100612 ignore.go:84] .s2iignore lists a file spec of README.md 
I0807 07:27:00.219827  100612 ignore.go:116] using globspec upload/src/README.md 
I0807 07:27:00.219834  100612 ignore.go:122] Glob result upload/src/README.md 
I0807 07:27:00.219839  100612 ignore.go:84] .s2iignore lists a file spec of vendor 
I0807 07:27:00.219843  100612 ignore.go:116] using globspec upload/src/vendor 
I0807 07:27:00.219853  100612 ignore.go:45] attempting to remove file upload/src/.dockerignore 
I0807 07:27:00.219883  100612 ignore.go:45] attempting to remove file upload/src/.env.example 
I0807 07:27:00.219905  100612 ignore.go:45] attempting to remove file upload/src/.git 
I0807 07:27:00.223812  100612 ignore.go:45] attempting to remove file upload/src/.s2iignore 
I0807 07:27:00.223836  100612 ignore.go:45] attempting to remove file upload/src/README.md 
I0807 07:27:00.223855  100612 dockerfile.go:115] Constructing image build context directory at .
I0807 07:27:00.223893  100612 environment.go:41] Setting 7 environment variables provided by environment file in sources
I0807 07:27:00.223912  100612 dockerfile.go:223] Processing injected inputs: api.VolumeList(nil)
I0807 07:27:00.223918  100612 dockerfile.go:225] Processed injected inputs: api.VolumeList(nil)
I0807 07:27:00.223969  100612 dockerfile.go:296] Wrote custom Dockerfile to Containerfile.invalid
I0807 07:27:00.223977  100612 build.go:180] Application dockerfile generated in Containerfile.invalid

$ cat Containerfile.invalid 

FROM registry.access.redhat.com/ubi8/php-80:latest
LABEL "io.openshift.s2i.build.commit.date"="Sun Mar 20 08:16:49 2022 -0400" \
      "io.openshift.s2i.build.commit.id"="fefad4475746a38a4197f36bff312dd056155f48" \
      "io.openshift.s2i.build.commit.ref"="master" \
      "io.openshift.s2i.build.commit.message"="Upgrade Laravel 9 (Laravel 8.68.1 -> 9.5.1)" \
      "io.openshift.s2i.build.source-location"="/home/scuppett/Development/php/laravel-s2i/." \
      "io.openshift.s2i.build.image"="registry.access.redhat.com/ubi8/php-80:latest" \
      "io.openshift.s2i.build.commit.author"="Stephen Cuppett <[email protected]>"
ENV DOCUMENTROOT="/public" \
    APP_NAME="Laravel" \
    APP_DEBUG="true" \
    APP_URL="http://localhost:8080" \
    LOG_CHANNEL="errorlog" \
    DB_CONNECTION="sqlite" \
    DB_DATABASE=":memory:"
USER root
# Copying in source code
COPY upload/src /tmp/src
# Change file ownership to the assemble user. Builder image must support chown command.
RUN chown -R 1001:0 /tmp/src
USER 1001
# Assemble script sourced from builder image based on user input or image metadata.
# If this file does not exist in the image, the build will fail.
RUN /usr/libexec/s2i/assemble
# Run script sourced from builder image based on user input or image metadata.
# If this file does not exist in the image, the build will fail.
CMD /usr/libexec/s2i/run

I would expect the RUN and CMD to point to the local files, not the originals for the builder image.

Using a nonexistant builder image (with the same repository) yields what I believe is the correct results.

$ s2i build . registry.access.redhat.com/ubi8/php-83 --as-dockerfile Containerfile.valid --loglevel=5

I0807 07:29:37.883152  100867 build.go:52] Running S2I version "v1.3.8"
I0807 07:29:37.883197  100867 util.go:70] Getting docker credentials for registry.access.redhat.com/ubi8/php-83
W0807 07:29:38.222461  100867 build.go:167] could not inspect the builder image for labels: reading manifest latest in registry.access.redhat.com/ubi8/php-83: name unknown: Repo not found
I0807 07:29:38.222513  100867 build.go:170] 
Source:				.
Output Image Tag:		
Environment:			
Labels:				
Incremental Build:		disabled
Remove Old Build:		disabled
Builder Pull Policy:		if-not-present
Previous Image Pull Policy:	if-not-present
Quiet:				disabled
Layered Build:			disabled
Docker Endpoint:		unix:///run/user/1000/podman/podman.sock
Docker Pull Config:		/home/scuppett/.docker/config.json
Docker Pull User:		

I0807 07:29:38.222701  100867 install.go:269] Using "assemble" installed from "image:///usr/libexec/s2i/assemble"
I0807 07:29:38.222715  100867 install.go:269] Using "run" installed from "image:///usr/libexec/s2i/run"
I0807 07:29:38.222724  100867 install.go:269] Using "save-artifacts" installed from "image:///usr/libexec/s2i/save-artifacts"
I0807 07:29:38.222732  100867 scm.go:20] DownloadForSource .
I0807 07:29:38.223926  100867 clone.go:37] Downloading "." ...
I0807 07:29:38.223934  100867 clone.go:41] Cloning sources into "upload/src"
I0807 07:29:38.240659  100867 clone.go:57] Checked out "HEAD"
I0807 07:29:38.255107  100867 clone.go:63] Updated submodules for "HEAD"
I0807 07:29:38.261764  100867 install.go:269] Using "assemble" installed from "<source-dir>/.s2i/bin/assemble"
I0807 07:29:38.261790  100867 install.go:269] Using "run" installed from "<source-dir>/.s2i/bin/run"
I0807 07:29:38.261810  100867 ignore.go:84] .s2iignore lists a file spec of .dockerignore 
I0807 07:29:38.261815  100867 ignore.go:116] using globspec upload/src/.dockerignore 
I0807 07:29:38.261821  100867 ignore.go:122] Glob result upload/src/.dockerignore 
I0807 07:29:38.261825  100867 ignore.go:84] .s2iignore lists a file spec of .env* 
I0807 07:29:38.261829  100867 ignore.go:116] using globspec upload/src/.env* 
I0807 07:29:38.261853  100867 ignore.go:122] Glob result upload/src/.env.example 
I0807 07:29:38.261857  100867 ignore.go:84] .s2iignore lists a file spec of .git 
I0807 07:29:38.261860  100867 ignore.go:116] using globspec upload/src/.git 
I0807 07:29:38.261865  100867 ignore.go:122] Glob result upload/src/.git 
I0807 07:29:38.261868  100867 ignore.go:84] .s2iignore lists a file spec of .idea 
I0807 07:29:38.261871  100867 ignore.go:116] using globspec upload/src/.idea 
I0807 07:29:38.261877  100867 ignore.go:84] .s2iignore lists a file spec of .s2iignore 
I0807 07:29:38.261880  100867 ignore.go:116] using globspec upload/src/.s2iignore 
I0807 07:29:38.261885  100867 ignore.go:122] Glob result upload/src/.s2iignore 
I0807 07:29:38.261888  100867 ignore.go:84] .s2iignore lists a file spec of Dockerfile.s2i-fake 
I0807 07:29:38.261891  100867 ignore.go:116] using globspec upload/src/Dockerfile.s2i-fake 
I0807 07:29:38.261896  100867 ignore.go:84] .s2iignore lists a file spec of LICENSE 
I0807 07:29:38.261899  100867 ignore.go:116] using globspec upload/src/LICENSE 
I0807 07:29:38.261904  100867 ignore.go:84] .s2iignore lists a file spec of README.md 
I0807 07:29:38.261907  100867 ignore.go:116] using globspec upload/src/README.md 
I0807 07:29:38.261911  100867 ignore.go:122] Glob result upload/src/README.md 
I0807 07:29:38.261915  100867 ignore.go:84] .s2iignore lists a file spec of vendor 
I0807 07:29:38.261918  100867 ignore.go:116] using globspec upload/src/vendor 
I0807 07:29:38.261925  100867 ignore.go:45] attempting to remove file upload/src/.env.example 
I0807 07:29:38.261951  100867 ignore.go:45] attempting to remove file upload/src/.git 
I0807 07:29:38.264608  100867 ignore.go:45] attempting to remove file upload/src/.s2iignore 
I0807 07:29:38.264624  100867 ignore.go:45] attempting to remove file upload/src/README.md 
I0807 07:29:38.264636  100867 ignore.go:45] attempting to remove file upload/src/.dockerignore 
I0807 07:29:38.264648  100867 dockerfile.go:115] Constructing image build context directory at .
I0807 07:29:38.264663  100867 dockerfile.go:456] found override script file assemble
I0807 07:29:38.264667  100867 dockerfile.go:456] found override script file run
I0807 07:29:38.264689  100867 environment.go:41] Setting 7 environment variables provided by environment file in sources
I0807 07:29:38.264699  100867 dockerfile.go:209] Override scripts are included in directory "upload/scripts"
I0807 07:29:38.264705  100867 dockerfile.go:223] Processing injected inputs: api.VolumeList(nil)
I0807 07:29:38.264709  100867 dockerfile.go:225] Processed injected inputs: api.VolumeList(nil)
I0807 07:29:38.264845  100867 dockerfile.go:296] Wrote custom Dockerfile to Containerfile.valid
I0807 07:29:38.264850  100867 build.go:180] Application dockerfile generated in Containerfile.valid

$ cat Containerfile.valid 

FROM registry.access.redhat.com/ubi8/php-83:latest
LABEL "io.openshift.s2i.build.image"="registry.access.redhat.com/ubi8/php-83:latest" \
      "io.openshift.s2i.build.commit.author"="Stephen Cuppett <[email protected]>" \
      "io.openshift.s2i.build.commit.date"="Sun Mar 20 08:16:49 2022 -0400" \
      "io.openshift.s2i.build.commit.id"="fefad4475746a38a4197f36bff312dd056155f48" \
      "io.openshift.s2i.build.commit.ref"="master" \
      "io.openshift.s2i.build.commit.message"="Upgrade Laravel 9 (Laravel 8.68.1 -> 9.5.1)" \
      "io.openshift.s2i.build.source-location"="/home/scuppett/Development/php/laravel-s2i/."
ENV DOCUMENTROOT="/public" \
    APP_NAME="Laravel" \
    APP_DEBUG="true" \
    APP_URL="http://localhost:8080" \
    LOG_CHANNEL="errorlog" \
    DB_CONNECTION="sqlite" \
    DB_DATABASE=":memory:"
USER root
# Copying in override assemble/run scripts
COPY upload/scripts /tmp/scripts
# Copying in source code
COPY upload/src /tmp/src
# Change file ownership to the assemble user. Builder image must support chown command.
RUN chown -R 1001:0 /tmp/scripts /tmp/src
USER 1001
RUN /tmp/scripts/assemble
CMD /tmp/scripts/run

I'm not sure if I'm invoking this right or if I'm missing some other key attribute or behavior here.
@bparees
Copy link
Contributor

bparees commented Aug 7, 2023

looks like a long standing bug in

source-to-image/pkg/build/strategies/dockerfile/dockerfile.go

Lines 130 to 134 in 980ca19

imageScriptsDir, hasAllScripts := getImageScriptsDir(config)
var providedScripts map[string]bool
if !hasAllScripts {
providedScripts = scanScripts(filepath.Join(config.WorkingDir, builder.uploadScriptsDir))
}

since the image itself as a label that indicates it provides scripts, "hasAllScripts" ends up true and so we never scan the local repository for override scripts.

the logic looks almost deliberate, which is interesting but i'm pretty sure the behavior is incorrect. Looks like it went in here: 6815fa6

I reviewed/approved it but i think we missed this scenario when thinking about it.

@tizard34
Copy link

I have the same problem. As a workaround, I perform a "replacement" in the generated dockerfile.
I hope the fix will be included in the next release.

@openshift-ci openshift-ci bot closed this as completed in 0397929 Nov 3, 2023
openshift-ci bot added a commit that referenced this issue Nov 3, 2023
@openshift-ci
Merge pull request #1125 from cuppett/cuppett/fix-1124
13eb773 
Resolves #1124: Always scan the repository for provided scripts
@faust64
Copy link

faust64 commented Feb 29, 2024

I see that the last release does not include those fixes -- v1;3.9 from Oct 19th, while PR got merged after
Last images we can get from RH registry would still suffer from that bug:
https://catalog.redhat.com/software/containers/source-to-image/source-to-image-rhel8/64ee172747850a7cc4b335b6?architecture=amd64&image=6543cd204a64b4ac1990e32e

Any ETA for next release?

@faust64
Copy link

faust64 commented Feb 29, 2024

... oh... nevermind ...
I guess it's not yet full working, ... #1141 ... assuming that 1;3.9-dirty was built including fix mentioned here ... case filed 3 days ago ... odds are: it was ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@coreydaley coreydaley

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@faust64 @cuppett @bparees @coreydaley @tizard34