Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

When Tern executes fs_hash.sh, it cannot cd to the contents directory. #430

Closed
Katsunori04 opened this issue Aug 30, 2019 · 9 comments
Closed

When Tern executes fs_hash.sh, it cannot cd to the contents directory. #430

Katsunori04 opened this issue Aug 30, 2019 · 9 comments
Assignees
@nishakm
Labels
bug Something went wrong

Comments

@Katsunori04
Copy link

Katsunori04 commented Aug 30, 2019 •
edited
Loading

Hello, Tern Team.
When I tried to analyze the "yarara / base: v1" image, Tern output the following error.
Tern can't use command's "cd".
Is it my environmental problem or the Tern specification problem?

Problem Statement
When Tern executes fs_hash.sh, it cannot cd to the contens directory.

To Reproduce

vagrant@ubuntu-bionic:~$ tern -l  -k report -i yarara/base:v1
fatal: not a git repository (or any of the parent directories): .git
2019-08-30 06:46:33,865 - DEBUG - __main__ - Starting...
2019-08-30 06:46:33,866 - DEBUG - report - Setting up...
2019-08-30 06:46:33,866 - DEBUG - container - Checking if image "yarara/base:v1" is available on disk...
2019-08-30 06:46:33,871 - DEBUG - container - Attempting to pull image "yarara/base:v1"
2019-08-30 06:46:56,728 - DEBUG - container - Image "yarara/base:v1" downloaded
2019-08-30 06:47:05,844 - DEBUG - rootfs - Running command: sudo /usr/local/lib/python3.6/dist-packages/tern/tools/fs_hash.sh /home/vagrant/temp/3bf9dc6ca1042e8f1e9ce7b6a62bbe1ea0c224323b6fbc34e5e5e35905cd4206/contents
2019-08-30 06:47:08,952 - ERROR - rootfs - Command failed. /usr/local/lib/python3.6/dist-packages/tern/tools/fs_hash.sh: 22: cd: can't cd to /home/vagrant/temp/3bf9dc6ca1042e8f1e9ce7b6a62bbe1ea0c224323b6fbc34e5e5e35905cd4206/contents

2019-08-30 06:47:08,953 - WARNING - report - Cannot retrieve full image metadata
2019-08-30 06:47:08,953 - DEBUG - report - Teardown...
2019-08-30 06:47:08,954 - DEBUG - rootfs - Running command: sudo rm -rf temp/mergedir
2019-08-30 06:47:08,965 - DEBUG - rootfs - Running command: sudo rm -rf temp/workdir
2019-08-30 06:47:08,975 - DEBUG - __main__ - Report completed.
2019-08-30 06:47:08,976 - DEBUG - __main__ - Finished
@nishakm
Copy link
Contributor

nishakm commented Aug 30, 2019 via email

@nishakm nishakm added the bug Something went wrong label Aug 30, 2019
@nishakm nishakm added this to the Release 1.0.0 milestone Aug 30, 2019
@nishakm nishakm changed the title When Tern executes fs_hash.sh, it cannot cd to the contens directory. When Tern executes fs_hash.sh, it cannot cd to the contents directory. Aug 30, 2019
@nishakm nishakm self-assigned this Aug 30, 2019
@nishakm
Copy link
Contributor

nishakm commented Aug 30, 2019

Python3.7's tarfile module will not extract these layers.

@PrajwalM2212
Copy link
Contributor

PrajwalM2212 commented Aug 30, 2019 •
edited
Loading

Yes, tar.extractall(directory_path) is raising the error.
I had got the extraction error for another image as well but was not sure if it was related to this but now we now that tarfile module is the source of error.

@nishakm
Copy link
Contributor

nishakm commented Aug 30, 2019

@Katsunori04 it looks like the image you are using is corrupted. See tar error below.

vagrant@ubuntu-bionic:~/temp/1b73c719e179f6a7702a653f2753c4ab459d5e379a483d6ce8888104f5c1ae1c$ tar xvf layer.tar 
./
.wh..wh.aufs
.wh..wh.orph/
.wh..wh.plnk/
dev/
dev/ptmx
etc/
etc/apt/
etc/apt/apt.conf.d/
etc/apt/apt.conf.d/no-cache
etc/apt/apt.conf.d/no-languages
etc/dpkg/
etc/dpkg/dpkg.cfg.d/
etc/dpkg/dpkg.cfg.d/02apt-speedup
sbin/
sbin/initctl
sbin/initctl.distrib
usr/
usr/sbin/
usr/sbin/policy-rc.d
var/
var/lib/
var/lib/dpkg/
var/lib/dpkg/diversions
var/lib/dpkg/diversions-old
var/lib/dpkg/status
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now

What happened was that the first layer that tern hit was an empty tarfile, and because of this, tar exited with no error but because nothing was extracted it failed.
We have a bug being worked on that checks tarfile elements before extracting: #226.

Meanwhile, I think the image you have used is corrupted.

@nishakm
Copy link
Contributor

nishakm commented Aug 30, 2019

Yes, tar.extractall(directory_path) is raising the error.
I had got the extraction error for another image as well but was not sure if it was related to this but now we now that tarfile module is the source of error.

What was the other image you used?

@nishakm nishakm removed this from the Release 1.0.0 milestone Aug 30, 2019
@PrajwalM2212
Copy link
Contributor

@nishakm ubuntu:14.04

@nishakm
Copy link
Contributor

nishakm commented Aug 30, 2019

OK, the issue with the ubuntu:14.04 is a permissions issue rather than a tarball extraction issue. So closing this one.

@nishakm nishakm closed this as completed Aug 30, 2019
@Katsunori04
Copy link
Author

Katsunori04 commented Aug 30, 2019 •
edited
Loading

@nishakm
Thank you so much. I feel that the image is damaged. I didn't notice.
I also found similar errors in other images.

I feel this is not a corruption error. However, the contents of the tarfile are missing. Is this issue affected by # 226?

To Reproduce

vagrant@ubuntu-bionic:~$ tern -l -k report -i nubs/arch-build:latest
fatal: not a git repository (or any of the parent directories): .git
2019-08-30 16:41:21,882 - DEBUG - __main__ - Starting...
2019-08-30 16:41:21,883 - DEBUG - report - Setting up...
2019-08-30 16:41:21,913 - DEBUG - container - Checking if image "nubs/arch-build:latest" is available on disk...
2019-08-30 16:41:21,919 - DEBUG - container - Image "nubs/arch-build:latest" found
2019-08-30 16:42:21,245 - DEBUG - rootfs - Running command: sudo /usr/local/lib/python3.6/dist-packages/tern/tools/fs_hash.sh /home/vagrant/temp/3bf9dc6ca1042e8f1e9ce7b6a62bbe1ea0c224323b6fbc34e5e5e35905cd4206/contents
2019-08-30 16:42:59,468 - ERROR - rootfs - Command failed. /usr/local/lib/python3.6/dist-packages/tern/tools/fs_hash.sh: 22: cd: can't cd to /home/vagrant/temp/3bf9dc6ca1042e8f1e9ce7b6a62bbe1ea0c224323b6fbc34e5e5e35905cd4206/contents

2019-08-30 16:42:59,469 - WARNING - report - Cannot retrieve full image metadata
2019-08-30 16:42:59,469 - DEBUG - report - Teardown...
2019-08-30 16:42:59,485 - DEBUG - rootfs - Running command: sudo rm -rf temp/mergedir
2019-08-30 16:42:59,494 - DEBUG - rootfs - Running command: sudo rm -rf temp/workdir
2019-08-30 16:42:59,505 - DEBUG - __main__ - Report completed.
2019-08-30 16:42:59,505 - DEBUG - __main__ - Finished

@nishakm
Copy link
Contributor

nishakm commented Aug 30, 2019 •
edited
Loading

@Katsunori04 I'm not sure about the status of the images you are testing here, but if the contents of the tarfile are missing, tern can't give you any information anyway. #226 will add extra error messages informing as such but the script will exit out gracefully with no information.

This was referenced Aug 30, 2019
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nishakm nishakm

Labels
bug Something went wrong
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@nishakm @Katsunori04 @PrajwalM2212