Tests behave oddly in certain environments (debian packages) #1973
Comments
|
Thank you for reporting it separately! |
|
As a sideeffect https://build.libelektra.org/jenkins/job/elektra-git-buildpackage-jessie/ seems to be working again since yesterday... |
|
GPG seems to have a problem with the cleared environment. I will experiment with setting the |
|
Yes, I also thought that the failing crypto test cases are related to #1928. It seems that gpg uses .gnupg below the TMPDIR. (At least you see a .gnupg leftover after running The failing fcrypt testcases might show a bug in the handling of temporary files, also related to TMPDIR. |
this is a wrong assumption. gpg uses ~/.gnupg as a default so it is not related to TMPDIR but to HOME. Please also take note that setting GNUPGHOME to the same directory for all tests might influence parallel test execution. |
Which assumption do you mean? I was only referring that I get a |
|
that is because something in the tests(ctest?) sets the home directory for each test suite to a tmpdir in TMPDIR. I think it is a good idea to have seperate home dirs for each test so they do not influence each other (parallelisation) and hence do not think that overriding gpg directories to share a single directory is a good idea. |
|
Yes, I agree. Does #1977 make sense? Or should it be done after Or asked differently: Does the gpg-agent survives a run of a single test suite or are they respawned for every test suite? |
|
@markus2330 As mentioned in #2040 . I followed along the instructions on building the Docker images. Inside the docker container I ran: |
|
So (a part of) the problem is that the test cases assumes the TMPDIR to be present. And you could not reproduce it because you always created the directory before running the tests 😄 But, for me fcrypt nevertheless fails, even when x is present (you did not run fcrypt). But with other messages: |
|
But it works in my container (I've been waiting a long time to drop that line 😆 )... |
|
some more interesting stuff:
|
|
well... check the warnings for tests.c and everything becomes much more clearer... I am prepping a PR for it though which should resolve some of the issues. If somebody could find out if fcrypt has some hard paths specified somewhere that could lead to the tests failing that would be lovely. |
|
Ìn #2056 I cleanup the generated TMPDIR properly. This resolves the lingering gpg-agent issue. However the fcrypt test still fails as soon as TMPDIR is set manually and I think there might be a real bug in the code somewhere. jenkins@0585a86e6f5f:~/workspace/build$ pidof gpg-agent
jenkins@0585a86e6f5f:~/workspace/build$ TMPDIR=`pwd`/tmp ctest --output-on-failure -R crypt
Test project /home/jenkins/workspace/build
Start 51: testmod_crypto_openssl
1/4 Test #51: testmod_crypto_openssl ........... Passed 1.58 sec
Start 52: testmod_crypto_gcrypt
2/4 Test #52: testmod_crypto_gcrypt ............ Passed 1.34 sec
Start 53: testmod_crypto_botan
3/4 Test #53: testmod_crypto_botan ............. Passed 1.83 sec
Start 64: testmod_fcrypt
4/4 Test #64: testmod_fcrypt ...................***Failed 1.17 sec
FCRYPT TESTS
==================
/home/jenkins/workspace/src/plugins/fcrypt/testmod_fcrypt.c:187: error in test_file_crypto_operations: kdb set failed
/home/jenkins/workspace/src/plugins/fcrypt/testmod_fcrypt.c:188: error in test_file_crypto_operations: file content did not change during encryption
/home/jenkins/workspace/src/plugins/fcrypt/testmod_fcrypt.c:191: error in test_file_crypto_operations: kdb get (pregetstorage) failed
/home/jenkins/workspace/src/plugins/fcrypt/testmod_fcrypt.c:195: error in test_file_crypto_operations: kdb get (postgetstorage) failed
/home/jenkins/workspace/src/plugins/fcrypt/testmod_fcrypt.c:196: error in test_file_crypto_operations: postgetstorage did not encrypt the file again
/home/jenkins/workspace/src/plugins/fcrypt/testmod_fcrypt.c:231: error in test_file_signature_operations: kdb set failed
/home/jenkins/workspace/src/plugins/fcrypt/testmod_fcrypt.c:232: error in test_file_signature_operations: file content did not change during encryption
/home/jenkins/workspace/src/plugins/fcrypt/testmod_fcrypt.c:235: error in test_file_signature_operations: kdb get failed
fcrypt Results: 39 Tests done — 8 errors.
75% tests passed, 1 tests failed out of 4
Label Time Summary:
memleak = 1.34 sec (1 test)
Total Test time (real) = 5.98 sec
The following tests FAILED:
64 - testmod_fcrypt (Failed)
Errors while running CTest
jenkins@0585a86e6f5f:~/workspace/build$ pidof gpg-agent
jenkins@0585a86e6f5f:~/workspace/build$ ls -l tmp
total 0 |
You are right!
I suppose the correct prioritization would be:
Do you agree? |
…ary directory See ElektraInitiative#1973 for further details
Since we are using mkdtemp (POSIX 2008) anyway you could probably use that as well and get rid of some code. |
|
Thank you for the info! Have you been able to verify that the proposed fix in #2090 fixes the |
See ElektraInitiative#1973 for further details.
…ary directory See ElektraInitiative#1973 for further details
See ElektraInitiative#1973 for further details.
|
There still seems to be some kind of oddity with the tests. I run some builds of master mostly to finetune executor and parallel settings since we don't do hyperthreading anymore. And in one case the tests suddenly failed with the same 'could not install key' problem again: |
|
Sorry to pile on here, but I ran into the I can provide the vagrantfile via PR if it helps to reproduce the errors. |
|
Thank you for reporting! Can you tell us your TMPDIR and HOME? Try to reproduce it with @petermax2 Do you have time to look at it? |
|
For the errors above: As for Not sure if this is a coincidence, but this was the first time that this test failed. |
|
I can investigate over the weekend. @mpranj could you send me the vagrant file please? |
|
@petermax2 the Vagrantfile & steps to reproduce are in #2211. I hope this helps do debug the problem more easily. Maybe it is really just an ENV problem. |
|
EDIT: EDIT v2: |
|
Thank you for the investigation! Is there some overview in which version which env vars do what and since when the gpg agent closes when the files are removed? What is the best way to fix this? |
|
Sorry for the delay, I had no time to further investigate so far.
All the documentation I could find was (GNUPGHOME) If set directory used instead of "~/.gnupg". [1]
I am not sure. It does not seem like there is a single solution that solves all the problems we are facing with GnuPG. I once tried to save the PIDs of startetd GnuPG agents but it did not turn out well. Do you have any ideas? Sources |
|
Maybe we should not try to solve every possible problem and only support some versions of gnupg? Which versions of gnupg do not clean themselves up after removing their files? Are they relevant enough to still support them? Maybe we simply disable the tests for them? |
Ubuntu Artful (17.10)'s version of gpg is the only version so far where the agent does not terminate after deleting the The GPG in Ubuntu Artful is 2.1.15. In Ubuntu 18.04.1 gpg version 2.2.4 is used. The latest version is 2.2.10. We might add a dependency note that gpg version >= 2.2 is required. |
|
If if understand the release notes correctly the termination is implemented in gpg version 2.1.16. See Release Notes. EDIT: the release note states that the termination is only implemented for Linux. Meaning that on other OS we should get the same problem. I'll test it on FreeBSD. |
Yep, |
|
ATM I'm testing another solution. Executing |
Hopefully once and for all fixes the infamous ElektraInitiative#1973 .
Hopefully once and for all fixes the infamous ElektraInitiative#1973 .
|
Please re-open this issue if the problem occurs again. |
Before bc804c2 the following tests failed in some environments when building debian packages:
EDIT markus: You can also reproduce the blockresolver and fcrypt problems by manipulating HOME and TMPDIR. No building of a package is needed.
errorlogs
blockresolver
crypto_openssl
crypto_gcrypt
crypto_botan
fcrypt
discussion
In #1965 we had some bug finding discussions ultimately pointing towards 356317a as the commit that made the errors showing up. However cleaning the environment should not have the side effect of making tests pass.
If tests need certain environmental settings they should set them.
The text was updated successfully, but these errors were encountered: