Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ext/imap/config.m4: -Werror=implicit-function-declaration compatibility. #10948

Closed
wants to merge 3 commits into from
Closed

ext/imap/config.m4: -Werror=implicit-function-declaration compatibility. #10948

wants to merge 3 commits into from

Conversation

orlitzky
Copy link
Contributor

@orlitzky orlitzky commented Mar 26, 2023

The recent clang-16 throws errors for implicitly defined functions by default. In many ./configure tests, an undefined function (which is "implicitly defined" when you try to call it) is undefined because it really does not exist. But in one case, utf8_to_mutf7() is undefined because we forgot to include the header that defines it.

This commit updates the test for utf8_to_mutf7:

  • We now include the header (c-client.h) that defines it.
  • A "checking... yes/no" message was added to the test.
  • The test was switched from PHP_IMAP_TEST_BUILD to AC_LINK_IFELSE. This was necessary to avoid return-type mismatches that can run afoul of -Werror=implicit-int.
  • CPPFLAGS is temporarily amended with the -I flag needed to find c-client.h.

Closes GH-10947.

$TST_LIBS
], [
#include <c-client.h>
char utf8_to_mutf7_php(){ return utf8_to_mutf7(""); }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice trick but now it complains about type-casting because the function return pointer

/mnt $ cat /usr/include/imap/utf8aux.h |grep utf8_to_mutf7
unsigned char *utf8_to_mutf7 (unsigned char *src);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

checking whether utf8_to_mutf7 function present... no

configure:46303: checking whether utf8_to_mutf7 function present
configure:46326: clang -o conftest -O2 -fomit-frame-pointer -g  -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -fvisibility=hidden -Os -fomit-frame-pointer -D_GNU_SOURCE -I/usr/include/imap -Wl,--as-needed,-O1,--sort-common -Wl,-rpath,/lib -L/lib conftest.c 
       -lc-client -lcrypt  -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lssl -lcrypto
     -lrt -lm -lacl  -lxml2 -lpcre2-8 -lz >&5
conftest.c:327:48: warning: passing 'char[1]' to parameter of type 'unsigned char *' converts between pointers to integer types where one is of the unique plain 'char' type and the other is not [-Wpointer-sign]
char utf8_to_mutf7_php(){ return utf8_to_mutf7(""); }
                                               ^~
/usr/include/imap/utf8aux.h:43:46: note: passing argument to parameter 'src' here
unsigned char *utf8_to_mutf7 (unsigned char *src);
                                             ^
conftest.c:327:34: error: incompatible pointer to integer conversion returning 'unsigned char *' from a function with result type 'char' [-Wint-conversion]
char utf8_to_mutf7_php(){ return utf8_to_mutf7(""); }
                                 ^~~~~~~~~~~~~~~~~
1 warning and 1 error generated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Argh, and it's using the wrong type because PHP_TEST_BUILD declares every function as type char. I'll have to use one of the bare autoconf macros, it looks like.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the same reason it fails for utf8_mime2text

configure:45093: checking for utf8_mime2text signature
configure:45116: clang -c -I/usr/include/imap -Os -fomit-frame-pointer -D_GNU_SOURCE conftest.c >&5
conftest.c:329:32: error: too few arguments to function call, expected 3, have 2
        utf8_mime2text(src, dst);
        ~~~~~~~~~~~~~~         ^
/usr/include/imap/utf8aux.h:37:6: note: 'utf8_mime2text' declared here
long utf8_mime2text (SIZEDTEXT *src,SIZEDTEXT *dst,long flags);
     ^
1 error generated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the mime2text thing is intended? It's trying to figure out of the signature is the old one with two args or the new one with three.

@Girgias
Copy link
Member

Girgias commented Mar 26, 2023

Somehow tests now fails because it cannot detect the function anymore, not sure if this is legit or not, but maybe only the SKIPIF section of the relevant tests need to be amended

@orlitzky
Copy link
Contributor Author

Somehow tests now fails because it cannot detect the function anymore, not sure if this is legit or not, but maybe only the SKIPIF section of the relevant tests need to be amended

Well, the test certainly needs a SKIPIF (it has none), since the feature is optional... but that doesn't explain why the function is no longer detected. Is there a way to get config.log from the CI? Or are you lucky enough to be able to reproduce it?

@Girgias
Copy link
Member

Girgias commented Mar 26, 2023

Somehow tests now fails because it cannot detect the function anymore, not sure if this is legit or not, but maybe only the SKIPIF section of the relevant tests need to be amended

Well, the test certainly needs a SKIPIF (it has none), since the feature is optional... but that doesn't explain why the function is no longer detected. Is there a way to get config.log from the CI? Or are you lucky enough to be able to reproduce it?

I'm on Fedora now so it's a pain to get ext/imap to compile since CClient is not part of the RPMs any more. And I'm not aware of a good way to get the config.log from CI :/ @iluuu1994 maybe knows?

@orlitzky
Copy link
Contributor Author

The latest force-push refactors to use a cached check, and AC_LANG_PROGRAM instead of AC_LANG_SOURCE. I don't expect the behavior to change.

I could try switching it to a compile test, as opposed to a link test, to see if that helps. But regardless of the result, I would not want to move forward without an explanation of why the test is failing on the CI machine.

@iluuu1994
Copy link
Member

@orlitzky You could probably just cat the config.log by adding it to the failing entry in push.yml after the config step.

- if: always()
  run: cat config.log

@orlitzky
Copy link
Contributor Author

Brilliant, thanks. Let's see if I did that right...

@orlitzky
Copy link
Contributor Author

Switching away from PHP_IMAP_TEST_BUILD probably caused this. The config.log says,

2023-03-26T23:56:25.2427995Z configure:46316: checking for utf8_to_mutf7
2023-03-26T23:56:25.2429176Z configure:46335: ccache gcc -o conftest -g -O2 -fvisibility=hidden -pthread  -D_GNU_SOURCE -D_REENTRANT -I/usr/include/c-client  -Wl,-rpath,/usr/lib/x86_64-linux-gnu/mit-krb5 -L/usr/lib/x86_64-linux-gnu/mit-krb5 conftest.c -lc-client   -lcrypt -lpam -lgmp -llmdb -ltokyocabinet -lqdbm -lbz2 -lrt -lm  -lpthread -lxml2 -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lssl -lcrypto -lsqlite3 -lz -lcurl -lssl -lcrypto -lxml2 -lenchant-2 -lffi -lssl -lcrypto -lz -lpng16 -lz -lwebp -ljpeg -lXpm -lX11 -lfreetype -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err -lssl -lcrypto >&5
2023-03-26T23:56:25.2429437Z In file included from /usr/include/x86_64-linux-gnu/bits/libc-header-start.h:33,
2023-03-26T23:56:25.2429558Z                  from /usr/include/stdlib.h:26,
2023-03-26T23:56:25.2429746Z                  from /usr/include/c-client/osdep.h:43,
2023-03-26T23:56:25.2429921Z                  from /usr/include/c-client/c-client.h:42,
2023-03-26T23:56:25.2430020Z                  from conftest.c:313:
2023-03-26T23:56:25.2430347Z /usr/include/features.h:194:3: warning: #warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE" [-Wcpp]
2023-03-26T23:56:25.2430512Z   194 | # warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE"
2023-03-26T23:56:25.2430590Z       |   ^~~~~~~
2023-03-26T23:56:25.2430889Z /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/libc-client.so: undefined reference to `mm_status'
2023-03-26T23:56:25.2431193Z /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/libc-client.so: undefined reference to `mm_nocritical'
2023-03-26T23:56:25.2431494Z /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/libc-client.so: undefined reference to `mm_critical'
2023-03-26T23:56:25.2431786Z /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/libc-client.so: undefined reference to `mm_fatal'
2023-03-26T23:56:25.2432071Z /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/libc-client.so: undefined reference to `mm_list'
2023-03-26T23:56:25.2432360Z /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/libc-client.so: undefined reference to `mm_login'
2023-03-26T23:56:25.2432642Z /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/libc-client.so: undefined reference to `mm_log'
2023-03-26T23:56:25.2432937Z /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/libc-client.so: undefined reference to `mm_diskerror'
2023-03-26T23:56:25.2433222Z /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/libc-client.so: undefined reference to `mm_dlog'
2023-03-26T23:56:25.2433634Z /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/libc-client.so: undefined reference to `mm_exists'
2023-03-26T23:56:25.2433955Z /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/libc-client.so: undefined reference to `mm_flags'
2023-03-26T23:56:25.2434246Z /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/libc-client.so: undefined reference to `mm_notify'
2023-03-26T23:56:25.2434537Z /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/libc-client.so: undefined reference to `mm_expunged'
2023-03-26T23:56:25.2434817Z /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/libc-client.so: undefined reference to `mm_lsub'
2023-03-26T23:56:25.2435110Z /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib/libc-client.so: undefined reference to `mm_searched'
2023-03-26T23:56:25.2435212Z collect2: error: ld returned 1 exit status

The PHP_IMAP_TEST_BUILD macro somehow fixes this with,

# define PHP_IMAP_EXPORT __attribute__ ((visibility("default")))
PHP_IMAP_EXPORT void mm_log(void){}
PHP_IMAP_EXPORT void mm_dlog(void){}
PHP_IMAP_EXPORT void mm_flags(void){}
...

Of course, the only reason I switched away from it in the first place is because those signatures are wrong, although that was probably just for convenience. On the one hand, I'm now relatively confident that a compile test (as opposed to a link test) would be fine here. But on the other, this has made me curious why things work on my Gentoo system but not the Ubuntu CI.

I'll think about it some more tomorrow and decide what to do.

The recent clang-16 throws errors for implicitly defined functions by
default. In many ./configure tests, an undefined function (which is
"implicitly defined" when you try to call it) is undefined because it
really does not exist. But in one case, utf8_to_mutf7() is undefined
because we forgot to include the header that defines it.

This commit updates the test for utf8_to_mutf7:

  * We now include the header (c-client.h) that defines it.
  * A "checking... yes/no" message was added to the test.
  * The test was switched from PHP_IMAP_TEST_BUILD to AC_COMPILE_IFELSE.
    This was the easiest way to avoid a return-type mismatch that runs
    afoul of -Werror=implicit-int.
  * CPPFLAGS is temporarily amended with the -I flag needed to find
    c-client.h.

Closes GH-10947.
The PHP_IMAP_TEST_BUILD macro defines a bunch of public no-op
functions for apparently no reason before delegating to
PHP_TEST_BUILD. Well, there is a reason, and now that I've
rediscovered it, this commit adds a comment explaining it.
@orlitzky
Copy link
Contributor Author

It should all be sorted now.

First, c-client is in bad shape these days. If my count is correct, the last release was 16 years ago and we have 22 patches for it in Gentoo (14 of those come from Debian). It has two users that I know of, PHP and Asterisk, so it might be worthwhile to fork it. Right now it's being kept alive by distro patches, but we're duplicating a lot of work that could be avoided by having a canonical upstream repo. PHP is also working extra hard to detect all of the weird ways that the various distros might patch and package it.

But back to this PR: the c-client patch monster that everyone is shipping was never actually designed to be a shared library. The mm_foo() functions that PHP_IMAP_TEST_BUILD defines are "callbacks", meant to be supplied by the program linking with c-client. Without them, the library itself has undefined symbols and linking will fail.

I didn't notice it because (1) I have -Wl,--as-needed in my LDFLAGS, and (2) these tests are trivial enough to be optimized out. So tl;dr, I wasn't actually linking with c-client. But without --as-needed, or maybe without compiler optimizations, linking will fail unless you define all of those mm_foo() callbacks.

That satisfies my curiosity, and makes me confident that a compile test (not a link test) would be fine here. That way, the undefined symbols don't matter. (PHP does define those callbacks in the IMAP extension, so the real link phase works fine.) I've switched the test to AC_COMPILE_IFELSE, and have now added a comment to PHP_IMAP_TEST_BUILD containing more or less what I've just explained.

@TimWolla TimWolla removed their request for review March 27, 2023 16:03
@TimWolla
Copy link
Member

@orlitzky You could probably just cat the config.log by adding it to the failing entry in push.yml after the config step.

@iluuu1994 Should we add that as a step to all CI by default? It would automatically be collapsed anyway. Alternatively it could be uploaded as an artifact.

@andypost
Copy link
Contributor

Is there a way to prevent this warning?

configure:46311: checking for utf8_to_mutf7
configure:46330: clang -c -O2 -fomit-frame-pointer -g  -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64  -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -fvisibility=hidden -Os -fomit-frame-pointer -D_GNU_SOURCE -I/usr/include/imap conftest.c >&5
conftest.c:329:23: warning: passing 'char[1]' to parameter of type 'unsigned char *' converts between pointers to integer types where one is of the unique plain 'char' type and the other is not [-Wpointer-sign]
        utf8_to_mutf7("");
                      ^~
/usr/include/imap/utf8aux.h:43:46: note: passing argument to parameter 'src' here
unsigned char *utf8_to_mutf7 (unsigned char *src);
                                             ^
1 warning generated.

@iluuu1994
Copy link
Member

@iluuu1994 Should we add that as a step to all CI by default? It would automatically be collapsed anyway. Alternatively it could be uploaded as an artifact.

I've not needed this output until now, but seems harmless.

AC_LANG_PUSH(C)
AC_CACHE_CHECK(for utf8_to_mutf7, ac_cv_utf8_to_mutf7,
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <c-client.h>]],[[
utf8_to_mutf7("");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use unsigned char *c; utf8_to_mutf7(c);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or maybe unsigned char c; utf8_to_mutf7(&c); is better

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or maybe unsigned char c; utf8_to_mutf7(&c); is better

Yep, done. Thanks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, used as patch and it works for 8.2 and master

The existing test for utf8_to_mutf7() passes the empty string (of type
char*) to it, but the function expects an unsigned char pointer. This
raises a warning with -Wpointer-sign. We fix it by passing in an
argument of the correct type.
@andypost
Copy link
Contributor

andypost commented Mar 27, 2023

@Girgias
Copy link
Member

Girgias commented Mar 27, 2023

It should all be sorted now.

First, c-client is in bad shape these days. If my count is correct, the last release was 16 years ago and we have 22 patches for it in Gentoo (14 of those come from Debian). It has two users that I know of, PHP and Asterisk, so it might be worthwhile to fork it. Right now it's being kept alive by distro patches, but we're duplicating a lot of work that could be avoided by having a canonical upstream repo. PHP is also working extra hard to detect all of the weird ways that the various distros might patch and package it.

But back to this PR: the c-client patch monster that everyone is shipping was never actually designed to be a shared library. The mm_foo() functions that PHP_IMAP_TEST_BUILD defines are "callbacks", meant to be supplied by the program linking with c-client. Without them, the library itself has undefined symbols and linking will fail.

I didn't notice it because (1) I have -Wl,--as-needed in my LDFLAGS, and (2) these tests are trivial enough to be optimized out. So tl;dr, I wasn't actually linking with c-client. But without --as-needed, or maybe without compiler optimizations, linking will fail unless you define all of those mm_foo() callbacks.

That satisfies my curiosity, and makes me confident that a compile test (not a link test) would be fine here. That way, the undefined symbols don't matter. (PHP does define those callbacks in the IMAP extension, so the real link phase works fine.) I've switched the test to AC_COMPILE_IFELSE, and have now added a comment to PHP_IMAP_TEST_BUILD containing more or less what I've just explained.

Yeah, I think we really need to start considering moving ext/imap to PECL due to the whole c-client issue. But thank you for taking care of this!

@Girgias Girgias closed this in f9cbeaa Mar 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Building with clang 16
5 participants