Make Cython work without recipe

[ghost]

What this does: This merge request makes Cython work for the non-recipe regular build. This means CythonRecipe is now unnecessary if the project doesn't need Android-specific extra changes. That's good because a recipe with no actual content is redundant, and because CythonRecipe is problematic:

Why CythonRecipe is problematic: I noticed CythonRecipe results in the packages own setup.py not run for Cythonization, and instead, p4a engages some manual Cython compilation which is broken per design for a couple of reasons:

1. it assumes all .pyx files anywhere in the project dir must compile. This breaks projects that have any .pyx files that are not meant to be built for the main build and break when Cythonized out of the blue
2. it uses the wrong site-packages because it uses host python. This breaks .pxd cross-includes from any dependencies, since this requires proper site-packages during Cythonize (this affected me, see .pxd files of dependency targeted recipe'd library not found #1473 )
3. it assumes the setup.py won't produce new .pyx files on its own. This breaks projects where that happens

(Please note this pull request doesn't fix CythonRecipe in any way, it just makes it avoidable for Android-aware Cython projects)

Needs quite some testing, obviously!

[ghost]

Side note: I'm unhappy about how this pull request makes things based on CythonRecipe for the env vars of the regular build, since I think the CythonRecipe should be a special case of special needs Cython projects, not the other way around. However, I didn't take care of this yet because I wanted to first see if this works for other projects at all, and then gradually move stuff out of CythonRecipe and into the core later, stripping down CythonRecipe to its special uses in that process.

(Also, it would move quite a lot of stuff around, so just for cleanliness I think it'd be nicer to have it in a separate pull request later)

[tito]

About your 1. just to let you know Kivy source code have pyx that doesn't work and required on Android (like window_x11.pyx, etc). So maybe you should have an option to give a list of files to ignore?

[ghost]

About your 1. just to let you know Kivy source code have pyx that doesn't work and required on Android (like window_x11.pyx, etc). So maybe you should have an option to give a list of files to ignore?

That's like a hack of another hack. The real problem is that the setup.py already does this for any sane project, and p4a's CythonRecipe basically assumes the setup.py never works for any project for Cythonizing which IMHO is just a bad assumption. (It would be ok as a manual per-recipe enabled workaround, but as a default behavior for Cython projects I don't think it makes a lot of sense)

Also, I don't have any non-compiling .pyx files, I was just explaining why CythonRecipe is problematic.

(Nevertheless, thanks for your input! Interesting to know @ kivy, I wonder how it works around this - but still, I don't find it directly relevant for this pull request)

[ghost]

It appears to be working mostly fine, but I can't fully test since I'm currently stuck on #1487

edit: resolved, since #1487 is now fixed

[ghost]

Edit: resolved (was an error due to wrong LDFLAGS env var)

[ghost]

I spent quite some time making the process much more similar to the CythonRecipe, even down to stripping the object files at the end - but sadly, I still get above error. Edit: now solved! yay

@inclement or @AndreMiras you wouldn't happen to have a clue why with the above gcc command line, I appear to be getting broken .so files that give that cannot locate symbol "PyExc_DeprecationWarning" error?

edit: done, see below

[ghost]

Ok thanks to a hint from @AndreMiras and some more debugging I CAN NOW CONFIRM IT WORKING for python3crystax, full build process up to deploying & actually running the app.

[ghost]

(this is still WIP though, I want to clean up some stuff - but I'd love to hear test results!! Even if your app doesn't use Cython, just to make sure other things aren't broken by this)

edit: done, see below

[ghost]

Okay, I cleaned up the parts I still wanted to clean up and it works for me.

To test this and give me useful feedback, it is enough to just build your project with this pull request. While your project/libraries would need adjustments to use the new functionality, the much more important question is whether this pull request breaks existing stuff. It changes the regular non-recipe python module install env, so it has the potential to cause some interference, so some wider testing wouldn't be a bad idea.

Anyone who's interested in testing, please do and report back here! It works fine for me is all I can say

[ghost]

Just a tiny note: latest force pushes were just rebasing, not changes. It works for me, feel free to test to see if it works for you!

[ghost]

@AndreMiras by the way, any clue on when we're going to merge this? It's pretty much been sitting here for three weeks, I use it for my active project and it's done as far as I'm concerned... so IMHO it would make sense to review & merge this if you guys want this feature. I personally think moving away from recipes where they aren't strictly necessary is always an improvement, but that's just my view on things

Edit: force pushes are just me rebasing, I'm not changing it or anything

[inclement]

Thanks, sorry for the delay on this one.

[AndreMiras]

Minor: isn't this redundant with the one two lines above?

[ghost]

oops yeah, I had a conflict there and must have messed that line during merge/rebase

…

On January 13, 2019 6:08:19 PM GMT+01:00, Andre Miras ***@***.***> wrote: AndreMiras commented on this pull request. > info('Stripping object files') if self.ctx.python_recipe.name == 'python2legacy': + info('Stripping object files') Minor: isn't this redundant with the one two lines above? -- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: #1483 (review)

[AndreMiras]

Well that's not a big deal anyway, very good work as usual and sorry for letting the PR rot for so long. Great that inclement took over and merged