-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Generate Drake exports from CPS #6014
Generate Drake exports from CPS #6014
Conversation
I will look late tonight or first thing tomorrow. |
cbd6cce
to
b83b6c9
Compare
Review status: 0 of 9 files reviewed at latest revision, 4 unresolved discussions. WORKSPACE, line 298 at r1 (raw file):
Use the PyPI version:
WORKSPACE, line 307 at r1 (raw file):
This needs a tools/semantic_version.BUILD, line 3 at r1 (raw file):
The tools/semantic_version.BUILD, line 11 at r1 (raw file):
Remove Comments from Reviewable |
Review status: 0 of 9 files reviewed at latest revision, 7 unresolved discussions. tools/pycps.BUILD, line 9 at r1 (raw file):
Probably should be named tools/pycps.BUILD, line 11 at r1 (raw file):
tools/pycps.BUILD, line 15 at r1 (raw file):
Should be named Comments from Reviewable |
Review status: 0 of 9 files reviewed at latest revision, 7 unresolved discussions. WORKSPACE, line 298 at r1 (raw file):
Quoted 4 lines of code…> "https://pypi.python.org/packages/source/s/semantic_version/semantic_version-2.6.0.tar.gz", > "https://d2tbce6hkathzp.cloudfront.net/pypi/semantic_version/semantic_version-2.6.0.tar.gz", > "https://s3.amazonaws.com/drake-mirror/pypi/semantic_version/semantic_version-2.6.0.tar.gz", > ]
Comments from Reviewable |
Review status: 0 of 9 files reviewed at latest revision, 7 unresolved discussions. WORKSPACE, line 298 at r1 (raw file): Previously, jamiesnape (Jamie Snape) wrote…
Why? What is the advantage of PyPI vs. upstream? WORKSPACE, line 307 at r1 (raw file): Previously, jamiesnape (Jamie Snape) wrote…
Um... why? It seems to be Doing The Right Thing without it... tools/pycps.BUILD, line 15 at r1 (raw file): Previously, jamiesnape (Jamie Snape) wrote…
Can't; conflicts with the source file. This is (IMHO) really, really ugly, but AFAICT Bazel forces me to add some junk to the target name to disambiguate. If there's a better way to export a source file that is already an executable script (that also needs dependencies), I'd love to know! (Note: at least one person on bazel-discuss already implied that there is not... 😢) tools/semantic_version.BUILD, line 11 at r1 (raw file): Previously, jamiesnape (Jamie Snape) wrote…
Is it picked up automatically? Comments from Reviewable |
Review status: 0 of 9 files reviewed at latest revision, 7 unresolved discussions. tools/pycps.BUILD, line 9 at r1 (raw file): Previously, jamiesnape (Jamie Snape) wrote…
Done. tools/pycps.BUILD, line 11 at r1 (raw file): Previously, jamiesnape (Jamie Snape) wrote…
Done. Comments from Reviewable |
b83b6c9
to
2997a8f
Compare
Review status: 0 of 9 files reviewed at latest revision, 5 unresolved discussions. WORKSPACE, line 298 at r1 (raw file): Previously, mwoehlke-kitware (Matthew Woehlke) wrote…
It is consistent with the other Python packages that we use (and I already mirrored the PyPI version). WORKSPACE, line 307 at r1 (raw file): Previously, mwoehlke-kitware (Matthew Woehlke) wrote…
It see, you are copying to change the file extension. Nevermind. tools/semantic_version.BUILD, line 11 at r1 (raw file): Previously, mwoehlke-kitware (Matthew Woehlke) wrote…
That level will be removed. If it were not, you would add an Comments from Reviewable |
Review status: 0 of 9 files reviewed at latest revision, 4 unresolved discussions. tools/pycps.BUILD, line 15 at r1 (raw file): Previously, mwoehlke-kitware (Matthew Woehlke) wrote…
Not that I can think of at the moment. Comments from Reviewable |
Review status: 0 of 9 files reviewed at latest revision, 3 unresolved discussions. WORKSPACE, line 298 at r1 (raw file): Previously, jamiesnape (Jamie Snape) wrote…
https://pypi.python.org/packages/source/s/semantic_version/semantic_version-2.6.0.tar.gz → 404? Comments from Reviewable |
Review status: 0 of 9 files reviewed at latest revision, 3 unresolved discussions. WORKSPACE, line 298 at r1 (raw file): Previously, mwoehlke-kitware (Matthew Woehlke) wrote…
https://files.pythonhosted.org/packages/source/s/semantic_version/semantic_version-2.6.0.tar.gz, I guess. Read https://bitbucket.org/pypa/pypi/issues/438/backwards-compatible-un-hashed-package for why. Comments from Reviewable |
Reviewed 8 of 9 files at r1, 1 of 1 files at r2. WORKSPACE, line 302 at r2 (raw file):
We could probably live with LGPL 2+ or 3+, but really file-only-viral (MPL2) or non-viral would be best if possible. It would also be a bonus if the tool's documentation explicitly disclaimed copyright on its outputs. tools/pycps.BUILD, line 6 at r2 (raw file):
FYI Does this work instead? alias(
name = "cps2cmake.py",
actual = ":cps2cmake",
) (I assume you already tried Comments from Reviewable |
Add new helper drake_generate_file to generate a file with specified content. Use this instead of a genrule invoking echo to generate a file for LCM.
2997a8f
to
f6b1290
Compare
Review status: 2 of 9 files reviewed at latest revision, 5 unresolved discussions. WORKSPACE, line 298 at r1 (raw file): Previously, jamiesnape (Jamie Snape) wrote…
Done. WORKSPACE, line 302 at r2 (raw file): Previously, jwnimmer-tri (Jeremy Nimmer) wrote…
This shouldn't be a problem, any more than building Drake with GCC is a problem. We are not incorporating any code from PyCPS, and it is widely understood that the output of a GPL tool is not itself subject to GPL. (Also, for the record, the author hereby says as much.) I'm not aware of any precedent for such disclaimers (e.g. this wasn't a problem when drake-config.cmake was generated by CMake). Do you have any recommendations? (All that said, it does make sense on further reflection for PyCPS to be LGPL, but I don't think that should necessarily block this PR.) tools/semantic_version.BUILD, line 3 at r1 (raw file): Previously, jamiesnape (Jamie Snape) wrote…
Done. tools/semantic_version.BUILD, line 11 at r1 (raw file): Previously, jamiesnape (Jamie Snape) wrote…
Done. Comments from Reviewable |
Review status: 2 of 9 files reviewed at latest revision, 5 unresolved discussions. WORKSPACE, line 302 at r2 (raw file):
tools/pycps.BUILD, line 6 at r2 (raw file): Previously, jwnimmer-tri (Jeremy Nimmer) wrote…
Nope:
In my opinion, the "right" thing to do here is to somehow just tell Bazel "hey, this source file is a Python script, it has some (run-time) dependencies but you don't need to do anything special with it". Alas, I don't know how to do that, and @laszlocsomor says this ugly hoop-jumping is the "right" thing to do. (FWIW, I'm more bothered that I can't use the script's real name as the Bazel target name 😢.) Comments from Reviewable |
@drake-jenkins-bot retest this please |
Please update #4045 if/when this merges. |
Reviewed 2 of 7 files at r3, 4 of 4 files at r4, 1 of 1 files at r5. WORKSPACE, line 302 at r2 (raw file): Previously, mwoehlke-kitware (Matthew Woehlke) wrote…
OK I will take a note to think about this further, and allow it for now. I would be slightly happier if Comments from Reviewable |
Review status: 8 of 9 files reviewed at latest revision, 3 unresolved discussions. WORKSPACE, line 302 at r2 (raw file):
I started to do that, but it just rubs me wrong 😄. Conceptually, PyCPS consists of a) the
...I went with this. (It's private anyway, with no expectation that it would ever be used at other than build time in the worst case, but if it helps you feel better, I'm more than happy adding a comment why we want it private.) Comments from Reviewable |
Reviewed 1 of 9 files at r1, 2 of 7 files at r3, 4 of 4 files at r4, 1 of 1 files at r5, 1 of 1 files at r6. Comments from Reviewable |
+(status: curate commits before merging) Reviewed 1 of 1 files at r6. Comments from Reviewable |
Add PyCPS as a new Bazel external, along with its dependency, (the Python module) semantic_version. This will allow us to generate drake-config.cmake from drake.cps as part of the build process, rather than doing it manually. (This is not needed for the CMake build, since that generates drake-config.cmake differently.)
Remove the hand-generated drake-config.cmake and replace with a rule to generate it from drake.cps at build time. Update packaging accordingly.
37956bd
to
34ac102
Compare
WORKSPACE, line 302 at r2 (raw file): Previously, mwoehlke-kitware (Matthew Woehlke) wrote…
The GPL FAQ explicitly states that a GPL program cannot affect the output of a program. Basically what goes in, must come out. Just to be explicit: https://www.gnu.org/licenses/gpl-faq.en.html#WhatCaseIsOutputGPL So it really shouldn't be a problem here given that it's use is to generate drake-config.cmake from the drake repository files. Nonetheless, nice to see it go LGPL and I like the private visibility (learn something new about bazel every day :)). Comments from Reviewable |
Yup. This is good. In my earlier review, I'd only considered the "Everything in WORKSPACE under platform review must pass a licensing cross-check", but in this particular context the rules are different and I hadn't considered the tool output exception. |
@mwoehlke-kitware : re:
After reading this I thought more about the problem and realized, if all you want is to set the runtime dependencies of the python script but otherwise don't care about anything the py_* rules would add (compilation, type checking, whatever they might do, honestly I don't know for sure), then you can just create a Here, "foo" is the python script, "shbin" can be built, "pybin" cannot:
|
@laszlocsomor, thanks for looking again. Alas, while I indeed don't care about compiling or whatnot, I do care about being able to bring in |
You would have to specify it as |
That may or may not be following transitive dependencies, and in any case, it doesn't seem to work with external |
Clean up mechanism for wholesale file generation in Bazel. Add PyCPS and dependency semantic_version. Move generation of
drake-config.cmake
viadrake.cps
from an offline step to a Bazel build rule.Besides that generating files during the build rather than offline is better anyway, this will also help streamline things when stuff in the CPS changes, as it will no longer be necessary to manually regenerate
drake-config.cmake
.This change is