-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support distribution of binaries #159
Comments
I have a possible PR that could be used to start a discussion on this. I also have https://github.com/AidanDelaney/maven-dist which completely factors the distribution logic out of the maven buildpack. In my experiments I have a |
+1 for supporting this use case. I have some thoughts on the implementation. First, typically an env variable for config is used if that option is meant to be a.) user configurable or b.) if a buildpack needs to communicate to a downstream buildpack (i.e. A -> B -> C where A needs to tell C to do something). I'm not sensing either of those scenarios in this use case, although please correct me if that's wrong. To me, it sounds like you have a downstream buildpack that wants to trigger a behavior in an upstream buildpack, which is the classic use case for buildplan manipulation. i.e. A -> B -> C where C wants to change A's behavior. Right now, we have a build plan like this in Maven. Just a single entry, and this is set if a
If there is no I haven't tried this, but I'm thinking that if instead of failing, we returned something like this:
That would mean a downstream buildpack would have to say I require maven or the buildpack would fail. That could be your custom buildpack. The only thing I'm not 100% certain of off the top of my head is if this would break the case where we do not want Maven at all. That would merit some exploration. We could then modify the build logic in the Maven buildpack to look at the buildplan. If a Maven entry is there, then it'll install Maven. This should be true for the original buildplan entry and also this new plan entry, so the buildpack would install Maven. Then if PlanEntryJVMApplicationPackage is in the buildplan, it would go on to do the actual Maven build. You could omit that in your use case and it would just contribute Maven. The benefit here is that we a.) don't have to introduce any new env variables or require a user to set any env variables and b.) we don't need to add any logic to Maven CNB about your use case. That can all be encapsulated in your buildpack. What do you think? Does it sound like this would give you the flexibility you require? |
This sounds like a good approach. It's very similar to an approach were we might provide a |
There isn't a right or wrong answer here. Decomposing to the smallest units possible does make smaller buildpacks, which are individually easier to understand. It can also make them easier to reuse. On the flip side, the composition of different buildpacks can be tricky to understand, you have more complicated order groups and you have to keep how they all work in your head to understand the flow. You also have to maintain a lot more individual buildpack projects (update, release, etc...). Decomposing more also increases the number of layers you'll end up with in your builder images, which is presently a problem for the Paketo builders (each buildpack is a layer). With the Java buildpacks, we've generally taken a moderate, function-based approach to decomposing buildpacks. We're not trying to split them up into the smallest possible units, but rather group them by reasonably reusable functional units. So we have Maven, Gradle, Bellsoft (or another JVM vendor), Tomcat, Executable JAR, etc... It's easy to know what these do just based on the name & the order group is one flat order group regardless of your application type. At any rate, I'd like to stick with that pattern at least for now. |
Thanks, I now understand the context. |
We have need for
mvn
ormvnd
to be distributed on a build layer. This makes a maven binary available for subsequent buildpacks to use. In this scenario, the source project does not necessarily contain apom.xml
.Describe the Enhancement
The enhancement allows the maven buildpack to operate as a distribution buildpack. Subsequent buildpacks can use the maven binary to perform maven operations.
Possible Solution
The maven buildpack could succeed to detect if
BP_MAVEN_COMMAND
is set to eithermvn
ormvnd
. During the build phase, ifBP_MAVEN_COMMAND
is set to eithermvn
ormvnd
then the corresponding binary is contributed as a build layer.A buildpack that requires the maven buildpack might be expected to explicitly set a flag to disable running the maven command. The default operation of the buildpack, in the absence of the flag, would continue to run the build command.
The current cache layer on which
mvn
/mvd
are distributed would be made available as a build and cache layer.Motivation
Our motivation is to support injection of Jars into an application image in a non-JVM source project. For example, python is a common language for data science projects and data science projects in python often use PySpark for distributed computation. Such use-cases commonly need to install a single Jar file, or a small number of Jar files, and make them available in the CLASSPATH. The set of Jars to install depend on the dependencies used in the Python project. Application developers do not want to manage both a python build system and a maven/gradle/sbt build system. Supporting a "distribution only" operation in the maven buildpack allows us to provide a subsequent buildpack to detect a PySpark project and inject appropriate Jar files.
Given this motivation, a data-science oriented build image might set
BP_MAVEN_COMMAND=mvn
by default. The buildpack that contributes the Jars would depend on the maven buildpack and setRunBuild: false
. This moves the problem of managing dependencies in apom.xml
from the data-scientist to the platform operator.The text was updated successfully, but these errors were encountered: