Publish to clojars (or whatever) so this can be used directly from clojure

[kerinin]

I have a clojure project that is about to include some EMR code. I already have a lein project, so it would be nice if I could just add a dependency on lemur and then build my own tooling around it.

[amontalenti]

+1 to this! Any plans?

[mlimotte]

Hi Andrew, Ryan,

Can you be more specific about which aspects of lemur you want to use as a
library? Which functions or namespaces? Is it just the stuff in
"com.climate.services.aws.emr"?

Just deploying something to clojars would be easy, but the problem is that
lemur.core manages a bunch of state in a global atom (not a great practice,
but I didn't know any better Clojure patterns at the time). It's not
really a problem for command line, but if you start to use it as a lib, I'm
afraid there will be conflicts when multiple jobs are submitted.

I don't think the changes would be huge, but some refactoring of state
management would need to be done in lemur.core in order to use it as a lib.
But see my first question... if all you want is the emr namespace, that
alone would be easier to export into it's own jar.

marc

On Thu, Jan 30, 2014 at 9:05 PM, Andrew Montalenti <[email protected]

wrote:

+1 to this! Any plans?

Reply to this email directly or view it on GitHubhttps://github.com//issues/20#issuecomment-33755997
.

[amontalenti]

The reason I'm requesting this is because I'd like to use lemur from an existing Clojure project (that happens to be using Apache Storm). Personally, my initial use case is being able to describe job flows, wire it up with Pig scripts, and run them from a Clojure REPL.

Of course, you might be pointing to a broader problem: perhaps you should open a ticket yourself about converting lemur from a "CLI-only" application to a "library with CLI wrapper", and describe what needs to change for this to happen? Someone in the community may be able to pick up the work.

[mlimotte]

That's a good call, Andrew. I don't think the work is large (roughly
create a context object and change functions use the global context to
accept an extra arg instead); unfortunately I won't have time to do this
myself -- at least not right now.

marc

On Fri, Jan 31, 2014 at 10:13 AM, Andrew Montalenti <
[email protected]> wrote:

The reason I'm requesting this is because I'd like to use lemur from an
existing Clojure project (that happens to be using Apache Storm).
Personally, my initial use case is being able to describe job flows, wire
it up with Pig scripts, and run them from a Clojure REPL.

Of course, you might be pointing to a broader problem: perhaps you should
open a ticket yourself about converting lemur from a "CLI-only" application
to a "library with CLI wrapper", and describe what needs to change for this
to happen? Someone in the community may be able to pick up the work.

Reply to this email directly or view it on GitHubhttps://github.com//issues/20#issuecomment-33801779
.

[mlimotte]

Issue created: #23

On Sun, Feb 2, 2014 at 9:33 AM, Marc Limotte [email protected] wrote:

That's a good call, Andrew. I don't think the work is large (roughly
create a context object and change functions use the global context to
accept an extra arg instead); unfortunately I won't have time to do this
myself -- at least not right now.

marc

On Fri, Jan 31, 2014 at 10:13 AM, Andrew Montalenti <
[email protected]> wrote:

The reason I'm requesting this is because I'd like to use lemur from an
existing Clojure project (that happens to be using Apache Storm).
Personally, my initial use case is being able to describe job flows, wire
it up with Pig scripts, and run them from a Clojure REPL.

Of course, you might be pointing to a broader problem: perhaps you should
open a ticket yourself about converting lemur from a "CLI-only" application
to a "library with CLI wrapper", and describe what needs to change for this
to happen? Someone in the community may be able to pick up the work.

Reply to this email directly or view it on GitHubhttps://github.com//issues/20#issuecomment-33801779
.