Bintray publish settings #501

jamescway · 2015-07-24T06:15:43Z

Problem

We need to release artifacts built in travis to Bintray openzipkin/zipkin repo.

Solution

How it works

user creates a tag git tag v1.2.3
then does git push origin master --tags
triggers the publish script to be called after travis build is successful
script creates a bintray credentials file in travis...
then using bintray-sbt pushes artifacts to packages/modules in bintray (i.e. openzipkin/zipkin/zipkin-collector-core)
- based on encrypted values in travis.yml
- gem install travis
- travis encrypt BINTRAY_API_KEY=<Bintray API key>--add
- travis encrypt BINTRAY_USER="<Bintray username" --add

Caveats

not detecting if the tagged branch is master due to bug in travis (on: tags condition for deploys still not working travis-ci/travis-ci#1675)
still running tests when tag is pushed, needs workaround
modules/packages in Openzipkin repo need to be created manually (i.e. zipkin-collector-core) due to {"message":"Package 'dadagen' was not found"} sbt/sbt-bintray#58
current version number in Project.scala
building from oraclejdk7

codefromthecrypt · 2015-07-24T15:13:05Z

.travis.yml

@@ -1,35 +1,31 @@
 language: scala


supernit: can you get rid of the fuzz here? looks like the only needed one is the last few lines.

codefromthecrypt · 2015-07-24T15:24:04Z

Thanks for giving this a go, @jamescway! I think once you rebase against upstream master, this gets smaller. I've made some notes, mostly on cleanup.

jamescway · 2015-07-27T21:19:20Z

having some problems after rebase...
traced it down to this line
addSbtPlugin("com.twitter" %% "scrooge-sbt-plugin" % "3.19.0")
previously it was
addSbtPlugin("com.twitter" %% "scrooge-sbt-plugin" % "3.16.3")

With the newer lib sbt publish now causes:
[error] (zipkin-collector-core/*:publish) java.lang.NoSuchMethodError: org.jboss.netty.handler.codec.http.HttpRequest.setHeader(Ljava/lang/String;Ljava/lang/Object;)V

here's the full stack trace

> last zipkin-collector-core/*:publish
java.lang.NoSuchMethodError: org.jboss.netty.handler.codec.http.HttpRequest.setHeader(Ljava/lang/String;Ljava/lang/Object;)V
    at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.construct(NettyAsyncHttpProvider.java:693)
    at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.buildRequest(NettyAsyncHttpProvider.java:650)
    at com.ning.http.client.providers.netty.NettyConnectListener$Builder.build(NettyConnectListener.java:144)
    at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.doConnect(NettyAsyncHttpProvider.java:1070)
    at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.execute(NettyAsyncHttpProvider.java:935)
    at com.ning.http.client.AsyncHttpClient.executeRequest(AsyncHttpClient.java:499)
    at dispatch.HttpExecutor$class.apply(execution.scala:47)
    at dispatch.Http.apply(execution.scala:12)
    at dispatch.HttpExecutor$class.apply(execution.scala:42)
    at dispatch.Http.apply(execution.scala:12)
    at bintry.Requests.request(Client.scala:42)
    at bintry.Methods$Repo$Package$Version$Upload.apply(Methods.scala:177)
    at bintray.BintrayIvyRepository.put(Resolver.scala:54)
    at org.apache.ivy.plugins.resolver.RepositoryResolver.put(RepositoryResolver.java:234)
    at org.apache.ivy.plugins.resolver.RepositoryResolver.publish(RepositoryResolver.java:216)
    at sbt.IvyActions$$anonfun$publish$3.apply(IvyActions.scala:336)
    at sbt.IvyActions$$anonfun$publish$3.apply(IvyActions.scala:335)
    at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772)
    at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
    at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
    at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:771)
    at sbt.IvyActions$.publish(IvyActions.scala:335)
    at sbt.IvyActions$$anonfun$publish$1$$anonfun$apply$1.apply$mcV$sp(IvyActions.scala:121)
    at sbt.IvyActions$$anonfun$publish$1$$anonfun$apply$1.apply(IvyActions.scala:121)
    at sbt.IvyActions$$anonfun$publish$1$$anonfun$apply$1.apply(IvyActions.scala:121)
    at sbt.IvyActions$.withChecksums(IvyActions.scala:130)
    at sbt.IvyActions$.sbt$IvyActions$$withChecksums(IvyActions.scala:125)
    at sbt.IvyActions$$anonfun$publish$1.apply(IvyActions.scala:121)
    at sbt.IvyActions$$anonfun$publish$1.apply(IvyActions.scala:114)
    at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:155)
    at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:155)
    at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:132)
    at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:57)
    at sbt.IvySbt$$anon$4.call(Ivy.scala:65)
    at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93)
    at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78)
    at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97)
    at xsbt.boot.Using$.withResource(Using.scala:10)
    at xsbt.boot.Using$.apply(Using.scala:9)
    at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58)
    at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48)
    at xsbt.boot.Locks$.apply0(Locks.scala:31)
    at xsbt.boot.Locks$.apply(Locks.scala:28)
    at sbt.IvySbt.withDefaultLogger(Ivy.scala:65)
    at sbt.IvySbt.withIvy(Ivy.scala:127)
    at sbt.IvySbt.withIvy(Ivy.scala:124)
    at sbt.IvySbt$Module.withModule(Ivy.scala:155)
    at sbt.IvyActions$.publish(IvyActions.scala:114)
    at sbt.Classpaths$$anonfun$publishTask$1.apply(Defaults.scala:1283)
    at sbt.Classpaths$$anonfun$publishTask$1.apply(Defaults.scala:1282)
    at scala.Function3$$anonfun$tupled$1.apply(Function3.scala:35)
    at scala.Function3$$anonfun$tupled$1.apply(Function3.scala:34)
    at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
    at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
    at sbt.std.Transform$$anon$4.work(System.scala:63)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
    at sbt.Execute.work(Execute.scala:235)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
    at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

Seems like maybe the lib bintry is using an old version of dispatch maybe?

jamescway · 2015-07-27T23:01:18Z

looks like we have netty-3.10.1.Final.jar and async-http-client-1.8.10.jar both loading netty

eirslett · 2015-07-28T06:31:49Z

Where does async-http-client come from? Should be easy to replace with finagle-http if we have control over the source.

eirslett · 2015-07-28T12:14:58Z

Aah, I think I can spot the problem - publish to bintray uses Dispatch, while Scrooge uses Finagle (with an older version of Netty, probably)
I thought sbt used separate class loaders for each plugin, maybe not...

codefromthecrypt · 2015-07-28T14:53:22Z

Could an exclude on scrooge work? I doubt the compiler actually needs to load the netty classes. I can give this a try in 30m or so, if no one else does.

jamescway · 2015-07-28T16:11:46Z

@eirslett yeah, dispatch uses it, I have a PR merged into bintry going to find out if he will release it. Actually scrooge has the newer version of netty, bintry (dep of bintray-sbt has an older version.

@adriancole rolling back and using scrooge-sbt-plugin" % "3.16.3" would solve the problem in the meantime

What do u guys think? Going to roll it back to 3.16.3 feel free to thumbs down :p

codefromthecrypt · 2015-07-28T16:46:10Z

@adriancole rolling back and using scrooge-sbt-plugin" % "3.16.3" would
solve the problem in the meantime
I'd be fine with adding a commit to use this version with a comment
about why including the link to the fix. However, it looks like the
commit didn't change any source files. Another way could be to make a
commit that overrides the version of dispatch-json4s-native. This
helps as we already need to update scrooge, and mixed classpath seems
worse. wdyt?

rtyler · 2015-07-29T06:41:28Z

This probably isn't going to work now that #503 has been merged <_<

abesto · 2015-07-29T08:24:08Z

OTOH Bintray publishes an official Gradle plugin: https://bintray.com/jfrog/jfrog-jars/gradle-bintray-plugin

jamescway · 2015-07-29T16:56:13Z

After update to use gradle #503, much of this stuff is not relevant, going to close and start another PR

jamescway · 2015-07-29T17:03:48Z

@abesto just noticed that tool, I'll give it a shot

codefromthecrypt reviewed Jul 24, 2015
View reviewed changes

add bintray publish settings

49ad8a1

jamescway closed this Jul 29, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bintray publish settings #501

Bintray publish settings #501

jamescway commented Jul 24, 2015

codefromthecrypt Jul 24, 2015

codefromthecrypt commented Jul 24, 2015

jamescway commented Jul 27, 2015

jamescway commented Jul 27, 2015

eirslett commented Jul 28, 2015

eirslett commented Jul 28, 2015

codefromthecrypt commented Jul 28, 2015 via email

jamescway commented Jul 28, 2015

codefromthecrypt commented Jul 28, 2015

rtyler commented Jul 29, 2015

abesto commented Jul 29, 2015

jamescway commented Jul 29, 2015

jamescway commented Jul 29, 2015

Bintray publish settings #501

Bintray publish settings #501

Conversation

jamescway commented Jul 24, 2015

Problem

Solution

How it works

Caveats

codefromthecrypt Jul 24, 2015

Choose a reason for hiding this comment

codefromthecrypt commented Jul 24, 2015

jamescway commented Jul 27, 2015

jamescway commented Jul 27, 2015

eirslett commented Jul 28, 2015

eirslett commented Jul 28, 2015

codefromthecrypt commented Jul 28, 2015 via email

jamescway commented Jul 28, 2015

codefromthecrypt commented Jul 28, 2015

rtyler commented Jul 29, 2015

abesto commented Jul 29, 2015

jamescway commented Jul 29, 2015

jamescway commented Jul 29, 2015