Expose syncing with sonatype staging repo without maven central release

[lukaszwawrzyk]

bintraySyncSonatypeStaging

I added it as a new task. It does the same as bintraySyncMavenCentral, except that it passes close flag set to false, so that artifacts are placed in sonatype staging repo, but not yet released.

Bintry version bump

This is required for PR to actually work, because the new version fixes bug with closing. It seems like 0.5.2 didn't get released to maven central, so this PR will have to wait until fixed.

Retries

Genesis of this change can be found in this travis build.
Log line 1546 says:

[error] java.lang.RuntimeException: failed to sync slamdata-inc/[email protected] with maven central: {"status":"Validation Failed","messages":"[Missing Signature: '/org/quasar-analytics/quasar-common-internal_2.12/33.0.0/quasar-common-internal_2.12-33.0.0-sources.jar.asc' does not exist for 'quasar-common-internal_2.12-33.0.0-sources.jar'., Dropping existing partial staging repository.]"}

So it says, that file is missing, but it shouldn't be, because line 1249 says that it was published:

[info] 	published quasar-common-internal_2.12 to https://api.bintray.com/maven/slamdata-inc/maven-public/maven-public/org/quasar-analytics/quasar-common-internal_2.12/33.0.0/quasar-common-internal_2.12-33.0.0-sources.jar.asc

This is probably due to replicas are not yet consistent at that point of time. It can be observed that soon after publishing a version, some artifacts are visible, and some are not, or version doesn't exist at all, depending on when you refresh the page.
For this reason, to limit number of failed builds, I implemented retry mechanism. It retries according to bintraySyncMavenCentralRetries setting. If the seq is empty, it behaves just like before, and this is the default I set, to not alter behavior.
One can set it for example to:

bintraySyncMavenCentralRetries := {
  import scala.concurrent.duration._
  Seq(5.seconds, 1.minute, 5.minutes)
}

In case of exception from mavenSync task, (i.e. status other than 200 or 404) it will wait 5s ant try again, if it fails, it will retry after a minute, and finally after 5 minutes. If it still fails after 5, the exception is rethrown. If at any point it succeeds, result is returned as if nothing bad happened.

If this change is too custom or hacky, let me know, I will just remove the commit. It can be implemented in client project of this plugin in a similar way. Although I think it could be helpful if someone faces similar problem.

[2m]

Is the bintraySyncSonatypeStaging same as process described in https://github.com/sbt/sbt-bintray#staging-optional ?

[lukaszwawrzyk]

@2m I think not. The process that you linked stages artifacts in bintray, so they won't be available through your bintray repo, or jCenter if that is linked. At this point there is no mention of sonatype. Then it can be released to bintray with bintrayRelease. Artifacts appear in your repo and jCenter if linked. At this point we can start talking about this PR, because only once version is available in jCenter it can be synchronized with maven central through sonatype. This is the action described here: https://bintray.com/docs/api/#_sync_version_artifacts_to_maven_central Currently, the plugin always sends this request with close set to "1". This means that bintray will sync artifacts with nexus staging repository and close it causing release to maven central.
This PR, through bintraySyncSonatypeStaging task enables possibility to put your artifacts to sonatype staging repo but without closing and hence without putting them to maven central.

For a backstory, the ultimate goal is to, in a multi module/package project, on CI, run bintraySyncSonatypeStaging for all modules, and only once that succeeds, so all artifacts reach nexus and are valid, we can run bintraySyncMavenCentral for each module to make them public in maven central. This is to avoid the situation when synchronizing of 10 packages works and for 10 packages it fails which puts incomplete product to maven central publicly.

[djspiewak]

Bumping this for attention. (sorry! it's blocking our release process right now) @jvican @eed3si9n?

[jvican]

Won't have time to have a look at this any time soon, sorry. I defer my review to @eed3si9n or @dwijnand.

[djspiewak]

This PR is now published on my personal bintray, so we'll see how well it works. Hopefully this also alleviates the need for me to bug you guys over and over again; I know y'all are busy. 🙂

resolvers += Resolver.bintrayIvyRepo("djspiewak", "ivy")
addSbtPlugin("org.foundweekends" % "sbt-bintray" % "0.5.3+4-b634b339")

[2m]

Change LGTM. Just one thing, in the motivation you write:

For a backstory, the ultimate goal is to, in a multi module/package project, on CI, run bintraySyncSonatypeStaging for all modules, and only once that succeeds, so all artifacts reach nexus and are valid, we can run bintraySyncMavenCentral for each module to make them public in maven central.

I am not sure if that is going to work. Running bintraySyncMavenCentral again after bintraySyncSonatypeStaging will probably create another staging repo in sonatype and close the new one. Leaving the staging repo open. I am not sure if bintray/sonatype has this documented somewhere, but it needs to be checked out.

[lukaszwawrzyk]

@2m This is still not verified due to some pretty crazy problem with bintray, but completely unrelated to to this. But regardless, retry mechanism seems to work and is useful.
Task to sync with staging repo without closing is possible thorugh bintray api, but not from the plugin, so I think it is a good addition as well. Even if rerunning with close trick wouldn't work it maybe can be worked around using sonatype plugin to close everything.

So, basically if you agree, could we merge and release this?

[2m]

LGTM