Plugin causing crash and stopping pushing to bitbucket after upgrading.

[DomhnaillByrne]

When trying to configure plugin after upgrade from gitstash 3.8.0 to bitbucket 5.0.5 it's causing it to crash.

2017-10-05 18:54:31,845 ERROR [https-jsse-nio-8443-exec-52] 56602 @9MA5APx1133x241598x38 9e70mb 10.8.0.70 "POST /rest/prnfb-admin/1.0/settings HTTP/1.1" o.s.t.i.TransactionInterceptor Application exception overridden by rollback exception
java.lang.StackOverflowError: null

[DomhnaillByrne]

Bitbucket version is 5.0.5 and plugin version is 3.14

I've created this ticket here: https://getsupport.atlassian.com/servicedesk/customer/portal/24/SSP-25324 if you'd like me to add you.

[tomasbjerre]

You should be able to find a longer stack trace in the server log. There is probably a very long stack trace with repeating methods.

[DomhnaillByrne]

2017-10-05 09:48:32,384 ERROR [https-jsse-nio-8443-exec-6] 46858 @47LJ2Zx588x460x13 7ra829 10.8.0.70 "GET /rest/prnfb-admin/1.0/settings/buttons/repository/1271/pullrequest/156 HTTP/1.1" o.s.t.i.TransactionInterceptor Application exception overridden by rollback exception
java.lang.StackOverflowError: null
at oracle.jdbc.driver.T4CTTIMsg.marshalTTCcode(T4CTTIMsg.java:284)
at oracle.jdbc.driver.T4CTTIrxd.marshal(T4CTTIrxd.java:205)
at oracle.jdbc.driver.T4C8Oall.marshalBinds(T4C8Oall.java:1860)
at oracle.jdbc.driver.T4C8Oall.marshal(T4C8Oall.java:1176)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:247)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:566)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:215)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:58)
at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:776)
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:897)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1034)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3820)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3867)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1502)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:71)
at org.hibernate.persister.entity.AbstractEntityPersister.loadEntityIdByNaturalId(AbstractEntityPersister.java:5078)
at org.hibernate.event.internal.DefaultResolveNaturalIdEventListener.loadFromDatasource(DefaultResolveNaturalIdEventListener.java:116)
at org.hibernate.event.internal.DefaultResolveNaturalIdEventListener.resolveNaturalId(DefaultResolveNaturalIdEventListener.java:83)
at org.hibernate.event.internal.DefaultResolveNaturalIdEventListener.onResolveNaturalId(DefaultResolveNaturalIdEventListener.java:40)
at org.hibernate.internal.SessionImpl.fireResolveNaturalId(SessionImpl.java:1246)
at org.hibernate.internal.SessionImpl.access$2000(SessionImpl.java:203)
at org.hibernate.internal.SessionImpl$BaseNaturalIdLoadAccessImpl.resolveNaturalId(SessionImpl.java:2978)
at org.hibernate.internal.SessionImpl$NaturalIdLoadAccessImpl.load(SessionImpl.java:3094)
at com.atlassian.stash.internal.plugin.HibernatePluginSettingDao.getByKey(HibernatePluginSettingDao.java:103)
at com.atlassian.stash.internal.plugin.HibernatePluginSettingDao.get(HibernatePluginSettingDao.java:80)
at com.atlassian.stash.internal.plugin.DefaultPluginSettings.lambda$getActual$0(DefaultPluginSettings.java:41)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133)
at com.atlassian.stash.internal.plugin.DefaultPluginSettings.getActual(DefaultPluginSettings.java:41)
at com.atlassian.sal.core.pluginsettings.AbstractStringPluginSettings.get(AbstractStringPluginSettings.java:113)
at com.atlassian.stash.internal.plugin.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:24)
at se.bjurr.prnfb.settings.legacy.SettingsStorage.getSettingsAsFormValues(SettingsStorage.java:264)
at se.bjurr.prnfb.settings.legacy.SettingsStorage.getPrnfbSettings(SettingsStorage.java:226)
at se.bjurr.prnfb.service.SettingsService.doGetPrnfbSettings(SettingsService.java:322)
at se.bjurr.prnfb.service.SettingsService.doSetPrnfbSettings(SettingsService.java:343)
at se.bjurr.prnfb.service.SettingsService.doGetPrnfbSettings(SettingsService.java:324)

[tomasbjerre]

If that is the only stack trace available, then the stack overflow is not in the plugin.

Looking at the statistics I see 126 installations using Stash 3.x and 1624 using Bitbucket Server 4.x or 5.x. I might just remove the legacy code here and stop trying to migrate the old settings from 3.x. I will give it some thought and perhaps do that tomorrow.

A quick fix might be to just remove the settings directly from the database. You can find the key(s) with:

SELECT * FROM PLUGIN_SETTING WHERE KEY_NAME LIKE '%se.bjurr%';

And the settings are stored in:

SELECT KEY_VALUE FROM PLUGIN_SETTING WHERE KEY_NAME='the key'

I think you can just delete that row to reset the plugin settings.

[DomhnaillByrne]

We've since uninstalled the plugin. Will the settings still be in the db?

This is our production instance so I'll have to get the DBA to run that query.
Can I add you to the ticket in atlassian?

[DomhnaillByrne]

The behaviour before the crash was that no push operations were able to happen and there were hundreds of git processes hung on the server.

The plugin is integral to how we work so we want to find a fix soon

[tomasbjerre]

I get "You do not have permission to view this request." when I try to view the ticket.

[tomasbjerre]

Here is a build wihtout the legacy code:
https://github.com/tomasbjerre/pull-request-notifier-for-bitbucket/releases/download/untagged-c76dad310df0823834ac/pull-request-notifier-for-bitbucket-3.15-SNAPSHOT.jar

You may try installing that one by manual upload to Bitbucket Server.

[DomhnaillByrne]

Thanks for the speedy responses! I can test that tomorrow but will it still require the settings to be removed?

There's more info in the atlassian ticket. If you log in I can get you added to see logs etc. as I can't seem to upload them here.

Atlassian have gotten back with the below:

I see you have already pulled the relevant stacktrace from the logs and you are already in contact with Tomas in this bug report: #251.
If Tomas is able to login at getsupport.atlassian.com I can add his user to this ticket at your request.

[tomasbjerre]

I dont think you need to remove the settings if you use that build.

Removing the settings is also another possible solution if the new build does not work.

[DomhnaillByrne]

Hi Tomas,

That link to download the new build doesn't seem to be working. Would you be able to re-upload?

Thanks,
Domhnaill

[tomasbjerre]

Try this: https://drive.google.com/file/d/0Bw-W9c28iP80Tjg1bk1ZOEJqZG8/view?usp=sharing

[DomhnaillByrne]

Initial testing looks good, I'll keep an eye on it and let you know.

[DomhnaillByrne]

After a day of testing, that looks to have resolved the problem.
As this is a snapshot, will the 3.15 release ignore legacy settings going forward?

[tomasbjerre]

Released in 3.15 now.