-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[fix][build] Override inherrited build time to now #17754
Conversation
/pulsarbot rerun-failure-checks |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
/pulsarbot rerun-failure-checks |
1 similar comment
/pulsarbot rerun-failure-checks |
Codecov Report
@@ Coverage Diff @@
## master #17754 +/- ##
=============================================
+ Coverage 34.91% 47.62% +12.70%
- Complexity 5707 19767 +14060
=============================================
Files 607 1594 +987
Lines 53396 143144 +89748
Branches 5712 17527 +11815
=============================================
+ Hits 18644 68166 +49522
- Misses 32119 68388 +36269
- Partials 2633 6590 +3957
Flags with carried forward coverage won't be shown. Click here to find out more.
|
9a37435
to
49c5f00
Compare
Rebased to see if that helps with flaky tests. |
/pulsarbot rerun-failure-checks |
748527f
to
a9d33b2
Compare
CI is really flaky right now. This change is trivial, and I still haven't gotten tests to pass once. |
a9d33b2
to
0e4eb7d
Compare
The sql step is failing with the following error:
@lhotari - have you seen this exit status 4 before? It seems like it would be a flaky issue, but I seem to be consistently failing at the sql job. |
It seems that SQL system tests should be completed normally in 15 minutes. But...how can this patch make those tests so costly? |
It seems we're now always fail to star the container: Logs:2022-12-06T14:41:44.7029477Z 2022-12-06T14:41:44,702 - INFO - [docker-java-stream--1991487674:DockerUtils$4@383] - DOCKER.exec(pulsar-sql-test-pulsar-broker-0:tail -f /var/log/pulsar/broker.log): STDOUT: 2022-12-06T14:41:43,954+0000 [pulsar-load-manager-1-1] INFO org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Writing local data to metadata store because maximum change 14.196531474590302% exceeded threshold 10%; time since last report written is 55.088 seconds 2022-12-06T14:41:44.7031040Z 2022-12-06T14:41:43,984+0000 [main-EventThread] INFO org.apache.pulsar.metadata.coordination.impl.ResourceLockImpl - Acquired resource lock on /loadbalance/brokers/pulsar-broker-0:8080 2022-12-06T14:41:49.7035404Z 2022-12-06T14:41:49,702 - INFO - [docker-java-stream--1991487674:DockerUtils$4@383] - DOCKER.exec(pulsar-sql-test-pulsar-broker-0:tail -f /var/log/pulsar/broker.log): STDOUT: 2022-12-06T14:41:48,736+0000 [pulsar-load-manager-1-1] INFO org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Only 1 broker available: no load shedding will be performed 2022-12-06T14:42:44.7095422Z 2022-12-06T14:42:44,708 - INFO - [docker-java-stream--1991487674:DockerUtils$4@383] - DOCKER.exec(pulsar-sql-test-pulsar-broker-0:tail -f /var/log/pulsar/broker.log): STDOUT: 2022-12-06T14:42:43,954+0000 [pulsar-load-manager-1-1] INFO org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Writing local data to metadata store because maximum change 11.071531474590302% exceeded threshold 10%; time since last report written is 59.999 seconds 2022-12-06T14:42:44.7097363Z 2022-12-06T14:42:43,964+0000 [main-EventThread] INFO org.apache.pulsar.metadata.coordination.impl.ResourceLockImpl - Acquired resource lock on /loadbalance/brokers/pulsar-broker-0:8080 2022-12-06T14:42:49.7099579Z 2022-12-06T14:42:49,709 - INFO - [docker-java-stream--1991487674:DockerUtils$4@383] - DOCKER.exec(pulsar-sql-test-pulsar-broker-0:tail -f /var/log/pulsar/broker.log): STDOUT: 2022-12-06T14:42:48,736+0000 [pulsar-load-manager-1-1] INFO org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Only 1 broker available: no load shedding will be performed 2022-12-06T14:43:49.7173597Z 2022-12-06T14:43:49,716 - INFO - [docker-java-stream--1991487674:DockerUtils$4@383] - DOCKER.exec(pulsar-sql-test-pulsar-broker-0:tail -f /var/log/pulsar/broker.log): STDOUT: 2022-12-06T14:43:48,737+0000 [pulsar-load-manager-1-1] INFO org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Only 1 broker available: no load shedding will be performed 2022-12-06T14:44:49.7234512Z 2022-12-06T14:44:49,722 - INFO - [docker-java-stream--1991487674:DockerUtils$4@383] - DOCKER.exec(pulsar-sql-test-pulsar-broker-0:tail -f /var/log/pulsar/broker.log): STDOUT: 2022-12-06T14:44:48,737+0000 [pulsar-load-manager-1-1] INFO org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Only 1 broker available: no load shedding will be performed 2022-12-06T14:45:49.7295766Z 2022-12-06T14:45:49,728 - INFO - [docker-java-stream--1991487674:DockerUtils$4@383] - DOCKER.exec(pulsar-sql-test-pulsar-broker-0:tail -f /var/log/pulsar/broker.log): STDOUT: 2022-12-06T14:45:48,738+0000 [pulsar-load-manager-1-1] INFO org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Only 1 broker available: no load shedding will be performed 2022-12-06T14:45:55.2496634Z 2022-12-06T14:45:55,248 - ERROR - [main:GenericContainer@529] - Could not start container 2022-12-06T14:45:55.2497644Z org.testcontainers.containers.ContainerLaunchException: Timed out waiting for URL to be accessible (http://localhost:49179/v1/info/state should return HTTP [200]) 2022-12-06T14:45:55.2593061Z at org.testcontainers.containers.wait.strategy.HttpWaitStrategy.waitUntilReady(HttpWaitStrategy.java:318) ~[testcontainers-1.17.6.jar:1.17.6] 2022-12-06T14:45:55.2596482Z at org.testcontainers.containers.wait.strategy.AbstractWaitStrategy.waitUntilReady(AbstractWaitStrategy.java:52) ~[testcontainers-1.17.6.jar:1.17.6] 2022-12-06T14:45:55.2597468Z at org.testcontainers.containers.GenericContainer.waitUntilContainerStarted(GenericContainer.java:953) ~[testcontainers-1.17.6.jar:1.17.6] 2022-12-06T14:45:55.2598324Z at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:485) ~[testcontainers-1.17.6.jar:1.17.6] 2022-12-06T14:45:55.2599115Z at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:344) ~[testcontainers-1.17.6.jar:1.17.6] 2022-12-06T14:45:55.2599886Z at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81) ~[duct-tape-1.0.8.jar:?] 2022-12-06T14:45:55.2600660Z at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:334) ~[testcontainers-1.17.6.jar:1.17.6] 2022-12-06T14:45:55.2601421Z at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:322) ~[testcontainers-1.17.6.jar:1.17.6] 2022-12-06T14:45:55.2602196Z at org.apache.pulsar.tests.integration.containers.PulsarContainer.start(PulsarContainer.java:209) ~[test-classes/:?] 2022-12-06T14:45:55.2603059Z at org.apache.pulsar.tests.integration.topologies.PulsarCluster.startPrestoWorker(PulsarCluster.java:420) ~[test-classes/:?] 2022-12-06T14:45:55.2603949Z at org.apache.pulsar.tests.integration.topologies.PulsarCluster.startPrestoWorker(PulsarCluster.java:411) ~[test-classes/:?] 2022-12-06T14:45:55.2604813Z at org.apache.pulsar.tests.integration.presto.TestBasicPresto.setupPresto(TestBasicPresto.java:58) ~[test-classes/:?] 2022-12-06T14:45:55.2605645Z at org.apache.pulsar.tests.integration.presto.TestBasicPresto.setupCluster(TestBasicPresto.java:70) ~[test-classes/:?] 2022-12-06T14:45:55.2606530Z at org.apache.pulsar.tests.integration.topologies.PulsarClusterTestBase.setup(PulsarClusterTestBase.java:42) ~[test-classes/:?] 2022-12-06T14:45:55.2607633Z at org.apache.pulsar.tests.TestRetrySupport.stateCheck(TestRetrySupport.java:60) ~[buildtools-2.11.0-SNAPSHOT.jar:2.11.0-SNAPSHOT] 2022-12-06T14:45:55.2608225Z at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] 2022-12-06T14:45:55.2608796Z at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?] 2022-12-06T14:45:55.2609398Z at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] 2022-12-06T14:45:55.2609897Z at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?] 2022-12-06T14:45:55.2610582Z at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2611514Z at org.testng.internal.invokers.MethodInvocationHelper.invokeMethodConsideringTimeout(MethodInvocationHelper.java:69) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2612540Z at org.testng.internal.invokers.ConfigInvoker.invokeConfigurationMethod(ConfigInvoker.java:361) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2613464Z at org.testng.internal.invokers.ConfigInvoker.invokeConfigurations(ConfigInvoker.java:296) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2614202Z at org.testng.internal.invokers.TestInvoker.runConfigMethods(TestInvoker.java:816) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2614874Z at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:590) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2615566Z at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:221) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2616265Z at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2616970Z at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:962) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2617679Z at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:194) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2618434Z at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:148) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2619147Z at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2619608Z at java.util.ArrayList.forEach(ArrayList.java:1511) ~[?:?] 2022-12-06T14:45:55.2620090Z at org.testng.TestRunner.privateRun(TestRunner.java:806) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2620596Z at org.testng.TestRunner.run(TestRunner.java:601) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2621100Z at org.testng.SuiteRunner.runTest(SuiteRunner.java:433) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2621655Z at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:427) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2622209Z at org.testng.SuiteRunner.privateRun(SuiteRunner.java:387) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2622719Z at org.testng.SuiteRunner.run(SuiteRunner.java:330) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2623255Z at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2623831Z at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2624394Z at org.testng.TestNG.runSuitesSequentially(TestNG.java:1256) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2624930Z at org.testng.TestNG.runSuitesLocally(TestNG.java:1176) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2625427Z at org.testng.TestNG.runSuites(TestNG.java:1099) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2625999Z at org.testng.TestNG.run(TestNG.java:1067) ~[testng-7.6.1.jar:7.6.1] 2022-12-06T14:45:55.2626634Z at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:283) ~[surefire-testng-3.0.0-M3.jar:3.0.0-M3] 2022-12-06T14:45:55.2627586Z at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:75) ~[surefire-testng-3.0.0-M3.jar:3.0.0-M3] 2022-12-06T14:45:55.2628404Z at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:120) ~[surefire-testng-3.0.0-M3.jar:3.0.0-M3] 2022-12-06T14:45:55.2629289Z at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) ~[surefire-booter-3.0.0-M3.jar:3.0.0-M3] 2022-12-06T14:45:55.2630190Z at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) ~[surefire-booter-3.0.0-M3.jar:3.0.0-M3] 2022-12-06T14:45:55.2630980Z at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) ~[surefire-booter-3.0.0-M3.jar:3.0.0-M3] 2022-12-06T14:45:55.2631718Z at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418) ~[surefire-booter-3.0.0-M3.jar:3.0.0-M3] 2022-12-06T14:45:55.2813355Z 2022-12-06T14:45:55,280 - ERROR - [main:GenericContainer@536] - Log output from the failed container: |
@michaeljmarshall I think this patch is a fix to the current failure and works well tisonkun#21. You can make use of it :) |
Superseded by #18820 |
Motivation
While working on #17752, I noticed our build time was incorrect. It was set to
2020-01-22T09:10:15-0600
. This timestamp was inherited from the org.apache:apache:23 dependency, as you can see here: https://repo1.maven.org/maven2/org/apache/apache/23/apache-23.pom. Maven usesproject.build.outputTimestamp
for reproducible builds, as documented here: https://maven.apache.org/guides/mini/guide-reproducible-builds.html.The
git.build.time
we use in thePulsarVersion
comes from theproject.build.outputTimestamp
(when set), as documented here: https://github.com/git-commit-id/git-commit-id-maven-plugin/releases/tag/v4.0.2. When the property is null, the plugin usesnew Date()
to get the build time.https://github.com/git-commit-id/git-commit-id-maven-plugin/blob/80c6740ee268e1a524b2d0b1233801f2ac83881b/core/src/main/java/pl/project13/core/cibuild/BuildServerDataProvider.java#L138
If or when we decide to do reproducible builds, we will need to set this property anyway, so this solution does not prevent us from reproducible builds.
Modifications
project.build.outputTimestamp
to an empty value to get more accurate build timestamp.Verifying this change
I manually verified that this change enables
PulsarVersion.java
to generate correctly.Documentation
doc-not-needed
Matching PR in forked repository
PR in forked repository: michaeljmarshall#4