Reactivate tests: disabled tests collection #525

HeikoKlare · 2023-06-28T15:52:02Z

Current Situation

Several tests have been deactivated over the years for different reasons, such as temporary incompatibilities with a specific platform/OS, bugs in the code, indeterministic behavior of the test etc.

These disablements are done by adding some prefix to test... methods in JUnit 3, by removing or commenting out @Test in JUnit 4 and 5, or by using @Ignore or @Disabled. In addition, several tests are only deactivated for specific platforms using early returns or assume statements within the test methods.

Goal

Since in most cases there is no additional, unique documentation or tagging of these disablements, they are not easy to find. This issue is supposed to be an umbrella to document disabled tests (existing ones and potentially further ones).

The purpose of this issue is to collect the disabled tests to have a central overview and provide the chance to reactivate and, if necessary, fix them. It also serves as a "good first issue" (see eclipse-platform/eclipse.platform.text#212 (comment)) for interested new contributors, which can take any of the collected tests and try to fix them to learn about the Eclipse platform code while, at the same time, already having the chance to make valuable contributions to the project.

Process

I see the following ideas as good starting points for identifying disabled tests:

First concentrate on a single project, such as org.eclipse.core.tests.resources
Use different techniques to find disabled tests:
1. Completely disabled tests:
  - Search for missing or commented out @Test annotation (JUnit 4 and 5)
  - Search for tests not starting with test (JUnit 3), in particular those starting with _test
2. Tests disabled for specific platforms:
  - Search for accesses to OS constants in Platform, such as OS_WIN32 and validate if they are used to disable tests for specific platforms
  - Search for accesses to methods isLinux() etc. in ResourceTest subclasses and validate if they are used to disable tests for specific platforms
  - Search for assumeFalse or assumeTrue statements that make tests only run on specific platforms

Examples

The tests modified in fix ContextInformationTest #35 eclipse.platform.text#212
Tests in org.eclipse.core.tests.resources.regression.Bug_032076
The skipTest helper in org.eclipse.core.tests.resources.usecase.SnapshotTest

Disabled in Maven

Some tests are disabled in Maven builds, such that integration builds do not reveal whether they fail or not.

The bundle org.eclipse.core.tests.net is completely disabled. Tests run fine withing Eclipse on Windows, but Tycho execution fails, probably because of missing platform-dependent fragments (maybe have a look at how SWT deals with that).
org.eclipse.team.tests.core.AllTeamUITests and org.eclipse.team.tests.ui.synchronize.AllTeamSynchronizeTests are not executed as the test suite only executes org.eclipse.team.tests.core.AllTeamTests. Tests run fine within Eclipse on WIndows, but Tycho fails to execute some of the ScopeTests and the AllTeamSynchronizeTest.

The text was updated successfully, but these errors were encountered:

Wittmaxi · 2023-06-29T11:59:14Z

split function into private subfunctions

…rm#525 The test testPerformanceOneResource is working fine even though it is disabled. Contributes to eclipse-platform#525

Reactivation of working tests that were forgotten after the corresponding bug was fixed. Contributes to eclipse-platform#525

In this commit, the performance test from the Marker Test class is reactivated because it can be run quickly and successfully. Contributes to eclipse-platform#525

Michael5601 · 2023-07-31T08:24:05Z

Addition to #525 (comment).
Checked tests have pending or accepted PR:

org.eclipse.core.tests.resources

org.eclipse.core.tests.runtime

PreferencesServiceTest#testApplyWithTransfers() - not implemented yet
YieldTest#testILockTransfer() - test is commented out

org.eclipse.ant.tests.ui

org.eclipse.core.expressions.tests

org.eclipse.e4.core.tests

ServiceSupplierTestCase#testUselessUpdates(), issue - NEW BUG

org.eclipse.debug.tests

org.eclipse.ua.tests

Reactivation of working tests that were forgotten after the corresponding bug was fixed. Contributes to eclipse-platform#525

Reactivation of working tests that were forgotten after the corresponding bug was fixed. Contributes to #525

This change reactivates all working disabled tests from project org.eclipse.debug.tests. The tests were forgotten after the corresponding bug was fixed. Contributes to eclipse-platform#525

This change reactivates all working disabled tests from project org.eclipse.core.tests.net. The status of the corresponding but is CLOSED WONTFIX but the test runs successfully. Contributes to eclipse-platform#525

This change reactivates all working disabled tests from project org.eclipse.team.tests.core. Contributes to eclipse-platform#525

Deleted useless test class Good.java because the test is trivial. Furthermore the package org.eclipse.tips.tests was also deleted because it is empty now. Contributes to eclipse-platform#525.

This change reactivates all working disabled tests from project org.eclipse.ua.tests. Bug 210024 has status CLOSED WONTFIX but still runs successfully. Bug 339274 has status RESOLVED FIXED. Contributes to eclipse-platform#525

Deleted useless test class Good.java because the test is trivial. Furthermore the package org.eclipse.tips.tests was also deleted because it is empty now. Contributes to #525.

This commit fixes the tests testLessThanOrEqualToExpression and testGreaterThanOrEqualToExpression of class CountExpression.java by adding additional functionality similar to the already existing. Contributes to eclipse-platform#525.

In this commit the test testCheckReceiver() from the class CheckTests.java in org.eclipse.debug.tests is reactivated and fixed. It is about checking if a checkbox is properly toggled in the ui after changing its state. After fixing the issue the checked box can be clearly seen while debuggin the code.

In this commit the test testCheckReceiver() from the class CheckTests.java in org.eclipse.debug.tests is reactivated and fixed. It is about checking if a checkbox is properly toggled in the ui after changing its state. After fixing the issue the checked box can be clearly seen while debuging the code. Contributes to eclipse-platform#525.

This commit reactivates and fixes the test testVerify() from FilterTest.java in org.eclipse.compare.tests. The test was outcommented in the past probably because Filter.validateResourceFilters(String) could not be performed anymore. Now validateResourceFilters(String) is performed on CompareResourceFilter. Contributes to eclipse-platform#525.

In this commit the test testCheckReceiver() from the class CheckTests.java in org.eclipse.debug.tests is reactivated and fixed. It is about checking if a checkbox is properly toggled in the ui after changing its state. After fixing the issue the checked box can be clearly seen while debuging the code. Contributes to eclipse-platform#525.

This commit reactivates and fixes the test testVerify() from FilterTest.java in org.eclipse.compare.tests. The test was outcommented in the past probably because Filter.validateResourceFilters(String) could not be performed anymore. Now validateResourceFilters(String) is performed on CompareResourceFilter. Contributes to #525.

In this commit the test testCheckReceiver() from the class CheckTests.java in org.eclipse.debug.tests is reactivated and fixed. It is about checking if a checkbox is properly toggled in the ui after changing its state. After fixing the issue the checked box can be clearly seen while debuging the code. Contributes to eclipse-platform#525.

In this commit the test testCheckReceiver() from the class CheckTests.java in org.eclipse.debug.tests is reactivated and fixed. It is about checking if a checkbox is properly toggled in the ui after changing its state. After fixing the issue the checked box can be clearly seen while debuging the code. Contributes to #525.

In this commit the two Tests setExtension and setExtension2 are reactivated and fixed. For this the class TipImage and especially its method setExtension have been changed. Now if setExtension is called, not only the instance variable fExtension is updated but also the instance variables fBase64Image or, if the URL constructor was used, fURL. These two instance variable are no final variables anymore, so they can be changed and tested. Contributes to eclipse-platform#525.

…-platform#525 In this commit the tests setExtension and setExtension2 are reactivated and changed. For this the method setExtension of TipImage.java is corrected to properly change the extension. Now if setExtension is called, not only the field fExtension is updated but also the field fBase64Image. Mind that for this change the field fBase64Image can't be final. Contributes to eclipse-platform#525.

…-platform#525 In this commit the method setExtension of TipImage.java is corrected to properly change the extension. Now if setExtension is called, not only the extension is updated but also the base64image because the extension is a part of this image. Mind that for this change the field fBase64Image can't be final because it needs to be updated when the extension changes. setExtension is not used in the constructor anymore to set the fExtension field, rather it is directly set. Also the tests setExtension and setExtension2 are reactivated and updated to test the functionality of setExtension. Contributes to eclipse-platform#525.

In this commit the method setExtension of TipImage.java is corrected to properly change the extension. Now if setExtension is called, not only the extension is updated but also the base64image because the extension is a part of this image. Mind that for this change the field fBase64Image can't be final because it needs to be updated when the extension changes. setExtension is not used in the constructor anymore to set the fExtension field, rather it is directly set. Also the tests setExtension and setExtension2 are reactivated and updated to test the functionality of setExtension. Contributes to #525.

…-platform#525 In this commit the method setExtension of TipImage.java is corrected to properly change the extension. Now if setExtension is called, not only the extension is updated but also the base64image because the extension is a part of this image. Mind that for this change the field fBase64Image can't be final because it needs to be updated when the extension changes. setExtension is not used in the constructor anymore to set the fExtension field, rather it is directly set. Also the tests setExtension and setExtension2 are reactivated and updated to test the functionality of setExtension. Contributes to eclipse-platform#525.

HeikoKlare added the good first issue Good for newcomers label Jun 28, 2023

HeikoKlare mentioned this issue Jun 28, 2023

Collect, Reenable and Fix Disabled Test Cases vi-eclipse/Eclipse-Platform#5

Open

Wittmaxi added a commit to Wittmaxi/eclipse.platform that referenced this issue Jul 10, 2023

Refactor CharsetManager#setCharsetFor eclipse-platform#525

4c0812f

split function into private subfunctions

Wittmaxi added a commit to Wittmaxi/eclipse.platform that referenced this issue Jul 10, 2023

Refactor CharsetManager#setCharsetFor eclipse-platform#525

fd424dc

split function into private subfunctions

HeikoKlare pushed a commit that referenced this issue Jul 11, 2023

Refactor CharsetManager#setCharsetFor #525

18f4a48

split function into private subfunctions

Michael5601 mentioned this issue Jul 28, 2023

Reactivate tests #525 #603

Merged

Michael5601 added a commit to CodeLtDave/eclipse.platform that referenced this issue Jul 28, 2023

Reactivate tests eclipse-platform#525

0100c37

Reactivation of working tests that were forgotten after the corresponding bug was fixed. Contributes to eclipse-platform#525

Michael5601 mentioned this issue Jul 28, 2023

Reactivate tests in org.eclipse.core.tests.resources #525 #605

Closed

HeikoKlare pushed a commit to CodeLtDave/eclipse.platform that referenced this issue Jul 31, 2023

Reactivate tests eclipse-platform#525

1d7e10f

Reactivation of working tests that were forgotten after the corresponding bug was fixed. Contributes to eclipse-platform#525

HeikoKlare pushed a commit to CodeLtDave/eclipse.platform that referenced this issue Jul 31, 2023

Reactivate tests eclipse-platform#525

59003c7

Reactivation of working tests that were forgotten after the corresponding bug was fixed. Contributes to eclipse-platform#525

HeikoKlare pushed a commit that referenced this issue Jul 31, 2023

Reactivate tests #525

d9dd334

Reactivation of working tests that were forgotten after the corresponding bug was fixed. Contributes to #525

Michael5601 mentioned this issue Aug 3, 2023

Reactivate tests in org.eclipse.debug.tests #525 #617

Merged

Michael5601 mentioned this issue Aug 3, 2023

Reactivate tests in org.eclipse.core.tests.net #525 #618

Merged

Michael5601 added a commit to CodeLtDave/eclipse.platform that referenced this issue Aug 3, 2023

Reactivate tests eclipse-platform#525

17779c9

This change reactivates all working disabled tests from project org.eclipse.team.tests.core. Contributes to eclipse-platform#525

Michael5601 mentioned this issue Aug 3, 2023

Reactivate tests in org.eclipse.team.tests.core #525 #619

Merged

Michael5601 mentioned this issue Aug 3, 2023

Remove unnecessary test class #525 #620

Merged

Michael5601 mentioned this issue Aug 3, 2023

Reactivate tests in org.eclipse.ua.tests #525 #621

Merged

akurtakov pushed a commit that referenced this issue Aug 3, 2023

Remove unnecessary test class #525

668133e

Deleted useless test class Good.java because the test is trivial. Furthermore the package org.eclipse.tips.tests was also deleted because it is empty now. Contributes to #525.

Michael5601 mentioned this issue Aug 7, 2023

Reactivate tests in org.eclipse.core.expressions.tests #525 CodeLtDave/eclipse.platform#2

Closed

Michael5601 mentioned this issue Sep 13, 2023

Reactivate tests in org.eclipse.debug.tests #525 #681

Merged

Michael5601 mentioned this issue Sep 25, 2023

Fixes the method setExtension of TipImage.java in org.eclipse.tips.tests #525 #723

Merged

jukzi added the test junit test related things label Nov 28, 2023

This was referenced Dec 7, 2023

Enabled tests eclipse-jdt/eclipse.jdt.ui#991

Merged

Enable tests eclipse-platform/eclipse.platform.ui#1389

Open

Enable tests #954

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reactivate tests: disabled tests collection #525

Reactivate tests: disabled tests collection #525

HeikoKlare commented Jun 28, 2023 •

edited

Loading

Wittmaxi commented Jun 29, 2023 •

edited by HeikoKlare

Loading

Michael5601 commented Jul 31, 2023 •

edited

Loading

Reactivate tests: disabled tests collection #525

Reactivate tests: disabled tests collection #525

Comments

HeikoKlare commented Jun 28, 2023 • edited Loading

Current Situation

Goal

Process

Examples

Disabled in Maven

Wittmaxi commented Jun 29, 2023 • edited by HeikoKlare Loading

org.eclipse.core.tests.resources

org.eclipse.core.tests.runtime

Michael5601 commented Jul 31, 2023 • edited Loading

org.eclipse.core.tests.resources

org.eclipse.core.tests.runtime

org.eclipse.ant.tests.ui

org.eclipse.core.expressions.tests

org.eclipse.e4.core.tests

org.eclipse.debug.tests

org.eclipse.core.tests.net.NetTest

org.eclipse.team.tests.core

org.eclipse.tips.tests

org.eclipse.ua.tests

HeikoKlare commented Jun 28, 2023 •

edited

Loading

Wittmaxi commented Jun 29, 2023 •

edited by HeikoKlare

Loading

Michael5601 commented Jul 31, 2023 •

edited

Loading