Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partially reinstate problem fail tests #658

Merged
merged 18 commits into from
Sep 26, 2022

Conversation

bobismijnnaam
Copy link
Contributor

@bobismijnnaam bobismijnnaam commented Jul 14, 2021

The PR marks two unstably failing tests (leader elect & OpenMPSectionReducedFail1) with --stop-before-backend, so they will not be verified with silicon anymore. We track their instability in #514.

This pr marks all tests in problem-fail that fail because of the back-end as --stop-before-backend, and marks all tests that fail because of a pass with --stop-after-typecheck. These tests are removed from problem-fail if they are marked with one of these flags, and added to the suite problem-partial.

Finally, some tests are removed from problem-fail, as in the meantime the problems have been fixed.

This PR closes #651. It needs to be rebased from main, as it builds upon commits in #657.

  • ☑️ = moved to regular suite
  • ⏩ = moved to problem-partial with --stop-before-backend
  • ▶️ = moved to problem-partial with --stop-after-typecheck
  • ❎ = kept in problem-fail

Files left:

@bobismijnnaam bobismijnnaam temporarily deployed to Default July 14, 2021 08:09 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 14, 2021 08:26 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 14, 2021 09:02 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 14, 2021 09:02 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 14, 2021 09:49 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 14, 2021 09:49 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 14, 2021 09:54 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 14, 2021 09:54 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 14, 2021 11:59 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 14, 2021 11:59 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 14, 2021 12:20 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 14, 2021 12:20 Inactive
@bobismijnnaam bobismijnnaam marked this pull request as draft July 14, 2021 12:21
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 14, 2021 12:38 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 14, 2021 12:43 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 14, 2021 12:57 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 14, 2021 12:57 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 14, 2021 13:13 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 14, 2021 13:17 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 07:57 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 07:57 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 08:13 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 08:15 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 08:41 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 08:41 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 08:47 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 08:47 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 09:05 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 09:06 Inactive
Required some rewriting since the grammar has changed since it was
removed from the test suite. Also, changed silicon to carbon, since it
seems a bit faster that way.
@bobismijnnaam bobismijnnaam force-pushed the partially-reinstate-problem-fail-tests branch from 8261b95 to ee05bb1 Compare July 15, 2021 14:49
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 14:49 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 14:49 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 15:07 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 15:07 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 15:19 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 15:19 Inactive
This required turning off Flatten.java inside ADTs, since there no
flattening is necessary given a correct input program.  If strict
typechecking (e.g. prevention of use of methods in ADTs) is needed
inside ADTs in silver, this should be implemented in the future in the
typechecker.
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 15:35 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 15:35 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 15:55 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 15, 2021 15:55 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 16, 2021 09:32 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 16, 2021 09:32 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 16, 2021 09:49 Inactive
@bobismijnnaam bobismijnnaam temporarily deployed to Default July 16, 2021 09:49 Inactive
@sonarcloud
Copy link

sonarcloud bot commented Jul 16, 2021

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

100.0% 100.0% Coverage
0.0% 0.0% Duplication

@bobismijnnaam bobismijnnaam marked this pull request as ready for review July 19, 2021 08:47
@pieter-bos pieter-bos temporarily deployed to Default September 26, 2022 14:01 Inactive
@pieter-bos pieter-bos merged commit c29c233 into dev Sep 26, 2022
@pieter-bos pieter-bos temporarily deployed to Default September 26, 2022 14:06 Inactive
@pieter-bos pieter-bos temporarily deployed to Default September 26, 2022 14:14 Inactive
@pieter-bos pieter-bos deleted the partially-reinstate-problem-fail-tests branch September 27, 2022 11:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Staying on top of e.g. problem-fail cases
2 participants