Implement mutation switching 👽🔀 #1514

nicojs · 2019-04-19T11:29:45Z

Intro

Mutation switching is a technique that can be used to improve the performance of mutation testing. It is used in Stryker.NET as well as Stryker4s. We believe this has a lot of benefits for JavaScript/TypeScript as well.

In short, with mutation switching all mutants will be placed into the source code at once. Only one mutant can be active at the same time. Here is a small example:

/* 1*/ // original
/* 2*/ function add(a, b) {
/* 3*/   return a + b;
/* 4*/ }
/* 5*/ 
/* 6*/ // Might be instrumented to:
/* 7*/ function add(a, b) {
/* 8*/   if (global.activeMutant === 0) {
/* 9*/      // block mutation, empty function
/*10*/   } else {
/*11*/     return global.activeMutant === 1? a - b: a+b;
/*12*/   }
/*13*/ }

Potential performance improvements

Although JavaScript is not a compiled language, there is still huge performance improvements to be had.

Less I/O. This can be a big performance overhead, since not only the mutated file has to be cleared from node's require cache, but the entire object graph that is dependent on that file, which in practice means clearing the entire cache (100's of files for running Stryker on Stryker). For karma it means a reload between test runs with sometimes MB's of files to reload.
One-time transpiling. A lot of projects depend on some kind of transpiler/bundler, also to run the tests. Think of TypeScript, Webpack, Vue CLI, Angular CLI, Jest-ts, ts-node, etc. They will all have to transpile only once, instead of once per mutant.
Hot reload. Test runners won't have to clear their test suite between tests (there is a caveat here regarding static code, see hurdles).
Test coverage analysis. Since we're instrumenting the code, it's pretty much free to add mutation coverage instrumentation in the code. For example, line 11 could be mutated to: return global.activeMutant === 1 ? a - b : (global__coverMutant__(1), a + b);. The __coverMutant__ here is an example of how we can count coverage analysis. Mutants that aren't covered, don't need to be tested. If we add the context of which test is running at that point in time, we can only run the tests that cover a particular mutant. We currently have a very complex coverage analysis in place, which we can totally get rid of once we have this new test coverage analysis system in place. It will work for a lot more use cases, no matter which transpiler or bundler or even minified you use. However, we will need to change all test runners to report this new mutation coverage statistic. There is also complication here for static code, see hurdles.

Mutation switch instrumenter PoC.

I've created a Proof of Concept of mutation switching. It works for a couple of our mutations and I'm confident we can add them all. It has a lot of benefits:

It supports both JS and TS with one AST (thanks to babeljs).
It supports JS, TS, Vue and HTML file formats (thanks to angular-html-parser).
Support mutant coverage instrumentation.

Hurdles

There are some hurdles to overcome.

We need to make sure that mutated code can be transpiled by the supported transpilers out there. Webpack, Babel, TypeScript, and friends will have to produce JavaScript to run. This is not trivial since we are definitely introducing type errors into the source code.
We will still need to type check each mutant. A.t.m. this is the job of the transpiler. We will need to change that, as the transpiler will have to produce code no matter what. I would suggest adding a new plugin type for that "TypeChecker". We might want to not make it a public API at first, to save time and maintenance.
Test runners will have to support at least running the tests. Reporting the mutation coverage and hot reload are a nice to have, although Stryker might actually slow down without it (due to larger file sizes).
There's also the concept of static code. Code that is executed once when imported and then never again. If we mutate in that part, we would need to detect it and turn off hot reload for those scenarios. Detecting static code can be done by reporting mutant coverage. Mutants that were covered before a test runs can be considered "static".
Our current way of test filtering in Mocha will no longer work in combination with hot reload.

Rethinking `Transpilers`

We currently have a Transpiler API. It is responsible for transpiling the code for the initial test run, as well as transpiling each mutant. Stryker takes care to load the transpiler and keep it alive, so the transpiler can use in-memory --watch functionality if it can.

Examples of transpilers:

Specific transpiler plugins are no longer needed when using mutation switching. We only need to transpiler once (with all mutants in the code), so there is no need to painstakingly transpile each mutant. The amount of maintenance is simply not worth the effort to keep this transpiler API, if only to run it once.

Instead, we should allow a user to configure a "build command" (name pending). It can be tsc -b for example, or webpack --configFile webpack.conf.js. This build command is then executed inside the sandbox before the initial test run starts. This sandbox is reused for the mutant test runs.

When copying files to the sandbox, we might need to change them slightly. Take this tsconfig file for example:

{
  "extends": "../../tsconfig.settings.json",
  "references": [ 
    { "path": "../api/tsconfig.src.json" },
    { "path": "../util/tsconfig.src.json" }
  ]
}

Simply picking this file up and dropping it in the sandbox won't work. Assuming the sandbox is copied 2 directories deep, we need to preprocess it to be:

{
  "extends": "../../../../tsconfig.settings.json",
  "references": [ 
    { "path": "../../../api/tsconfig.src.json" },
    { "path": "../../../util/tsconfig.src.json" }
  ]
}

The goal of a transpiler

Migration scenario

I think a small step-by-step migration is out of the question, as approximately half of our code will have to be rewritten, or completely scrapped. Our public API will also need to be changed in a couple of places too.

If we want to keep feature parity with the current features, we will at least need support for transpiling using babel, webpack, typescript, jest, ts-node, karma-webpack, mocha-webpack, vue cli, angular CLI and react scripts and test runner support for karma, jasmine, mocha, and jest (I think we can drop WCT).

A high-level overview of the new way of working of a mutation test run:

Read config
Instrument the code
1. Parse
2. Generate mutants
3. Place mutants
4. Print the code in memory
Transpile the code (once)
Fill one sandbox, run tests once for the control run (initial test run)
Check mutants using the configured "check" plugins
Test the remaining mutants in a sandbox

Investigation progress

There's a lot of stuff we need to figure out. We'll have to investigate how all features are possible. Note: there is a lot of stuff "nice to have" here, so we shouldn't block implementation.

Implementation plan

Implementation should be done in steps. To make it easier on ourselves, we should allow us to break parts of Stryker in the meantime. I suggest therefore to work in a feature branch. We'll be implementing mutation switching on that branch.

In the meantime, our master branch will keep getting security patches, but we want to keep the functional changes to a minimum.

Question: Do we want to keep Dependabot during this time? I think it is better to point Dependabot to the feature branch instead of master.

The plan:

Preparation

Implementation

I'll be keeping this post up to date with our current progress.

The text was updated successfully, but these errors were encountered:

nicojs · 2019-07-01T05:31:00Z

This would require a major change in how mutation is done. I've discussed this face to face with @simondel and the easiest thing will be to build it next to the current mutation implementation and let people opt-in with an --expermimentalMutationSwitching command line flag (or something like it). That way we can actually release it before we remove the old implementation while keeping it all backward compatible.

We'll be starting development on this next week probably. We'll update this issue along the way.

nicojs · 2020-04-17T14:36:50Z

I've been working on a PoC for a couple of weeks now and it seems to be working with some success! https://github.com/nicojs/mutation-switch-instrumenter#mutation-switch-instrumenter

I'll be using that PoC to see if we can run tests on Angular/Vue/React projects with Karma/Mocha/Jest, etc.

bartekleon · 2020-04-17T21:23:16Z

Sounds great @nicojs! If you want, you can use my BigMath library to check the performance of this solution :)

nicojs · 2020-04-21T05:56:21Z

Not so fast there, cowboy 🤠. I've updated the original issue text to reflect the current progress in research and thoughts. As you can see, still some work to be done before we can start implementing.

nicojs · 2020-04-23T08:07:37Z

I've started to work on an implementation plan (see original issue text)

vitalets · 2020-04-24T11:32:23Z

Hi @nicojs !

I'm impressed with your Mutation switching new approach. Looking forward for it!

By the way I was thinking about another way of applying mutations in JS/TS - using debugger API. On the first glance the flow can be following:

set the breakpoint on the required line of source code
run tests
when debugger paused on the breakpoint - capture variable values and mutate it
resume execution and check tests result

Seems like such approach has it's own pros and cons.
Pros:

no instrumentation step: easier integration with frameworks

Cons:

less mutations: we can only change variable values and function return value

What do you think about it?

nicojs · 2020-04-24T15:39:05Z

That's interesting. It does seem seriously limited. I'm also not sure if it is truly easier to implement since you'll need to inspect the test runner process, calculate the breakpoint position back to the original source code, and than understand the code enough to know which value you want to debug.

If you want to talk more about it, let's do that on slack: https://app.slack.com/client/TTHUR6NNP

nicojs · 2020-05-08T19:09:46Z

As for the karma mutation coverage reporting. This seems pretty trivial with Karma's plugin system. A small PoC:

// stryker-karma-plugin.js
module.exports = {
  'framework:stryker': ['factory', strykerFramework],
  'middleware:stryker': ['value', strykerMiddleware],
  'middleware:body-parser': ['value', require('body-parser').json()]
}

function strykerFramework (config) {
  config.files.unshift({
    pattern: require.resolve('./adapter.js'), 
    included: true, 
    watched: false,
    served: true
  });
}

strykerFramework.inject = ['config'];

function strykerMiddleware(request, _response, next) {
  if (request.method === 'POST' && request.url === '/stryker-info') {
    console.log('Received from stryker:')
    console.log(request.body);
  } else {
    next();
  }
}

// adapter.js
const originalComplete = window.__karma__.complete.bind(window.__karma__);
window.__karma__.complete = (...args) => {
  fetch('/stryker-info', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      test: true
    })
  }).then(() => {
    originalComplete(...args);
  });
}

Activate with:

// karma.conf.js

module.exports = function(config){
  config.set({
    files: ['src/*.js', 'test/*.js'],
    frameworks: [
      'jasmine',
      'stryker' // Activate the "stryker" framework
    ],
    plugins: [
      'karma-*',
      require.resolve('./stryker-karma-plugin') // Add plugin
    ],
    singleRun: true,
    browsers: [
      'Chrome'
    ],
    middleware: [
      'body-parser', // body-parser used to parse json
      'stryker' // Activate the "stryker" middleware
    ]
  })
}

nicojs · 2020-05-15T08:38:52Z

Yow! We've started work on this in epic/mutation-switching.

Now first, let me get a coffee ☕

nicojs · 2020-06-01T19:09:48Z

About the transpilers plugin. I was wondering if we still need the complexity we now have. Couldn't we just copy files to the sandbox and run the transpiler there? We could allow people to configure an arbitrary command to be run before we're running files.

The files that are copied to the sandbox should still be transformed somehow. For example, we should add // @ts-nocheck to each typescript and javascript file. We should also transform tsconfig.json files to make relative paths work. For example, "extends": "../../tsconfig.settings.json" becomes "extends": "../../../../tsconfig.settings.json". Maybe we can use this transform plugin and add a command transpiler, which runs a simple command once.

@simondel @hugo-vrijswijk thoughts?

simondel · 2020-06-03T16:58:39Z

If TypeScript projects can be supported as easily as they can now, I'm all for it :)

nicojs · 2020-08-14T11:32:52Z

For anyone that wants to help. We're using the 4.0 milestone for this.

https://github.com/stryker-mutator/stryker/milestone/9

jimmyandrade · 2020-08-17T20:47:23Z

Wow, looking forward for this ❤️

nicojs · 2020-08-18T16:02:37Z

If anyone wants to give it a try, we've released a beta 🤗
https://stryker-mutator.io/blog/2020-07-13/announcing-stryker-4-beta-mutation-switching

I'm hoping to release v4 in a few weeks 🤐

nicojs · 2020-08-18T16:08:47Z

@tummerhore

Sadly my initial test with Stryker took nearly 2 days (!) to complete (running on a laptop with i7-6600U) so it would be very interesting to see how mutation switching (eventually along with coverage analysis for jest) may impact the total execution time in this case and if that would make it more feasible

Wow! I think you are my personal hero 🦸‍♂️.

Mutation switching by itself will not have a significant impact on Jest projects unfortunately. You might have some performance improvement because Jest would be able to cache build results, but definitely not an order of magnitude. However, it does open up the way for coverage analysis in jest. Hot reload should theoretically also be possible with Jest, but not with their current public api alone. We'll be focussing on jest next, after we've released v4.

bartekleon · 2020-08-18T16:13:54Z

I think I'll have to check it on my BigMath project soon :D

brodycj · 2020-08-18T19:36:40Z

I tried this with Prettier on an 8 vCPU cloud instance, see brodycj/prettier#42 but encountered some issues:

memory issues with a couple large source files
timeout error message on initial test run, even with --timeoutMS 5000000 command line option, then went quiet

I think brodycj/prettier#42 should be enough to reproduce my issues, but please let me know if there is anything else I can do to help isolate them.

lukas-schaetzle · 2020-08-18T19:41:12Z

@brodybits As far as I know --timoutMS is not used for the initial test run but for the actual mutation testing. Currently (or at least in the old stable version) the maximum time for the inital test run is hard coded as 5 minutes.
Also see my issue: #2302

bartekleon · 2020-08-18T20:17:17Z

@brodybits there could be some memory leak 🤔

brodycj · 2020-08-19T04:00:14Z

For Prettier, the initial test run went much faster when switching from Jest to the command runner. But for some reason I am getting many test failures from the initial test run. I have a feeling that something is going very wrong with what the mutation switching does to the JavaScript and will try to investigate it another day.

nicojs · 2020-08-19T10:39:58Z

@brodybits

I complement your bravery, running Stryker on such huge open source projects. But yes, why shouldn't we?

Unsupported type ConditionalExpression

See #2400

TypeError: Property id of DeclareModule expected node

See #2399 (this one should not be that hard, string literal mutator shouldn't mutate DeclareModule identifiers.

For Prettier, the initial test run went much faster when switching from Jest to the command runner. But for some reason I am getting many test failures from the initial test run.

See #2401

🎉

Lakitna · 2020-08-19T10:54:06Z

I've managed to get it working too, though I did encounter a few bugs. See #2398 and #2402 for details.

Some background. I'm working on a .js project without Typescript, bundling, or other instrumentation on the source code. I'm using Mocha for my test runner.

Things I noticed:

There are fewer mutations. This is not necessarily a bad thing though, as long as it's intended.
Function attribute defaults are now properly mutated and tested (ie: function(hello=true) {}). I believe this is a known issue with Stryker@3 that has now been solved.
I am now running 15 instances, where I used to run 16. Again not necessarily a bad thing, it feels like a new default for max concurrent processes option.
The biggest thing for me though is that it's actually a lot slower. Below I've added the progress reporter output for both Stryker@3 and [email protected]. Note how the new one is 2 whole minutes slower, that's a time increase of +70%.

Stryker@3
11:08:32 (16712) WARN InputFileResolver Globbing expression "" did not result in any files.
11:08:32 (16712) WARN InputFileResolver Globbing expression "" did not result in any files.
11:08:33 (16712) INFO InputFileResolver Found 87 of 279 file(s) to be mutated.
11:08:33 (16712) INFO InitialTestExecutor Starting initial test run. This may take a while.
11:08:44 (16712) INFO InitialTestExecutor Initial test run succeeded. Ran 1028 tests in 12 seconds (net 1379 ms, overhead 5139 ms).
11:08:45 (16712) INFO MutatorFacade 3196 Mutant(s) generated
11:08:45 (16712) INFO SandboxPool Creating 16 test runners (based on CPU count)
Mutation testing  [] 8% (elapsed: <1m, remaining: ~3m) 266/3064 tested (3 survived, 0 timed out)11:09:08 (16712) WARN Sandbox Failed find coverage data for this mutant, running all tests. This might have an impact on performance: StringLiteral: ("") file://{{projectRoot}}\src\cli\translators\detail.js:97:36
Mutation testing  [] 72% (elapsed: ~2m, remaining: <1m) 2223/3064 tested (73 survived, 2 timed out)11:10:49 (16712) WARN Sandbox Failed find coverage data for this mutant, running all tests. This might have an impact on performance: BooleanLiteral: (false) file://{{projectRoot}}\src\core\xml\xml-path.js:44:31
Mutation testing  [] 73% (elapsed: ~2m, remaining: <1m) 2251/3064 tested (73 survived, 2 timed out)11:10:50 (16712) WARN Sandbox Failed find coverage data for this mutant, running all tests. This might have an impact on performance: BooleanLiteral: (false) file://{{projectRoot}}\src\core\xml\xml-path.js:87:24
Mutation testing  [] 74% (elapsed: ~2m, remaining: <1m) 2297/3064 tested (73 survived, 2 timed out)11:10:53 (16712) WARN Sandbox Failed find coverage data for this mutant, running all tests. This might have an impact on performance: BooleanLiteral: (false) file://{{projectRoot}}\src\core\xml\xml-path.js:192:31
Mutation testing  [] 100% (elapsed: ~2m, remaining: n/a) 3064/3064 tested (89 survived, 7 timed out)
11:11:29 (16712) INFO MutationTestReportCalculator Final mutation score of 92.30 is greater than or equal to break threshold 50
11:11:29 (16712) INFO HtmlReporter Your report can be found at: file:///{{projectRoot}}/reports/mutation/html/index.html
11:11:29 (16712) INFO Stryker Done in 2 minutes 56 seconds.

[email protected]
11:38:50 (18168) WARN InputFileResolver Globbing expression "" did not result in any files.
11:38:50 (18168) WARN InputFileResolver Globbing expression "" did not result in any files.
11:38:50 (18168) INFO InputFileResolver Found 87 of 279 file(s) to be mutated.
11:38:52 (18168) INFO Instrumenter Instrumented 87 source file(s) with 2890 mutant(s)
11:38:52 (18168) INFO ConcurrencyTokenProvider Creating 15 test runner process(es).
11:38:52 (18168) INFO DryRunExecutor Starting initial test run. This may take a while.
11:38:55 (18168) INFO DryRunExecutor Initial test run succeeded. Ran 1028 tests in 5 seconds (net 1282 ms, overhead 797 ms).
Mutation testing  [=========] 100% (elapsed: ~4m, remaining: n/a) 2890/2890 tested (199 survived, 10 timed out)
11:43:49 (18168) INFO MutationTestReportHelper Final mutation score of 93.10 is greater than or equal to break threshold 50
11:43:50 (18168) INFO HtmlReporter Your report can be found at: file:///{{projectRoot}}/reports/mutation/html/index.html
11:43:50 (18168) INFO MutationTestExecutor Done in 4 minutes 59 seconds.

brodycj · 2020-08-20T01:26:29Z

Thanks @nicojs. I think the other major thing with Prettier is the resource usage and long prep time I had encountered on a large cloud instance, as I listed in #2401 (comment). I did clean up my attempt in brodycj/prettier#42, in case it may help at all.

I can try to test new beta releases from time to time, otherwise leaving it in your hands for now. Yeah what a monster mutating Prettier!

P.S. I did minimize my long comment as "duplicate", since it is now covered by separate issues.

nicojs · 2020-08-20T16:34:27Z

@Lakitna thanks for giving the beta a spin!

There are fewer mutations

This is probably because we don't generate mutants anymore that we couldn't place (i.e. babel doesn't allow invalid syntax). A good example is property keys:

const foo = {
  'bar': baz
}

In this example, the string literal bar was mutated in 3.0 but no longer in 4, because inserting a ternary operator there isn't allowed there. We could place the mutant with a switch case for example, but I decided against it, because we don't mutate regular property names, so why mutate them when they appear as literals?

I believe this is a known issue with Stryker@3 that has now been solved.

Awesome! 🍺

I am now running 15 instances, where I used to run 16.

Yes, during the refactoring of the concurrency (for the checker api), I've decided to reserve 1 core for the stryker main process by default. You can override it with --concurrency. Note: the max concurrency is no longer capt by Stryker, so don't use a too high concurrency.

The biggest thing for me though is that it's actually a lot slower.

Hmm interesting. Let's take a closer look at your use case and the possible performance improvements with mutation switching:

Less I/O. This improvement might not have a big impacting on your use case. 🤷‍♂️
One-time transpiling. You don't use transpiling ❌
Hot reload. This would be a big performance improvement I think, but alas, this isn't implemented yet. ❌
Test coverage analysis. This is something Stryker 3 already supported with istanbul for your use case, correct? So no performance gain here. ❌

I was planning to pickup hot reload after the 4.0 release, but with this performance impact in mind, maybe we should already implement it in the beta? What do you think?

Just to be sure, you are running with --coverageAnalysis perTest correct?

Lakitna · 2020-08-20T19:37:48Z

I decided against it, because we don't mutate regular property names, so why mutate them when they appear as literals?

That was actually a minor annoyance for me. A good decision as far as I'm concerned

I've decided to reserve 1 core for the stryker main process by default. You can override it with --concurrency

I'll give it a run with --concurrency as a performance check. I'll also keep an eye on my CPU usage while it's running. My gut tells me that the reserved core makes sense, but I'm curious anyway 😊

I was planning to pickup hot reload after the 4.0 release, but with this performance impact in mind, maybe we should already implement it in the beta? What do you think?

The way you phrased it makes me think that hot reload is not a breaking change, right?

The thing is that it's worse for me, but a lot better for others in this thread. In fact, Stryker suddenly became feasible for bigger projects.

Thinking about this makes me curious about what the impact is on a small project with transpiling. Something like a React/Angular/Vue/... front-end written in Typescript. If someone knows about an open-source project like this, I'll be glad to run it in 3 and 4.beta.

In fact, I've been wondering about how Stryker scales in general. The relation between mutation count and runtime feels exponential-ish. Would you be up for including some benchmarks in the repo? I'm not sure how to set it up yet, but I have some ideas.

If the only projects that are worse off are those like mine, then I think its a no-brainer to release asap. When It's not as clear, and it rarely is, you can also release 4 with the note in the changelog that it comes with a performance hit for smaller projects.

Just to be sure, you are running with --coverageAnalysis perTest correct?

Yes, I am.

nicojs · 2020-08-21T11:38:38Z

Hot reload [...] The way you phrased it makes me think that hot reload is not a breaking change, right?

Well ... it might break some test suites that don't clean up nicely between test runs. See #2413

The thing is that it's worse for me, but a lot better for others in this thread. In fact, Stryker suddenly became feasible for bigger projects.

That's the goal yes! 😊

The relation between mutation count and runtime feels exponential-ish. Would you be up for including some benchmarks in the repo?

Yes, please! Benchmarking would be awesome! I started trying out some of them in the perf directory. My goal was to add use cases there, but I stopped at one (angular project).

Feel free to add more use cases there! A great example of a big project that might look like your use case is express. If someone wants to add, please make the PR against the current master branch, that way it will make it easy for us to test it in both versions. 🤘

Lakitna · 2020-08-25T09:16:57Z

Feel free to add more use cases there! A great example of a big project that might look like your use case is express.

Express would be an interesting one for sure. I've created an issue for it.

Though I've been thinking even further. I mentioned the relation between mutation count and runtime, I think it's interesting to explore that. So I was thinking of benchmarks that can be run with a variable amount of mutations at regular intervals. This would provide us with a performance graph, rather than a single number. I feel like something like that could help tremendously with decision making.

nicojs added the 🚀 Feature request New feature request label Apr 19, 2019

nicojs mentioned this issue Apr 19, 2019

Investigate mutation testing GoogleChrome/lighthouse#7607

Closed

bartekleon mentioned this issue Jun 27, 2019

slowly prepare to drop node 8 support #1611

Closed

nicojs mentioned this issue Jul 1, 2019

Support incremental analysis #322

Closed

bartekleon mentioned this issue Jul 2, 2019

perf: reduce next stryker runs #1610

Closed

nicojs mentioned this issue Jul 4, 2019

Ignore specific mutations #1472

Closed

simondel mentioned this issue Nov 11, 2019

Setting up Iteration Plan / Roadmaps #1834

Closed

nicojs mentioned this issue Apr 21, 2020

Add ability to run tests in a mocha instance multiple times mochajs/mocha#4234

Merged

10 tasks

nicojs changed the title ~~Investigate performance improvement: mutation switching~~ Implement mutation switching 👽🔀 Apr 23, 2020

This was referenced Apr 24, 2020

Coverage analysis based on mutants instead of code coverage #2104

Closed

Allow to mutate files in place #2163

Closed

nicojs mentioned this issue Apr 29, 2020

Interest in support for Intern #430

Closed

nicojs pinned this issue May 8, 2020

This was referenced May 18, 2020

NPM run postinstall in Contributing.md #2217

Closed

Specify lines to mutate #1980

Closed

nicojs mentioned this issue Jun 1, 2020

feat(checker): add checker api #2240

Merged

nicojs mentioned this issue Jul 29, 2020

Implement mutation switching #2340

Merged

nicojs pinned this issue Aug 4, 2020

brodycj mentioned this issue Aug 18, 2020

try Stryker Mutator 4 (beta) brodycj/prettier#42

Draft

2 tasks

This comment has been minimized.

Sign in to view

This was referenced Aug 19, 2020

Running Stryker 4.0 beta on prettier gives errors in initial test run #2401

Closed

Multiple config workarounds needed for dry run to work for React [Stryker 4.0.0-beta] #2403

Closed

nicojs mentioned this issue Aug 21, 2020

Implement hot reload for mocha test runner #2413

Closed

Lakitna mentioned this issue Aug 25, 2020

Performance benchmark: Express #2417

Closed

nicojs mentioned this issue Aug 28, 2020

Discussion: TypeScript compile errors with mutation switching #2438

Closed

nicojs mentioned this issue Sep 25, 2020

Analysis crashes with 'too many open files' #1227

Closed

nicojs closed this as completed in #2340 Oct 7, 2020

sanderkoenders unpinned this issue Oct 7, 2020

brodycj mentioned this issue Mar 31, 2021

test: Stryker Jest runner with config workarounds xmldom/xmldom#208

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement mutation switching 👽🔀 #1514

Implement mutation switching 👽🔀 #1514

nicojs commented Apr 19, 2019 •

edited

Loading

nicojs commented Jul 1, 2019

nicojs commented Apr 17, 2020

bartekleon commented Apr 17, 2020

nicojs commented Apr 21, 2020

nicojs commented Apr 23, 2020

vitalets commented Apr 24, 2020

nicojs commented Apr 24, 2020

nicojs commented May 8, 2020 •

edited

Loading

nicojs commented May 15, 2020

nicojs commented Jun 1, 2020

simondel commented Jun 3, 2020

nicojs commented Aug 14, 2020

jimmyandrade commented Aug 17, 2020

nicojs commented Aug 18, 2020

nicojs commented Aug 18, 2020

bartekleon commented Aug 18, 2020

brodycj commented Aug 18, 2020

lukas-schaetzle commented Aug 18, 2020 •

edited

Loading

bartekleon commented Aug 18, 2020

This comment has been minimized.

brodycj commented Aug 19, 2020

nicojs commented Aug 19, 2020

Lakitna commented Aug 19, 2020 •

edited

Loading

brodycj commented Aug 20, 2020 •

edited

Loading

nicojs commented Aug 20, 2020

Lakitna commented Aug 20, 2020 •

edited

Loading

nicojs commented Aug 21, 2020 •

edited

Loading

Lakitna commented Aug 25, 2020

Implement mutation switching 👽🔀 #1514

Implement mutation switching 👽🔀 #1514

Comments

nicojs commented Apr 19, 2019 • edited Loading

Intro

Potential performance improvements

Mutation switch instrumenter PoC.

Hurdles

Rethinking Transpilers

Migration scenario

Investigation progress

Implementation plan

Preparation

Implementation

nicojs commented Jul 1, 2019

nicojs commented Apr 17, 2020

bartekleon commented Apr 17, 2020

nicojs commented Apr 21, 2020

nicojs commented Apr 23, 2020

vitalets commented Apr 24, 2020

nicojs commented Apr 24, 2020

nicojs commented May 8, 2020 • edited Loading

nicojs commented May 15, 2020

nicojs commented Jun 1, 2020

simondel commented Jun 3, 2020

nicojs commented Aug 14, 2020

jimmyandrade commented Aug 17, 2020

nicojs commented Aug 18, 2020

nicojs commented Aug 18, 2020

bartekleon commented Aug 18, 2020

brodycj commented Aug 18, 2020

lukas-schaetzle commented Aug 18, 2020 • edited Loading

bartekleon commented Aug 18, 2020

This comment has been minimized.

brodycj commented Aug 19, 2020

nicojs commented Aug 19, 2020

Lakitna commented Aug 19, 2020 • edited Loading

brodycj commented Aug 20, 2020 • edited Loading

nicojs commented Aug 20, 2020

Lakitna commented Aug 20, 2020 • edited Loading

nicojs commented Aug 21, 2020 • edited Loading

Lakitna commented Aug 25, 2020

nicojs commented Apr 19, 2019 •

edited

Loading

Rethinking `Transpilers`

nicojs commented May 8, 2020 •

edited

Loading

lukas-schaetzle commented Aug 18, 2020 •

edited

Loading

Lakitna commented Aug 19, 2020 •

edited

Loading

brodycj commented Aug 20, 2020 •

edited

Loading

Lakitna commented Aug 20, 2020 •

edited

Loading

nicojs commented Aug 21, 2020 •

edited

Loading