also write file that tyche needs to differentiate between the same te…

[pppls]

…st run at differnt times.

specify default directory name and files.

Name of file containing testcases is {date:yyyy-M-dd}_testcases.jsonl. So "2024-10-19_testcases.jsonl".
Name of file containing testinfo is {date:yyyy-M-dd}_testinfo.jsonl. So "2024-10-19_testinfo.jsonl".

Located in ".cscheck\observed" directory, which is generated wherever the test is run.

I also verified that the test cases are generated in realtime by building the nuget package and copying the tests I write in a separate project.

[AnthonyLloyd]

Thanks

[pppls]

@hgoldstein95 would you be able to add the above location to Tyche?

[AnthonyLloyd]

@pppls @hgoldstein95 Should the location be by the instead test dll? I'm worried trying to get from the test dll to the project directory instead is a bit buggy.

[pppls]

Not sure. To be clear, what I meant was that Tyche probably needs to update their vscode settings.json to:

{
    "editor.autoSurround": "never",
    "editor.autoClosingBrackets": "never",
    "workbench.editorAssociations": {
        "*.ipynb": "jupyter-notebook"
    },
    "files.autoSave": "afterDelay",
    "[json]": {

        "editor.quickSuggestions": {
            "strings": true
        },
        "editor.suggest.insertMode": "replace"
    },
    "terminal.integrated.profiles.windows": {
        "PowerShell": {
          "source": "PowerShell",
          "icon": "terminal-powershell",
          "args": ["-ExecutionPolicy", "Bypass"]
        }
      },
    "terminal.integrated.defaultProfile.windows": "PowerShell",
    "git.enableSmartCommit": true,
    "git.confirmSync": false,
    "svelte.enable-ts-plugin": true,
    "tyche.observationGlobs": [
    
        "**/.cscheck/observed/*.jsonl", <-- NEW 
        "**/.hypothesis/observed/*.jsonl",
        "**/.quickcheck/observations/*.jsonl"
    ],
}

But I'm not sure if that's all there is to it.

[AnthonyLloyd]

That looks relative to the dll? Maybe it should just look for **/observed/*.jsonl or something only Tyche specific.

@pppls @hgoldstein95 Does Tyche work with tests running in parallel? I'm just considering if it can globally be turned on by config or if there is no need.

[pppls]

No, I completely forgot to mention that yesterday. The usage of the Channel ensures that testcases will be written sequentially to the "2024-10-19_testcases.jsonl" file within the test being run. But running two separate tests in parallel that both write to this file will very likely result in race conditions.

[hgoldstein95]

Tyche just reads updates to the provided file, so if the tests were synchronized to ensure that the actual jsonl data is valid, I think it shouldn't matter if the data for two tests were interleaved in the file.

As for whether Tyche should be looking for **/observed/*.jsonl, my original rationale for having one directory per framework was to avoid accidental overlap if there were multiple frameworks in the same project or something like that. And there's an added bonus that folks who implement Tyche for their frameworks are incentivized to get in touch with me, which means I can provide support and publicize the different frameworks who have built integrations.

By default I'll add **/.cscheck/observed/*.jsonl to the Tyche default configuration, but let me know if you disagree with my reasoning.

[hgoldstein95]

The directory is relative to the VSCode workspace, by the way. Not the test binary.