Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WinUI Unit testing with UIFact doesn't appear to work. #77

Open
ScottAtBarco opened this issue May 25, 2023 · 18 comments
Open

WinUI Unit testing with UIFact doesn't appear to work. #77

ScottAtBarco opened this issue May 25, 2023 · 18 comments

Comments

@ScottAtBarco
Copy link

I created a WinUI project for testing some BitmapImage-related code using MSTest along the lines recommended here: https://devblogs.microsoft.com/ifdef-windows/winui-desktop-unit-tests/ and immediately ran into problems trying to use UITestMethod with async calls (e.g. BitmapImage.SetSourceAsync) and went down a rabbit hole using TaskCompletionSource and dispatching the bulk of the test to the UI thread, and then tried to address issues with waiting for XAML to render (because, in WinUI, you can't get at the bits in a BitmapImage, but you can render it into the Xaml tree and then use a RenderTargetBitmap to get the rendered pixels). What a joke.

I'd really prefer to use Xunit for my tests (as I've done for non-UI code -- though this is only UI code by necessity as described above).

So...
This led me to modify this: https://github.com/AArnott/Xunit.StaFact/blob/main/test/Xunit.StaFact.Tests/WindowsDesktop/WpfFactTests.cs to use the WinUI synchronization context to try to get it to work on a WinUI project, like so:

using DesktopSyncContext = Microsoft.UI.Dispatching.DispatcherQueueSynchronizationContext;

I also had to remove the "ShouldShowWindow" test case because that's not how WinUI does what that test is trying to assert. With only those two changes, it fails to run. I get no errors, but see these warnings:

Starting test discovery for requested test run
========== Starting test discovery ==========
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.5+1caef2f33e (64-bit .NET 7.0.5)
No test is available in WinUI3TestApp1.exe. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
========== Test discovery finished: 0 Tests found in 4 sec ==========
========== Starting test run ==========
========== Test run finished: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in < 1 ms ==========

The no test available message is odd, given that all the tests show up in the test explorer.

If I set a breakpoint and debug the first test, it's never hit. To reiterate, this is not my code, but code from the Xunit.StaFact repository with the two changes noted above. I'm at a loss as to how to get this to work.

Is this supported? Has anyone been able to do WinUI/WindowsAppSdk 1.3/.Net 6+ UI testing with Xunit, with or without Xuint.StaFact? If so, what are the steps? If it's not supported, what would it take to add that support. I'm probably willing to participate in doing the work if it comes to that.

Thanks in advance.

@AArnott
Copy link
Owner

AArnott commented May 26, 2023

Thanks for the write-up. [UIFact] is a framework-agnostic attribute that just emulates common things like a generic SyncContext to keep test execution on a single thread.
It sounds like WinUI has its own specialized SyncContext, which is similar to WinForms and WPF and why they eachhave their own attributes. Cocoa has its own fact too. If you can help us understand the specific requirements for emulating a WinUI main thread, adding support for that sounds good.

As for the issues you ran into, that sounds like it might have just been an issue with how you patched and used the modified DLL.

I'm happy to take a look as well. Where did you get the DispatcherQueueSynchronizationContext type? I don't see it available.

@ScottAtBarco
Copy link
Author

ScottAtBarco commented May 26, 2023 via email

@AArnott
Copy link
Owner

AArnott commented May 27, 2023

The Microsoft.WindowAppSDK package doesn't appear on nuget.org. I can't help make this work without the sync context type that you're talking about.

@ScottAtBarco
Copy link
Author

ScottAtBarco commented May 30, 2023 via email

@AArnott
Copy link
Owner

AArnott commented May 30, 2023

No. I don't know how ***@***.*** would help.

@ScottAtBarco
Copy link
Author

ScottAtBarco commented May 30, 2023 via email

@ScottAtBarco
Copy link
Author

ScottAtBarco commented May 30, 2023 via email

@AArnott
Copy link
Owner

AArnott commented May 30, 2023

Thanks. Ya your image attachments are not coming from your email to the github issue comment. But your link worked. Apparently the nuget package ID is slightly different than you originally quoted (a typo, no doubt). I should be able to look into this now. Probably sometime next week.

@AArnott
Copy link
Owner

AArnott commented May 31, 2023

So I gave this a shot. You can review it here.

This isn't as simple as it sounded. To create a DispatcherQueueSynchronizationContext, I need a DispatcherQueue as well, but these things aren't directly creatable it seems. How does one get one of these?

It sounds like you actually had a compilable thing that you expected to work, though it didn't. I'm finding it quite difficult to get something remotely close to working. So anything you can share from your original prototype would help expedite this.

@ScottAtBarco
Copy link
Author

ScottAtBarco commented May 31, 2023 via email

@ScottAtBarco
Copy link
Author

ScottAtBarco commented May 31, 2023 via email

@ScottAtBarco
Copy link
Author

ScottAtBarco commented Jun 2, 2023 via email

@AArnott
Copy link
Owner

AArnott commented Jun 5, 2023

Thanks for the blog link. I guess I have to create a Window object to get its DispatcherQueue property from it. I'm trying with that now.

the exception text in WaitForOperationCompletionAsync says WPF not WinUI.

Thanks. Fixed.

The exception itself is concerning. I will need to test async methods that must be run on the UI thread. The message says no “async void”. Does that mean “async Task” are ok? You have such tests in WpfFactTests.cs, so I’m guessing they are, but staring at the code in the diff tells me nothing.

Yes, async Task test methods are fine. It's the async void tests that don't work.

@AArnott
Copy link
Owner

AArnott commented Jun 5, 2023

I've made a lot of progress. But the blog post is 2+ years old, and I had to make some changes to get WinUI tests building the way it proposed. But they don't run even with UITestMethod attributes. And the blog outlines some tight-knit interactions between the test runner and test framework, and assume that the test runner will execute the .exe instead of loading the test assembly as a library into another process. I don't know if xunit can do this. This could require xunit changes as well.

Latest changes

@ScottAtBarco
Copy link
Author

ScottAtBarco commented Jun 5, 2023 via email

@AArnott
Copy link
Owner

AArnott commented Jun 5, 2023

Yes, I understand the concession may include running an .exe. But unless you know of a way to get xunit tests to execute within an .exe test project that launches itself instead of by way of the xunit or dotnet test test runners, I don't see how we can make this work with xunit.

@ScottAtBarco
Copy link
Author

ScottAtBarco commented Jun 5, 2023 via email

@AArnott
Copy link
Owner

AArnott commented Jun 9, 2023

I don't know how you would make this work at this point. And xunit.stafact as an xunit library that your test project loads isn't properly positioned to spin up xunit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants