-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WinUI Unit testing with UIFact doesn't appear to work. #77
Comments
Thanks for the write-up. As for the issues you ran into, that sounds like it might have just been an issue with how you patched and used the modified DLL. I'm happy to take a look as well. Where did you get the |
Hey, Andrew, thanks for the follow up.
The DispatcherQueueSynchronizationContext is in Microst.UI.Dispatching:
using DesktopSyncContext = Microsoft.UI.Dispatching.DispatcherQueueSynchronizationContext;
This in turn is (I believe) in the Microsoft.WindowAppSDK 1.3.x package.
I suspect what’s happening is that the WpfFactAttribute doesn’t know about this context (so UIFact probably doesn’t either), so those would need an update to support WinUI. Or a new WinUIFact attribute would need to be created. Maybe. I don’t know. As it is, as I state in the original, the tests aren’t even attempted with this change alone.
Hope that helps in tracking down the required changes.
|
The |
Does this help?
***@***.***
From: Andrew Arnott ***@***.***>
Sent: Saturday, May 27, 2023 6:25 AM
To: AArnott/Xunit.StaFact ***@***.***>
Cc: Bigger(EXT), Scott ***@***.***>; Author ***@***.***>
Subject: Re: [AArnott/Xunit.StaFact] WinUI Unit testing with UIFact doesn't appear to work. (Issue #77)
The Microsoft.WindowAppSDK package doesn't appear on nuget.org. I can't help make this work without the sync context type that you're talking about.
—
Reply to this email directly, view it on GitHub<#77 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A5LLZC7ZFXSWEAUBULRBSVDXIH6BPANCNFSM6AAAAAAYPJZK2U>.
You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>
This message is subject to the following terms and conditions: MAIL DISCLAIMER<https://www.barco.com/en/maildisclaimer>
|
No. I don't know how |
Hmm… I pasted an image in that email. I saved it out and attached it. Any better?
From: Andrew Arnott ***@***.***>
Sent: Tuesday, May 30, 2023 9:29 AM
To: AArnott/Xunit.StaFact ***@***.***>
Cc: Bigger(EXT), Scott ***@***.***>; Author ***@***.***>
Subject: Re: [AArnott/Xunit.StaFact] WinUI Unit testing with UIFact doesn't appear to work. (Issue #77)
No. I don't know how ***@***.***<mailto:***@***.***> would help.
—
Reply to this email directly, view it on GitHub<#77 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A5LLZC7JOAEJ7JNXSWGSEALXIYN3RANCNFSM6AAAAAAYPJZK2U>.
You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>
This message is subject to the following terms and conditions: MAIL DISCLAIMER<https://www.barco.com/en/maildisclaimer>
|
It’s on nuget.org here:
NuGet Gallery | Microsoft.WindowsAppSDK 1.3.230502000<https://www.nuget.org/packages/Microsoft.WindowsAppSDK/1.3.230502000>
From: Bigger(EXT), Scott
Sent: Tuesday, May 30, 2023 9:33 AM
To: AArnott/Xunit.StaFact ***@***.***>; AArnott/Xunit.StaFact ***@***.***>
Cc: Author ***@***.***>
Subject: RE: [AArnott/Xunit.StaFact] WinUI Unit testing with UIFact doesn't appear to work. (Issue #77)
Hmm… I pasted an image in that email. I saved it out and attached it. Any better?
From: Andrew Arnott ***@***.******@***.***>>
Sent: Tuesday, May 30, 2023 9:29 AM
To: AArnott/Xunit.StaFact ***@***.******@***.***>>
Cc: Bigger(EXT), Scott ***@***.******@***.***>>; Author ***@***.******@***.***>>
Subject: Re: [AArnott/Xunit.StaFact] WinUI Unit testing with UIFact doesn't appear to work. (Issue #77)
No. I don't know how ***@***.***<mailto:***@***.***> would help.
—
Reply to this email directly, view it on GitHub<#77 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A5LLZC7JOAEJ7JNXSWGSEALXIYN3RANCNFSM6AAAAAAYPJZK2U>.
You are receiving this because you authored the thread.Message ID: ***@***.******@***.***>>
This message is subject to the following terms and conditions: MAIL DISCLAIMER<https://www.barco.com/en/maildisclaimer>
|
Thanks. Ya your image attachments are not coming from your email to the github issue comment. But your link worked. Apparently the nuget package ID is slightly different than you originally quoted (a typo, no doubt). I should be able to look into this now. Probably sometime next week. |
So I gave this a shot. You can review it here. This isn't as simple as it sounded. To create a It sounds like you actually had a compilable thing that you expected to work, though it didn't. I'm finding it quite difficult to get something remotely close to working. So anything you can share from your original prototype would help expedite this. |
Attached is a sample VS2022 project (rename to have just .zip to extract).
If you’d prefer not to open random attachments, all I did was create a blank WinUI3 project (with the WindowsAppSDK template extensions). I have code that manipulates the UI that I’d like to test, so I created a true WinUI app, as described here: WinUI Desktop Unit Tests - #ifdef Windows (microsoft.com)<https://devblogs.microsoft.com/ifdef-windows/winui-desktop-unit-tests/>, While I didn’t add the MSTest scaffolding described in the blog, I do create a real app project to house my tests. There is an experimental extension with templates for creating WinUI Unit tests, but I have yet to try it. Would be nice if there was one for spinning up Xunit projects that were WinUi compatible, including all the required NuGets.
Once I created the project, I added the following NuGet packages (in addition to the two WindowsAppSDK packages that were already there when I created the base app):
Microsoft.TestPlatform.TestHost {17.6.0} WinUI3TestApp
MSTest.TestAdapter {3.0.3} WinUI3TestApp
xunit.runner.visualstudio {2.4.5} WinUI3TestApp
Xunit.StaFact {1.1.11} WinUI3TestApp
MSTest.TestFramework {3.0.3} WinUI3TestApp
xunit {2.4.2} WinUI3TestApp
Not 100% sure the MSTest packages are required.
I then copied the WpfFactTests.cs file from the StaFact git repo and added it to this project, along with the two changes mentioned previously, namely changing this line:
using DesktopSyncContext = Microsoft.UI.Dispatching.DispatcherQueueSynchronizationContext;
and commenting out the ShouldShowWindow test.
If I run the program, it shows a blank window as expected. If I open TestExplorer, I see 7 tests, but if I run them (or debug them) the tests are never run. The TestExplore shows a System.Runtime.InteropServices.COMException “The method or operation is not implemented”. It spins for a while and eventually exits without running any tests, though it does open the blank app window.
Hope that helps.
|
Two comments on the code changes you linked.
One’s trivial: the exception text in WaitForOperationCompletionAsync says WPF not WinUI.
The exception itself is concerning. I will need to test async methods that must be run on the UI thread. The message says no “async void”. Does that mean “async Task” are ok? You have such tests in WpfFactTests.cs, so I’m guessing they are, but staring at the code in the diff tells me nothing.
|
Something else you might find interesting. Morten Nielsen did this for MsTest (though I haven’t tried it so can’t assert that it works for async UI testing).
UI Test Tools Framework (dotmorten.github.io)<https://dotmorten.github.io/WinUIEx/concepts/UITests.html>
|
Thanks for the blog link. I guess I have to create a
Thanks. Fixed.
Yes, |
I've made a lot of progress. But the blog post is 2+ years old, and I had to make some changes to get WinUI tests building the way it proposed. But they don't run even with |
Happy to see you’re still banging away on this.
FWIW, that sample app I sent you does launch the app that holds the tests—you can see the window—but that’s possibly because I created the project as a WinUI app, just as the #IFDEF Windows blog post suggests for MsTest UI testing. That would be a reasonable compromise (if it’s a compromise at all), since UI tests are going to want to create some UI to test. How all this works behind the scenes isn’t something I’ve looked into in much detail, but it seems like it should work. “It’s just code.”™
|
Yes, I understand the concession may include running an .exe. But unless you know of a way to get xunit tests to execute within an .exe test project that launches itself instead of by way of the xunit or |
Is there a way to listen for window to open? Or is there standard boiler plate that test writer needs to put into App.xaml.cs or MainWindow.xaml.cs as suggested by the #IFDEF Windows blog? Will that give you the hook you need to spin up xunit after the test runner launches the app?
|
I don't know how you would make this work at this point. And xunit.stafact as an xunit library that your test project loads isn't properly positioned to spin up xunit. |
I created a WinUI project for testing some BitmapImage-related code using MSTest along the lines recommended here: https://devblogs.microsoft.com/ifdef-windows/winui-desktop-unit-tests/ and immediately ran into problems trying to use UITestMethod with async calls (e.g. BitmapImage.SetSourceAsync) and went down a rabbit hole using TaskCompletionSource and dispatching the bulk of the test to the UI thread, and then tried to address issues with waiting for XAML to render (because, in WinUI, you can't get at the bits in a BitmapImage, but you can render it into the Xaml tree and then use a RenderTargetBitmap to get the rendered pixels). What a joke.
I'd really prefer to use Xunit for my tests (as I've done for non-UI code -- though this is only UI code by necessity as described above).
So...
This led me to modify this: https://github.com/AArnott/Xunit.StaFact/blob/main/test/Xunit.StaFact.Tests/WindowsDesktop/WpfFactTests.cs to use the WinUI synchronization context to try to get it to work on a WinUI project, like so:
using DesktopSyncContext = Microsoft.UI.Dispatching.DispatcherQueueSynchronizationContext;
I also had to remove the "ShouldShowWindow" test case because that's not how WinUI does what that test is trying to assert. With only those two changes, it fails to run. I get no errors, but see these warnings:
Starting test discovery for requested test run
========== Starting test discovery ==========
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.5+1caef2f33e (64-bit .NET 7.0.5)
No test is available in WinUI3TestApp1.exe. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
========== Test discovery finished: 0 Tests found in 4 sec ==========
========== Starting test run ==========
========== Test run finished: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in < 1 ms ==========
The no test available message is odd, given that all the tests show up in the test explorer.
If I set a breakpoint and debug the first test, it's never hit. To reiterate, this is not my code, but code from the Xunit.StaFact repository with the two changes noted above. I'm at a loss as to how to get this to work.
Is this supported? Has anyone been able to do WinUI/WindowsAppSdk 1.3/.Net 6+ UI testing with Xunit, with or without Xuint.StaFact? If so, what are the steps? If it's not supported, what would it take to add that support. I'm probably willing to participate in doing the work if it comes to that.
Thanks in advance.
The text was updated successfully, but these errors were encountered: