modify base test class analyzer, allow to extend from ft base test class #4810
Conversation
| private readonly ConcurrentDictionary<INamedTypeSymbol, bool> _knownTestAttributes = new ConcurrentDictionary<INamedTypeSymbol, bool>(); | ||
|
|
||
| public AnalyzerImpl(Compilation compilation, INamedTypeSymbol factAttribute) | ||
| { | ||
| _compilation = compilation; | ||
| _factAttribute = factAttribute; | ||
| _baseTestClass = _compilation.GetTypeByMetadataName("Microsoft.ML.TestFramework.BaseTestClass"); | ||
| _FTbaseTestClass = _compilation.GetTypeByMetadataName("Microsoft.ML.Functional.Tests.FunctionalTestBaseClass"); |
There was a problem hiding this comment.
Why do we have two base test classes? I know you have investigated this before. Can you please help me understand why we can't have a single BaseTestClass and have FunctionalTestBaseClass derive from BaseTestClass? (From what I understand, the only difference for functional tests is that they only use public API)
There was a problem hiding this comment.
Yes, please reference previous PR for this: #4346
Basically, we want FT to be able to dependent on public nuget package so we need to break FT from any direct or indirect project reference. TestFramework project itself has lots of project dependency and also been marked as BestFriend from some projects from source folder which allows TestFramework peoject to access some non public code. So, we extracted TestFrameworkCommon test project which only have public dependency. Previously FTs are inherited from BaseTestBaseline which has lots of BestFriend thing and can't be moved to TestFrameworkCommon project so after discuss we decide to write one base class for FT only with only public dependency.
So in short we have separate base class for FT because we don't want FT to take project dependency from our source code thus let FT able to run in NightlyBuild
In reply to: 376581212 [](ancestors = 376581212)
There was a problem hiding this comment.
You are right! That sounds good. In that case, please make sure the functionality between the two base classes are in sync. I recently added some code to extract the LogMessageKind attribute from the test and store it in the base class. I also changed the derived class to append to the logs correctly. Can you please make sure that the important parts from this PR are integrated into the the FunctionalTestBaseClass: #4710
In reply to: 376616140 [](ancestors = 376616140,376581212)
There was a problem hiding this comment.
The above mentioned work need not be part of this PR. I am approving this. Please use a different PR for that work.
There was a problem hiding this comment.
I am approving this. You can do the above mentioned work in a different PR.
There was a problem hiding this comment.
Sure, I will open a separate PR if we need certain fix
In reply to: 376622761 [](ancestors = 376622761)
ghost
locked as resolved and limited conversation to collaborators
We have separate base test class for all functional tests: FunctionalTestBaseClass
The reason for that is FT is also runned on NightlyBuild pipeline which not take project dependency directly but take ML.NET related Nuget package as dependency and we don't want FT to take dependency from TestFramework.