Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
We upgraded to the very bleeding edge of xUnit which has improved its internal message bus handling. In our case we were waiting in upwards of 10 seconds for the summary to be displayed on the console after the last test run. We now use a custom TestAssemblyRunner so that we are in complete control how tests are executed. This allows us to move away from xUnit [Collection]'s to initialize clusters in integration mode. Collections also form a concurrency limit which prevented most of our unit tests to run in parallel. Unit tests now run in parallel using xUnits default concurrency implementation. When running Integration tests we group the test classes based on which TClusterBase they implement as per inheriting from IClusterFixture<TClusterBase>. We initialize one ClusterBase per group and pass the same instance to all test classes. Then (after the cluster starts and bootstraps) we run the tests in parallel A cluster can override MaxParallism to limit concurrent execution, IntrusiveOperationCluster does this since it runs heavy es API's (reindex, forcemerge and the likes). Another huge performance gain is that tests now share clients where they can instead of new'ing them each time with new connectionsettings. This totally violated our own documented best practices! Improved console out in integration mode, we only elasticsearch.bat std/err up till we get a started confirmation. All the .IsValid.Should().BeTrue() are now replaced with .ShouldBeValid() or .ShouldNotBeValid() which dumps DebugInformation in the exception. At the end of the integration run each Cluster's total test time is printed in a short summary, so we know what to focus on if we want to make another big dent in our integration running times. Conflicts: paket.lock src/Elasticsearch.Net/Transport/Pipeline/RequestData.cs src/Profiling/ProfilingCluster.cs src/Profiling/Program.cs src/Tests/Aggregations/AggregationUsageTestBase.cs src/Tests/Aggregations/Metric/GeoCentroid/GeoCentroidAggregationUsageTests.cs src/Tests/Analysis/AnalysisCrudTests.cs src/Tests/Cat/CatAliases/CatAliasesApiTests.cs src/Tests/Cat/CatAllocation/CatAllocationApiTests.cs src/Tests/Cat/CatCount/CatCountApiTests.cs src/Tests/Cat/CatFielddata/CatFielddataApiTests.cs src/Tests/Cat/CatHealth/CatHealthApiTests.cs src/Tests/Cat/CatHelp/CatHelpApiTests.cs src/Tests/Cat/CatIndices/CatIndicesApiTests.cs src/Tests/Cat/CatMaster/CatMasterApiTests.cs src/Tests/Cat/CatNodeAttributes/CatNodeAttributesApiTests.cs src/Tests/Cat/CatNodes/CatNodesApiTests.cs src/Tests/Cat/CatPendingTasks/CatPendingTasksApiTests.cs src/Tests/Cat/CatPlugins/CatPluginsApiTests.cs src/Tests/Cat/CatRecovery/CatRecoveryApiTests.cs src/Tests/Cat/CatRepositories/CatRepositoriesApiTests.cs src/Tests/Cat/CatSegments/CatSegmentsApiTests.cs src/Tests/Cat/CatShards/CatShardsApiTests.cs src/Tests/Cat/CatSnapshots/CatSnapshotsApiTests.cs src/Tests/Cat/CatThreadPool/CatThreadpoolApiTests.cs src/Tests/ClientConcepts/Connection/HttpConnectionTests.cs src/Tests/ClientConcepts/ConnectionPooling/BuildingBlocks/RequestPipelines.doc.cs src/Tests/ClientConcepts/ConnectionPooling/Sniffing/RoleDetection.doc.cs src/Tests/ClientConcepts/Exceptions/ExceptionTests.cs src/Tests/ClientConcepts/HighLevel/Inference/IndexNameInference.doc.cs src/Tests/ClientConcepts/HighLevel/Inference/IndicesPaths.doc.cs src/Tests/ClientConcepts/HighLevel/Inference/PropertyInference.doc.cs src/Tests/ClientConcepts/LowLevel/Connecting.doc.cs src/Tests/Cluster/ClusterHealth/ClusterHealthApiTests.cs src/Tests/Cluster/ClusterPendingTasks/ClusterPendingTasksApiTests.cs src/Tests/Cluster/ClusterReroute/ClusterRerouteApiTests.cs src/Tests/Cluster/ClusterSettings/ClusterGetSettings/ClusterGetSettingsApiTests.cs src/Tests/Cluster/ClusterSettings/ClusterPutSettings/ClusterPutSettingsApiTests.cs src/Tests/Cluster/ClusterState/ClusterStatsApiTests.cs src/Tests/Cluster/ClusterStats/ClusterStatsApiTests.cs src/Tests/Cluster/NodesHotThreads/NodesHotThreadsApiTests.cs src/Tests/Cluster/NodesInfo/NodesInfoApiTests.cs src/Tests/Cluster/NodesStats/NodesStatsApiTests.cs src/Tests/Cluster/Ping/PingApiTests.cs src/Tests/Cluster/RootNodeInfo/RootNodeInfoApiTests.cs src/Tests/Cluster/TaskManagement/TasksCancel/TasksCancelApiTests.cs src/Tests/Cluster/TaskManagement/TasksList/TasksListApiTests.cs src/Tests/Document/Multiple/Bulk/BulkApiTests.cs src/Tests/Document/Multiple/Bulk/BulkInvalidApiTests.cs src/Tests/Document/Multiple/DeleteByQuery/DeleteByQueryApiTests.cs src/Tests/Document/Multiple/MultiGet/GetManyApiTests.cs src/Tests/Document/Multiple/MultiGet/MultiGetApiTests.cs src/Tests/Document/Multiple/MultiTermVectors/MultiTermVectorsApiTests.cs src/Tests/Document/Multiple/Reindex/ReindexApiTests.cs src/Tests/Document/Multiple/ReindexOnServer/ReindexOnServerApiTests.cs src/Tests/Document/Multiple/UpdateByQuery/UpdateByQueryApiTests.cs src/Tests/Document/Single/Attachment/AttachmentApiTests.cs src/Tests/Document/Single/Delete/DeleteApiTests.cs src/Tests/Document/Single/DocumentCrudTests.cs src/Tests/Document/Single/Exists/DocumentExistsApiTests.cs src/Tests/Document/Single/Get/GetApiTests.cs src/Tests/Document/Single/Index/IndexApiTests.cs src/Tests/Document/Single/Source/SourceApiTests.cs src/Tests/Document/Single/TermVectors/TermVectorsApiTests.cs src/Tests/Document/Single/Update/UpdateApiTests.cs src/Tests/Document/Single/Update/UpdateWithSourceApiTests.cs src/Tests/Framework/EndpointTests/ApiIntegrationTestBase.cs src/Tests/Framework/EndpointTests/ApiTestBase.cs src/Tests/Framework/EndpointTests/Clusters/IIntegrationCluster.cs src/Tests/Framework/EndpointTests/Clusters/OwnIndexCluster.cs src/Tests/Framework/EndpointTests/Clusters/ReadOnlyCluster.cs src/Tests/Framework/EndpointTests/CrudTestBase.cs src/Tests/Framework/EndpointTests/UrlTests.cs src/Tests/Framework/Integration/Bootstrappers/Seeder.cs src/Tests/Framework/Integration/Clusters/ClusterBase.cs src/Tests/Framework/Integration/Process/ElasticsearchNode.cs src/Tests/Framework/SerializationTests/Roundtrip.cs src/Tests/Framework/SerializationTests/SerializationTestBase.cs src/Tests/Framework/SimpleIntegration.cs src/Tests/Framework/TestClient.cs src/Tests/IndexModules/UsageTestBase.cs src/Tests/Indices/AliasManagement/Alias/AliasApiTests.cs src/Tests/Indices/AliasManagement/AliasExists/AliasExistsApiTests.cs src/Tests/Indices/AliasManagement/DeleteAlias/AliasDeleteApiTests.cs src/Tests/Indices/AliasManagement/GetAlias/GetAliasApiTests.cs src/Tests/Indices/AliasManagement/GetAliases/GetAliasesApiTests.cs src/Tests/Indices/AliasManagement/GetAliasesPointingToIndex/GetAliasesPointingToIndexTests.cs src/Tests/Indices/AliasManagement/GetIndicesPointingToAlias/GetIndicesPointingToAliasTests.cs src/Tests/Indices/AliasManagement/PutAlias/PutAliasApiTests.cs src/Tests/Indices/Analyze/AnalyzeApiTests.cs src/Tests/Indices/IndexManagement/CreateIndex/CreateIndexApiTests.cs src/Tests/Indices/IndexManagement/DeleteIndex/DeleteIndexApiTests.cs src/Tests/Indices/IndexManagement/GetIndex/GetIndexApiTests.cs src/Tests/Indices/IndexManagement/IndicesExists/IndexExistsApiTests.cs src/Tests/Indices/IndexManagement/OpenCloseIndex/CloseIndex/CloseIndexApiTests.cs src/Tests/Indices/IndexManagement/OpenCloseIndex/OpenIndex/OpenIndexApiTests.cs src/Tests/Indices/IndexManagement/TypesExists/TypeExistsApiTests.cs src/Tests/Indices/IndexSettings/GetIndexSettings/GetIndexSettingsApiTests.cs src/Tests/Indices/IndexSettings/IndexTemplates/DeleteIndexTemplate/DeleteIndexTemplateApiTests.cs src/Tests/Indices/IndexSettings/IndexTemplates/GetIndexTemplate/GetIndexTemplateApiTests.cs src/Tests/Indices/IndexSettings/IndexTemplates/IndexTemplateCrudTests.cs src/Tests/Indices/IndexSettings/IndexTemplates/IndexTemplateExists/IndexTemplateExistsApiTests.cs src/Tests/Indices/IndexSettings/IndexTemplates/PutIndexTemplate/PutIndexTemplateApiTests.cs src/Tests/Indices/IndexSettings/UpdateIndicesSettings/UpdateIndexSettingsApiTests.cs src/Tests/Indices/MappingManagement/GetFieldMapping/GetFieldMappingApiTests.cs src/Tests/Indices/MappingManagement/GetMapping/GetMappingApiTest.cs src/Tests/Indices/MappingManagement/PutMapping/PutMappingApiTest.cs src/Tests/Indices/Monitoring/IndicesRecovery/RecoveryStatusApiTests.cs src/Tests/Indices/Monitoring/IndicesSegments/SegmentsApiTests.cs src/Tests/Indices/Monitoring/IndicesShardStores/IndicesShardStoresApiTests.cs src/Tests/Indices/Monitoring/IndicesStats/IndicesStatsApiTests.cs src/Tests/Indices/StatusManagement/ClearCache/ClearCacheApiTests.cs src/Tests/Indices/StatusManagement/Flush/FlushApiTests.cs src/Tests/Indices/StatusManagement/ForceMerge/ForceMergeApiTests.cs src/Tests/Indices/StatusManagement/Optimize/OptimizeApiTests.cs src/Tests/Indices/StatusManagement/Refresh/RefreshApiTests.cs src/Tests/Indices/StatusManagement/SyncedFlush/SyncedFushApiTests.cs src/Tests/Indices/StatusManagement/Upgrade/UpgradeApiTests.cs src/Tests/Indices/Warmers/DeleteWarmer/DeleteWarmerApiTests.cs src/Tests/Indices/Warmers/GetWarmer/GetWarmerApiTests.cs src/Tests/Indices/Warmers/PutWarmer/PutWarmerApiTests.cs src/Tests/Indices/Warmers/WarmerCrudTests.cs src/Tests/Mapping/Metafields/MetafieldsMappingApiTestsBase.cs src/Tests/Modules/Scripting/DeleteScript/DeleteScriptApiTests.cs src/Tests/Modules/Scripting/GetScript/GetScriptApiTests.cs src/Tests/Modules/Scripting/PutScript/PutScriptApiTests.cs src/Tests/Modules/Scripting/ScriptingCrudTests.cs src/Tests/Modules/SnapshotAndRestore/Repositories/CreateRepository/CreateRepositoryApiTests.cs src/Tests/Modules/SnapshotAndRestore/Repositories/DeleteRepository/DeleteRepositoryApiTests.cs src/Tests/Modules/SnapshotAndRestore/Repositories/GetRepository/GetRepositoryApiTests.cs src/Tests/Modules/SnapshotAndRestore/Repositories/RepositoryCrudTests.cs src/Tests/Modules/SnapshotAndRestore/Repositories/VerifyRepository/VerifyRepositoryApiTests.cs src/Tests/Modules/SnapshotAndRestore/Restore/RestoreApiTests.cs src/Tests/Modules/SnapshotAndRestore/Snapshot/DeleteSnapshot/DeleteSnapshotApiTests.cs src/Tests/Modules/SnapshotAndRestore/Snapshot/GetSnapshot/GetSnapshotApiTests.cs src/Tests/Modules/SnapshotAndRestore/Snapshot/Snapshot/SnapshotApiTests.cs src/Tests/Modules/SnapshotAndRestore/Snapshot/SnapshotCrudTests.cs src/Tests/Modules/SnapshotAndRestore/Snapshot/SnapshotStatus/SnapshotStatusApiTests.cs src/Tests/QueryDsl/BoolDsl/BoolApiTests.cs src/Tests/QueryDsl/Geo/Shape/GeoShapeQueryUsageTestsBase.cs src/Tests/QueryDsl/QueryDslIntegrationTestsBase.cs src/Tests/QueryDsl/QueryDslUsageTestsBase.cs src/Tests/QueryDsl/Specialized/Script/ScriptQueryUsageTests.cs src/Tests/Reproduce/GithubIssue1863.cs src/Tests/Reproduce/GithubIssue2173.cs src/Tests/Search/Count/CountApiTests.cs src/Tests/Search/Explain/ExplainApiTests.cs src/Tests/Search/FieldStats/FieldStatsApiTests.cs src/Tests/Search/MultiSearch/MultiSearchApiTests.cs src/Tests/Search/MultiSearch/MultiSearchInvalidApiTests.cs src/Tests/Search/MultiSearch/MultiSearchLowLevelPostDataTests.cs src/Tests/Search/Percolator/MultiPercolate/MultiPercolateApiTests.cs src/Tests/Search/Percolator/MultiPercolate/MultiPercolateInvalidApiTests.cs src/Tests/Search/Percolator/Percolate/PercolateApiTests.cs src/Tests/Search/Percolator/PercolateCount/PercolateCountApiTests.cs src/Tests/Search/Percolator/RegisterPercolator/RegisterPercolatorApiTests.cs src/Tests/Search/Percolator/UnregisterPercolator/UnregisterPercolatorApiTests.cs src/Tests/Search/Request/InnerHitsUsageTests.cs src/Tests/Search/Scroll/ClearScroll/ClearScrollApiTests.cs src/Tests/Search/Scroll/Scroll/ScrollApiTests.cs src/Tests/Search/Search/Rescoring/RescoreUsageTests.cs src/Tests/Search/Search/SearchApiTests.cs src/Tests/Search/SearchExists/SearchExistsApiTests.cs src/Tests/Search/SearchShards/SearchShardsApiTests.cs src/Tests/Search/SearchTemplate/DeleteSearchTemplate/DeleteSearchTemplateApiTests.cs src/Tests/Search/SearchTemplate/GetSearchTemplate/GetSearchTemplateApiTests.cs src/Tests/Search/SearchTemplate/PutSearchTemplate/PutSearchTemplateApiTests.cs src/Tests/Search/SearchTemplate/RenderSearchTemplate/RenderSearchTemplateApiTests.cs src/Tests/Search/SearchTemplate/SearchTemplateCrudTests.cs src/Tests/Search/SearchUsageTestBase.cs src/Tests/Search/Suggesters/SuggestApiTests.cs src/Tests/Search/Validate/ValidateQueryApiTests.cs src/Tests/Tests.csproj src/Tests/XPack/Graph/Explore/GraphExploreApiTests.cs src/Tests/XPack/License/GetLicense/GetLicenseApiTests.cs src/Tests/XPack/License/PostLicense/PostLicenseApiTests.cs src/Tests/XPack/Security/Authenticate/AuthenticateApiTests.cs src/Tests/XPack/Security/ClearCachedRealms/ClearCachedRealmsApiTests.cs src/Tests/XPack/Security/Role/ClearCachedRoles/ClearCachedRolesApiTests.cs src/Tests/XPack/Security/Role/PutRole/PutRoleApiTests.cs src/Tests/XPack/Security/Role/RoleCrudTests.cs src/Tests/XPack/Security/ShieldCluster.cs src/Tests/XPack/Security/User/PutUser/PutUserApiTests.cs src/Tests/XPack/Security/User/UserCrudTests.cs src/Tests/project.json
- Loading branch information