v1.13

Release 1.13

This release 1.13 introduces a bunch of new features to assert the result-set returned by a query and a few bug fixes or improvements. This release is also supporting SSIS 2016.

This release is downloadable from there.

Version: 1.13.0.18 aka v1.13

What's new ?

New features

• #192 Support of SSIS 2016 ... more info ...
• #133 Use 'row-count' with a condition and support % of rows validating the condition compared to full result-set. ... more info ...
• #148 & #59 Assert that all/no rows return by a query validate an expression (less-than, equal, more-than). A replacement, with additional features, for the deprecated evaluate-rows. ... more info ...
• #154 Improvement in the task to start PowerBI ... more info ...
• #166 Clean-cache is now also available for SSAS (MD and tabular) ... more info ...
• #163 Possibility to specify that you don't need to take into account an inherited default parameter for some queries ... more info ...
• #175 Read a connection-string from an ODC file
• #160 Specify a timeout for an individual query
• #156 Support stored procedures when using a report element

genbiL

• #145 Support definition of settings in an external file
• #194 Allow generation of tests when the list of cases is empty

Bug fixes

• #159 In some conditions the command groupcould result in a list of string in place of an array
• #142 If the test fails then the tasks in the cleanup element are not executed
• #173 Rounding is invalid for negative values
• #162 Tests of structure for relational models only work with SqlConnections (not with OleDb)
• #164 Existing packages are not found when using the Catalog
• #197 Tests for "schema existence" fail when schema_owner is not "dbo"
• #193 genbiL doesn't support two consecutive comments

What's next ?

Main ideas for release 1.14 (or next)

• Continue to support xml as a data source. The final goal is to be able to query xml files from reports (SSRS).
• Add support for testing that a result-set has no duplicated rows
• Support to test if rows have been added (or updated or deleted) in a table after the execution of a SQL statement or a ETL.

v1.13-beta-2

Release 1.13-beta

This release 1.13 introduces a bunch of new features to assert the result-set returned by a query and a few bug fixes or improvements.

This release is downloadable from there.

Version: 1.13.0.8 aka v1.13-beta-2

What's new ?

New features

• #133 Use 'row-count' with a condition and support % of rows validating the condition compared to full result-set.
• #148 & #59 Assert that all/no rows return by a query validate an expression (less-than, equal, more-than). A replacement, with additional features, for the deprecated evaluate-rows.
• #154 Improvement in the task to start PowerBI
• #166 Clean-cache is now also available for SSAS (MD and tabular)
• #163 Possibility to specify that you don't need to take into account an inherited default parameter for some queries
• #175 Read a connection-string from an ODC file
• #160 Specify a timeout for an individual query
• #156 Support stored procedures when using a report element
• #145 Support definition of settings in an external file (genbiL)

Bug fixes

• #159 In some conditions the command groupcould result in a list of string in place of an array
• #142 If the test fails then the tasks in the cleanup element are not executed
• #173 Rounding is invalid for negative values
• #162 Tests of structure for relational models only work with SqlConnections (not with OleDb)
• #164 Existing packages are not found when using the Catalog

What's next ?

Main ideas for release 1.14 (or next)

• Continue to support xml as a data source. The final goal is to be able to query xml files from reports (SSRS).
• Add support for testing that a result-set has no duplicated rows
• Support to test if rows have been added (or updated or deleted) in a table after the execution of a SQL statement or a ETL.

v1.13-beta

Release 1.13-beta

This release 1.13 introduces a bunch of new features to assert the result-set returned by a query and a few bug fixes or improvements.

This release is downloadable from there.

Version: 1.13.0.5 aka v1.13-beta

What's new ?

New features

• #133 Use 'row-count' with a condition and support % of rows validating the condition compared to full result-set.
• #148 & #59 Assert that all/no rows return by a query validate an expression (less-than, equal, more-than). A replacement, with additional features, for the deprecated evaluate-rows.
• #154 Improvement in the task to start PowerBI

Bug fixes

• #159 In some conditions the command groupcould result in a list of string in place of an array
• #142 If the test fails then the tasks in the cleanup element are not executed

What's next ?

Main ideas for release 1.13 (or next)

• Continue to support xml as a data source. The final goal is to be able to query xml files from reports (SSRS).
• Add support for testing that a result-set has no duplicated rows
• Support to test if rows have been added (or updated or deleted) in a table after the execution of a SQL statement or a ETL.

v1.12

Release 1.12

This release 1.12 introduces a set of bug fixes and performance improvements when comparing huge result-sets but the most notable feature is probably the compatibility with Power BI Desktop solutions.

This release is downloadable from there.

Version: 1.12.0.7 aka v1.12 (RTM)

What's new ?

New features

• #151 You can define Power BI Desktop solutions and run tests for structure, members or queries.
• #152 Two new tasks to run during setup and cleanup: exe-kill and wait
• #132 You can now specify the environment when defining an SSIS package
• #112 Support for testing parameters of a routine
• #143 The ordering of members can be define by a query

genbiL

• #138 The notion of group of rows for test-cases has been reviewed to support two new actions and let you create more than one group
• #139 new action to split a column into several new columns
• #141 new action to split a column in order to create groups
• #140 Review some weird or not consistent syntax in genbiL and in predefined templates

Bug fixes

• #113 Undetermined runtime of tests due to large non-matching datasets
• #46 Hashes collision with huge result-sets
• #137 Fix a bug in csv-profile if at least one of the separators was not specified
• #134 Command Exe-Run was not correctly dealing with relative path
• #115 Fix bug when creating two single comments in genbiL
• #136 Fix early check for existence of an external dependency
• #144 Fix bug with currency-format equal to k€ or (€)

What's next ?

Main ideas for release 1.13 (or next)

• Continue to support xml as a data source. The final goal is to be able to query xml files from reports (SSRS).
• Support assertions about the count of rows
  • added (or updated or deleted) in a table after the execution of a SQL statement or a ETL.
  • validating a predicate (with a percentage)
• Support for testing the parameters of routine

v1.12-beta-2

Release 1.12-beta-2

This release 1.12 introduces a set of bug fixes and performance improvements when comparing huge result-sets.

This release is downloadable from there.

Version: 1.12.0.x aka v1.12-beta-2

What's new ?

New features

• #151 You can define Power BI Desktop solutions and run tests for structure, members or queries.
• #152 Two new tasks to run during setup and cleanup: exe-kill and wait
• #132 You can now specify the environment when defining an SSIS package
• #112 Support for testing parameters of a routine
• #143 The ordering of members can be define by a query

genbiL

• #138 The notion of group of rows for test-cases has been reviewed to support two new actions and let you create more than one group
• #139 new action to split a column into several new columns
• #141 new action to split a column in order to create groups
• #140 Review some weird or not consistent syntax in genbiL and in predefined templates

Bug fixes

• #113 Undetermined runtime of tests due to large non-matching datasets
• #46 Hashes collision with huge result-sets
• #137 Fix a bug in csv-profile if at least one of the separators was not specified
• #134 Command Exe-Run was not correctly dealing with relative path
• #115 Fix bug when creating two single comments in genbiL
• #136 Fix early check for existence of an external dependency
• #144 Fix bug with currency-format equal to k€ or (€)

What's next ?

Main ideas for release 1.12 (or next)

• Continue to support xml as a data source. The final goal is to be able to query xml files from reports (SSRS).
• Support assertions about the count of rows
  • added (or updated or deleted) in a table after the execution of a SQL statement or a ETL.
  • validating a predicate (with a percentage)
• Support for testing the parameters of routine

v1.11

Release 1.11

This release 1.11 introduces a set of new features and a bunch of bug fixes. The good news of this release are that we've a new contributor who successfully fixed three bugs and that I've started to work on xml as a data source.

This release is downloadable from there.

Version: 1.11.0.14 aka v1.11

What's new ?

New features

Framework

• Load result-sets from xml files More info
• You can test the data-type of a column in a relational model More info
• Support of SSIS 2008R2 and 2012 for ETL testing More info
• Add routines (stored procedures, table-valued-functions and scalar functions) to the testable objects of a relational model. More info
• Register database providers in the config file to improve support of non-Microsoft data sources More info
• One-sided tolerance More info

genbiL

• Escape XML predefined entities for attributes More info
• Support "suite include" with a test-suite and not only a test More info

Bug fixes

Framework

• #98 Failure profile defined in config file is not used
• #116 Impossible to record null values in a CSV file
• #114 Display-folder not correctly taken into account
• #111 Use same relative path for SettingsFilename as TestSuiteFilename
• #110 Unable to reference connection strings in setup and cleanup tasks
• #129 In assertion 'row-count', the xsd doesn't accept the value 0 for the option 'more-than'

genbi

• #100 Microsoft.AnalysisServices.AdomdClient missing in folder Genbi
• #118 Running macros on Genbi for the second time rises an error
• #117 Genbi generates invalid element 'default' in 'row-count' and 'equalTo' assertions when tests are preexisting

What's next ?

Main ideas for release 1.12 (or next)

• Improve performance for comparison of large datasets and fix bug about invalid duplicated rows
• Continue to support xml as a data source. The final goal is to be able to query xml files from reports (SSRS).
• Support assertions about the count of rows added (or updated or deleted) in a table after the execution of a SQL statement or a ETL.
• Support for testing the parameters of routine

Other ideas probably not for 1.12 but for later

• Create fakes or stubs of views, stored procedures or functions during the setup of a test and remove them during the cleanup. Work already started
• Support to surround the test with a transaction and rollback the transaction when the test has been executed.
• Execute tests with etls running on a remote server.
• Add a setup command to process parts of a cube (SSAS).
• Support some tests about result-sets, to express easily conditions such has "If this field is greater than X then this other field must be greater than Y".
• Support transformation of the result-set in the assert (move columns, remove columns, add columns).

I'm (still) working on a Release 2.0. This work is delayed to the lack of GUI for NUnit 3.0. Anyway the goals are still:

• Lot of internal refactoring meaning easier to integrate new features (mostly done)
• Compatible with NUnit 3.0 meaning easier to setup a test suite !
• Easier to port it to another test framework than NUnit (xUnit, Fixie, ...)
• Easier to write helpers to compare result-sets
• Easier to create a Visual Studio Add-in

Naturally this Release 2.0 has a big constraint linked to the availability of NUnit 3.0 (currently still not available) so no due date for the moment.

v1.12-beta

Release 1.12-beta

This release 1.12 introduces a set of bug fixes and performance improvements when comparing huge result-sets.

This release is downloadable from there.

Version: 1.12.0.1 aka v1.12-beta

What's new ?

New features

• #113 Undetermined runtime of tests due to large non-matching datasets More Info
• #46 Hashes collision with huge result-sets More Info

What's next ?

Main ideas for release 1.12 (or next)

• Continue to support xml as a data source. The final goal is to be able to query xml files from reports (SSRS).
• Support assertions about the count of rows added (or updated or deleted) in a table after the execution of a SQL statement or a ETL.
• Support for testing the parameters of routine

v1.11-Beta

Release 1.11

This release 1.11 introduces a set of new features and a bunch of bug fixes. The good news of this release are that we've a new contributor who successfully fixed three bugs and that I've started to work on xml as a data source.

This release is downloadable from there.

Version: 1.11.0.11 aka v1.11-BETA

What's new ?

New features

• Load result-sets from xml files More info
• You can test the data-type of a column in a relational model More info
• Support of SSIS 2008R2 and 2012 for ETL testing More info
• Add routines (stored procedures, table-valued-functions and scalar functions) to the testable objects of a relational model. More info
• Register database providers in the config file to improve support of non-Microsoft data sources More info
• One-sided tolerance More info

Bug fixes

Framework

• #98 Failure profile defined in config file is not used
• #116 Impossible to record null values in a CSV file
• #114 Display-folder not correctly taken into account
• #111 Use same relative path for SettingsFilename as TestSuiteFilename
• #110 Unable to reference connection strings in setup and cleanup tasks

genbi

• #100 Microsoft.AnalysisServices.AdomdClient missing in folder Genbi

What's next ?

Promote this release from status Beta to Stable

Main ideas for release 1.12 (or next)

• Continue to support xml as a data source. The final goal is to be able to query xml files from reports (SSRS).
• Support assertions about the count of rows added (or updated or deleted) in a table after the execution of a SQL statement or a ETL.
• Support for testing the parameters of routine

Other ideas probably not for 1.12 but for later

• Create fakes or stubs of views, stored procedures or functions during the setup of a test and remove them during the cleanup. Work already started
• Support to surround the test with a transaction and rollback the transaction when the test has been executed.
• Execute tests with etls running on a remote server.
• Add a setup command to process parts of a cube (SSAS).
• Support some tests about result-sets, to express easily conditions such has "If this field is greater than X then this other field must be greater than Y".
• Support transformation of the result-set in the assert (move columns, remove columns, add columns).

I'm (still) working on a Release 2.0. This work is delayed to the lack of GUI for NUnit 3.0. Anyway the goals are still:

• Lot of internal refactoring meaning easier to integrate new features (mostly done)
• Compatible with NUnit 3.0 meaning easier to setup a test suite !
• Easier to port it to another test framework than NUnit (xUnit, Fixie, ...)
• Easier to write helpers to compare result-sets
• Easier to create a Visual Studio Add-in

Naturally this Release 2.0 has a big constraint linked to the availability of NUnit 3.0 (currently still not available) so no planned date for the moment.

v1.10

Release 1.10

This release 1.10 introduces a few new features and fixes a list of bugs. But the more important change of this version is, probably, that we've moved from Codeplex to Github. It's also the opportunityto reveal our new website and url at www.nbi.io This move is important because the documentation needed to be partially rewritten and the software management system has been changed from Mercurial to Git ... more details here under.

This release and the next will be downloadable from there.

What's new ?

New features

• Support the testing of structure for relational database: you can use the assertions exists, contain, equivalent and subsetOf on objects from a relational model including table, views, schemas (More to come in next releases). More info
• Support the testing of sets for the structure of tabular and multidimensional models. More info
• Support the testing of members of sets for tabular and multidimensional models. More info
• Support of a default type (text, numeric, boolean, dateTime) for columns identified as value. More info
• Support of keys with a different type (text, numeric, boolean, dateTime) than text. More info
• Tolerance is now supporting a combination of a percentage and a minimal/maximal absolute value. More info
• Improved performances when loading a test-suite with NUnit GUI.
• Improved user-experience when creating a test-suite with ETL. If some attributes are missing in the definition of the etl, NBi will clearly report the missing attributes.
• Additional partial xsd are also available for the xml elements test, settings, setup and cleanup. See in folder framework, sub-folder schema.

Bug fixes

Framework

• #86: Tolerance and negative values When a value was negative the tolerance was not correctly calculated.
• #77: connectionString is not forwarded to some asserts
• #79: SSIS DTS Variables do not work with dates
• #84: Only "SubsetOf" (and not "Contain" or "EquivalentTo") supports comparaison of members based on another instance
• #91: In some cases when xml syntax is invalid, NBi returns an object reference not set

genbi

• #85: xml attribute timeout for element etl not correctly managed by genbi

Major refactoring

• The code supporting the assertions of structure (model) has been refactored to improve maintainability and speed up future development.

Move to github

Why do we leave Codeplex

In a few words: no new feature and no bug fix. To illustrate this, the tab "Source Code" is suffering from a bug since more than six months. Github is really further than codeplex and the gap between the two tools is growing everyday.

Consequences of this move

• The website is hosted http://www.nbi.io
• The downloads section has been moved to GitHub
• This repository (for source code) is now at https://github.com/Seddryck/nbi, Bitbucket will be a mirror and codeplex will be discarded.
• The issue tracker will also be moved to GitHub at https://github.com/seddryck/nbi/issues
• The documentation is now hosted by GitHub at http://www.nbi.io/docs and http://www.nbi.io/automation/ and is built with Jekyll. The source for this documentation is available and editable at https://github.com/Seddryck/NBi/tree/gh-pages

What's next ?

Main ideas for release 1.11 (or next)

• Create fakes or stubs of views, stored procedures or functions during the setup of a test and remove them during the cleanup. Work already started
• Support tests on the xml of a report (SSRS). Work already started
• Support assertions about the count of rows added (or updated or deleted) in a table after the execution of a SQL statement or a ETL.
• Support for asserting the type of a column or attribute

Other ideas probably not for 1.11 but for later

• Support to surround the test with a transaction and rollback the transaction when the test has been executed.
• Execute tests with etls running on a remote server.
• Add a setup command to process parts of a cube (SSAS).
• Support some tests about result-sets, to express easily conditions such has "If this field is greater than X then this other field must be greater than Y".
• Support transformation of the result-set in the assert (move columns, remove columns, add columns).

I'm (still) working on a Release 2.0. This work is delayed to the lack of GUI for NUnit 3.0. Anyway the goals are still:

• Lot of internal refactoring meaning easier to integrate new features (mostly done)
• Compatible with NUnit 3.0 meaning easier to setup a test suite !
• Easier to port it to another test framework than NUnit (xUnit, Fixie, ...)
• Easier to write helpers to compare result-sets
• Easier to create a Visual Studio Add-in

Naturally this Release 2.0 has a big constraint linked to the availability of NUnit 3.0 (currently still not available) so no planned date for the moment.

v1.10-beta2

Release 1.10

This release 1.10 introduces a few new features and fixes a list of bugs. But the more important change of this version is, probably, that we've moved from Codeplex to Github. It's also the opportunityto reveal our new website and url at www.nbi.io This move is important because the documentation needed to be partially rewritten and the software management system has been changed from Mercurial to Git ... more details here under.

This release and the next will be downloadable from there.

What's new ?

New features

• Support the testing of structure for relational database: you can use the assertions exists, contain, equivalent and subsetOf on objects from a relational model including table, views, schemas (More to come in next releases). More info
• Support the testing of sets for the structure of tabular and multidimensional models. More info
• Support the testing of members of sets for tabular and multidimensional models. More info
• Support of a default type (text, numeric, boolean, dateTime) for columns identified as value. More info
• Support of keys with a different type (text, numeric, boolean, dateTime) than text. More info
• Tolerance is now supporting a combination of a percentage and a minimal/maximal absolute value. More info
• Improved performances when loading a test-suite with NUnit GUI.
• Improved user-experience when creating a test-suite with ETL. If some attributes are missing in the definition of the etl, NBi will clearly report the missing attributes.
• Additional partial xsd are also available for the xml elements test, settings, setup and cleanup. See in folder framework, sub-folder schema.

Bug fixes

Framework

• #86: Tolerance and negative values When a value was negative the tolerance was not correctly calculated.
• #77: connectionString is not forwarded to some asserts
• #79: SSIS DTS Variables do not work with dates
• #84: Only "SubsetOf" (and not "Contain" or "EquivalentTo") supports comparaison of members based on another instance
• #91: In some cases when xml syntax is invalid, NBi returns an object reference not set

genbi

• #85: xml attribute timeout for element etl not correctly managed by genbi

Major refactoring

• The code supporting the assertions of structure (model) has been refactored to improve maintainability and speed up future development.

Move to github

Why do we leave Codeplex

In a few words: no new feature and no bug fix. To illustrate this, the tab "Source Code" is suffering from a bug since more than six months. Github is really further than codeplex and the gap between the two tools is growing everyday.

Consequences of this move

• The website is hosted http://www.nbi.io
• The downloads section has been moved to GitHub
• This repository (for source code) is now at https://github.com/Seddryck/nbi, Bitbucket will be a mirror and codeplex will be discarded.
• The issue tracker will also be moved to GitHub at https://github.com/seddryck/nbi/issues
• The documentation is now hosted by GitHub at http://www.nbi.io/docs and http://www.nbi.io/automation/ and is built with Jekyll. The source for this documentation is available and editable at https://github.com/Seddryck/NBi/tree/gh-pages

What's next ?

Main ideas for release 1.11 (or next)

• Create fakes or stubs of views, stored procedures or functions during the setup of a test and remove them during the cleanup. Work already started
• Support tests on the xml of a report (SSRS). Work already started
• Support assertions about the count of rows added (or updated or deleted) in a table after the execution of a SQL statement or a ETL.

Other ideas probably not for 1.11 but for later

• Support to surround the test with a transaction and rollback the transaction when the test has been executed.
• Execute tests with etls running on a remote server.
• Add a setup command to process parts of a cube (SSAS).
• Support some tests about result-sets, to express easily conditions such has "If this field is greater than X then this other field must be greater than Y".
• Support transformation of the result-set in the assert (move columns, remove columns, add columns).

I'm (still) working on a Release 2.0. This work is delayed to the lack of GUI for NUnit 3.0. Anyway the goals are still:

• Lot of internal refactoring meaning easier to integrate new features (mostly done)
• Compatible with NUnit 3.0 meaning easier to setup a test suite !
• Easier to port it to another test framework than NUnit (xUnit, Fixie, ...)
• Easier to write helpers to compare result-sets
• Easier to create a Visual Studio Add-in

Naturally this Release 2.0 has a big constraint linked to the availability of NUnit 3.0 (currently still not available) so no planned date for the moment.