Results for Simple.Data

[jovanpop-msft]

I have executed test on Windows 10 / SQL Server 2016 Express SP1 (local named instance - I have changed connection string to .\SQLEXPRESS), and I got very good results for Simple.Data that are not reported in benchmark. It seems that it is even faster than hand coded DAO, which is very surprising (but I don't see any error in code). Could you double-check results for Simple.Data and publish result?
My results are below:

.NET: 4.0.30319.42000
Dapper: Dapper.SqlMapper, Dapper, Version=1.50.2.0, Culture=neutral, PublicKeyToken=null
Using Connectionstring: Data Source=.\SQLEXPRESS;Initial Catalog=tempdb;Integrated Security=True
Loading native assemblies for SQL types...failed.
Error loading msvcr100.dll (ErrorCode: 126)
Running 500 iterations that load up a post entity

Running...
Simple.Data took 17ms
hand coded took 80ms
Mapper QueryFirstOrDefault took 83ms
DataTable via IDataReader.GetValues took 84ms
Dynamic Mapper QueryQueryFirstOrDefault took 85ms
Dynamic Mapper Query (buffered) took 86ms
Mapper Query (buffered) took 88ms
Dynamic Massive ORM Query took 94ms
Dapper.Contrib took 104ms
Dynamic Mapper Query (non-buffered) took 114ms
Mapper Query (non-buffered) took 118ms
Linq 2 SQL Compiled took 119ms
Entity framework SqlQuery took 152ms
Linq 2 SQL ExecuteQuery took 222ms
Entity framework took 280ms
Entity framework No Tracking took 301ms
Linq 2 SQL took 733ms
Press any key to continue . . .

[NickCraver]

Do you have a link for your tests? There are also errors in the output first thing, so I'm not sure what's happening there.

[jovanpop-msft]

What would be the best way to share results? I have just cloned repository, made some minimal changes, started test, and copied results from console.

• I have just cloned repository https://github.com/JocaPC/dapper-dot-net/
• Changed version in global.json because I have different version of dotnet (there was some compile error)
• Changed connection string in Dapper.Tests\Tests.cs

global.json

-    "version": "1.0.0-preview2-003121"
+    "version": "1.0.0-preview2-003156"

Dapper.Tests/Tests.cs

        public static string ConnectionString =>
            IsAppVeyor
-                ? @"Server=(local)\SQL2014;Database=tempdb;User ID=sa;Password=Password12!"
-                : "Data Source=.;Initial Catalog=tempdb;Integrated Security=True";
+               ? @"Server=(local)\SQLEXPRESS;Database=tempdb;User ID=sa;Password=QWER1234!"
+               : "Data Source=.\\SQLEXPRESS;Initial Catalog=tempdb;Integrated Security=True";

        public static string OleDbConnectionString =>
            IsAppVeyor
-                ? @"Provider=SQLOLEDB;Data Source=(local)\SQL2014;Initial Catalog=tempdb;User Id=sa;Password=Password12!"
-                : "Provider=SQLOLEDB;Data Source=.;Initial Catalog=tempdb;Integrated Security=SSPI";
+                ? @"Provider=SQLOLEDB;Data Source=(local)\SQLEXPRESS;Initial Catalog=tempdb;User Id=sa;Password=QWER1234!"
+                : "Provider=SQLOLEDB;Data Source=.\\SQLEXPRESS;Initial Catalog=tempdb;Integrated Security=SSPI";

System info:
LENOVO_MT_20A8_BU_Think_FM_ThinkPad X1 Carbon 2nd
Processor Intel(R) Core(TM) i7-4600U CPU @ 2.10GHz, 2694 Mhz, 2 Core(s), 4 Logical Processor(s)
select @@Version
Microsoft SQL Server 2016 (SP1) (KB3182545) - 13.0.4001.0 (X64) Oct 28 2016 18:17:30 Copyright (c) Microsoft Corporation Express Edition (64-bit) on Windows 10 Enterprise 6.3 (Build 14393: ) (Hypervisor)

[jovanpop-msft]

Error that is shown in console is not something important. It seems that it cannot load msvcr100.dll in LoadNativeAssembly method:

Error loading msvcr100.dll (ErrorCode: 126)

   at SqlServerTypes.Utilities.LoadNativeAssembly(String nativeBinaryPath, String assemblyName)
   at SqlServerTypes.Utilities.LoadNativeAssemblies(String rootApplicationPath)
   at Dapper.Tests.TestSuite..cctor()

I'm not sure is Simple.Data correctly used. Simple.Data.FindById() returns SimpleQuery instance. I'm not familiar with Simple.Data, but I'm not sure does this method actually do something or just returns something like IQueryable.
On the Simple.Data Gitub page, I see that they are using FirstOrDefault() after Find method. When I change this method and add FirstOrDefault(), performance of Simple.Data drop significantly. Here is my change:

#if SIMPLEDATA
                // Simple.Data
                Try(() =>
                {
                    var sdb = Simple.Data.Database.OpenConnection(TestSuite.ConnectionString);
                    tests.Add(id => sdb.Posts.FindById(id).FirstOrDefault(), "Simple.Data");
                }, "Simple.Data");
#endif

Results with added FirstOrDefault() in Simple.Data are:

.NET: 4.0.30319.42000
Dapper: Dapper.SqlMapper, Dapper, Version=1.50.2.0, Culture=neutral, PublicKeyToken=null
Using Connectionstring: Data Source=.\SQLEXPRESS;Initial Catalog=tempdb;Integrated Security=True
Loading native assemblies for SQL types...failed.
Error loading msvcr100.dll (ErrorCode: 126)
Running 500 iterations that load up a post entity

Running...
hand coded took 77ms
Dynamic Mapper QueryQueryFirstOrDefault took 82ms
Mapper Query (buffered) took 87ms
Mapper QueryFirstOrDefault took 87ms
Dynamic Massive ORM Query took 97ms
DataTable via IDataReader.GetValues took 98ms
Dapper.Contrib took 103ms
Dynamic Mapper Query (buffered) took 110ms
Dynamic Mapper Query (non-buffered) took 114ms
Linq 2 SQL Compiled took 116ms
Mapper Query (non-buffered) took 116ms
Entity framework SqlQuery took 150ms
Linq 2 SQL ExecuteQuery took 238ms
Entity framework took 283ms
Entity framework No Tracking took 286ms
Simple.Data took 428ms
Linq 2 SQL took 790ms
Press any key to continue . . .

Could you re-run the test in your environment with and without this change, and could someone who is more familiar with Simple.Data review that code?

[jovanpop-msft]

Any update on this issue? I believe that test is not correct, so could anyone confirm this? I can send PR to change it, but it would be nice to know is this actually an error.

[NickCraver]

@jovanpop-msft Sorry, I've been crazy on time latest - yes I agree that .First() is missing there, that's why the numbers are so surprisingly off.

Would you like to submit a PR to credit on finding/fixing here? We need to refactor all those tests into Benchmark.net (#666), but this definitely needs fixing regardless.

[jovanpop-msft]

Thanks @NickCraver, PR is submitted here: #723

[NickCraver]

Closing this out - fixed in #723!