Map List<T> operations to PostgreSQL arrays (old)

doc/mapping/array.md

@@ -7,29 +7,64 @@ PostgreSQL has the unique feature of supporting [*array data types*](https://www
 
 # Mapping arrays
 
-Simply define a regular .NET array or `List<>` property, and the provider
+Npgsql maps PostgreSQL arrays to generic `T[]` and `List<T>` types:
 
 ```c#
 public class Post
 {
     public int Id { get; set; }
-    public string Name { get; set; }
-    public string[] Tags { get; set; }
-    public List<string> AlternativeTags { get; set; }
+    public string[] SomeArray { get; set; }
+    public List<string> SomeList { get; set; }
 }
 ```
 
-The provider will create `text[]` columns for the above two properties, and will properly detect changes in them - if you load an array and change one of its elements, calling `SaveChanges()` will automatically update the row in the database accordingly.
+The provider will create `text[]` columns for the above two properties, and will properly detect changes in them—if you load an array and change one of its elements, calling `SaveChanges()` will automatically update the row in the database accordingly.
 
 # Operation translation
 
-The provider can also translate CLR array operations to the corresponding SQL operation; this allows you to efficiently work with arrays by evaluating operations in the database and avoids pulling all the data. The following table lists the range operations that currently get translated. If you run into a missing operation, please open an issue.
+The provider translates many operations on `T[]` and `List<T>` to corresponding SQL operations. This allows arrays to be worked with efficiently by evaluating operations in the database. 
 
-Note that operation translation on `List<>` is limited at this time, but will be improved in the future. It's recommended to use an array for now.
+The following table lists the operations that are currently translated. If you run into a missing operation, please open an issue.
 
-| C# expression                                              | SQL generated by Npgsql |
-|------------------------------------------------------------|-------------------------|
-| `.Where(c => c.SomeArray[1] = "foo")`                      | [`WHERE "c"."SomeArray"[1] = 'foo'`](https://www.postgresql.org/docs/current/static/arrays.html#ARRAYS-ACCESSING)
-| `.Where(c => c.SomeArray.SequenceEqual(new[] { 1, 2, 3 })` | [`WHERE "c"."SomeArray" = ARRAY[1, 2, 3])`](https://www.postgresql.org/docs/current/static/arrays.html)
-| `.Where(c => c.SomeArray.Contains(3))`                     | [`WHERE 3 = ANY("c"."SomeArray")`](https://www.postgresql.org/docs/current/static/functions-comparisons.html#AEN21104)
-| `.Where(c => c.SomeArray.Length == 3)`                     | [`WHERE array_length("c"."SomeArray, 1) = 3`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+| C# expression                                                    | SQL generated by Npgsql |
+|------------------------------------------------------------------|-------------------------|
+| `.Where(c => c.SomeArray[0] == "foo")`                           | [`WHERE "c"."SomeArray"[1] = 'foo'`](https://www.postgresql.org/docs/current/static/arrays.html#ARRAYS-ACCESSING)
+| `.Where(c => c.SomeList[0] == "foo")`                            | [`WHERE "c"."SomeList"[1] = 'foo'`](https://www.postgresql.org/docs/current/static/arrays.html#ARRAYS-ACCESSING)
+| `.Where(c => c.SomeArray.ElementAt(0) == "foo")`                 | [`WHERE "c"."SomeArray"[1] = 'foo'`](https://www.postgresql.org/docs/current/static/arrays.html#ARRAYS-ACCESSING)
+| `.Where(c => c.SomeList.ElementAt(0) == "foo")`                  | [`WHERE "c"."SomeList"[1] = 'foo'`](https://www.postgresql.org/docs/current/static/arrays.html#ARRAYS-ACCESSING)
+| `.Where(x => x.SomeArray.Length == 1)`                           | [`WHERE array_length(x."SomeArray", 1) = 1`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+| `.Where(x => x.SomeList.Count == 1)`                             | [`WHERE array_length(x."SomeList", 1) = 1`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+| `.Where(x => x.SomeArray.Count() == 1)`                          | [`WHERE array_length(x."SomeArray", 1) = 1`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+| `.Where(x => x.SomeList.Count() == 1)`                           | [`WHERE array_length(x."SomeList", 1) = 1`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+| `.Where(x => x.SomeArray == x.SomeList)`                         | [`WHERE x."SomeArray" = x."SomeList"`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+| `.Where(x => x.SomeArray.Equals(x.SomeList))`                    | [`WHERE x."SomeArray" = x."SomeList"`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+| `.Where(x => x.SomeArray.SequenceEquals(x.SomeList))`            | [`WHERE x."SomeArray" = x."SomeList"`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+| `.Where(x => x.SomeArray.Contains("foo"))`                       | [`WHERE 'foo' = ANY (x."SomeArray")`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+| `.Where(x => x.SomeList.Contains("foo"))`                        | [`WHERE 'foo' = ANY (x."SomeList")`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+| `.Select(x => x.SomeArray.Append("foo"))`                        | [`SELECT x."SomeArray" \|\| 'foo'`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+| `.Select(x => x.SomeList.Append("foo"))`                         | [`SELECT x."SomeList" \|\| 'foo'`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+| `.Select(x => x.SomeArray.Prepend("foo"))`                       | [`SELECT 'foo' \|\| x."SomeArray"`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+| `.Select(x => x.SomeList.Prepend("foo"))`                        | [`SELECT 'foo' \|\| x."SomeList"`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+| `.Select(x => x.SomeArray.Concat(x.SomeList))`                   | [`SELECT x."SomeArray" \|\| x."SomeList"`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+| `.Select(x => x.SomeList.Concat(x.SomeArray))`                   | [`SELECT x."SomeList" \|\| x."SomeArray"`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+| `.Select(x => EF.Functions.ArrayToString(x.SomeArray, ","))`     | [`SELECT array_to_string(x."SomeArray", ',')`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+| `.Select(x => EF.Functions.ArrayToString(x.SomeList, ",", "*"))` | [`SELECT array_to_string(x."SomeList", ',', '*')`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+| `.Select(x => Array.IndexOf(x.SomeArray, "foo"))`                | [`SELECT COALESCE(array_position(x."SomeArray", 'foo'), -1)`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+| `.Select(x => x.SomeList.IndexOf("foo"))`                        | [`SELECT COALESCE(array_position(x."SomeList", 'foo'), -1)`](https://www.postgresql.org/docs/current/static/functions-array.html#ARRAY-FUNCTIONS-TABLE)
+
+# Pattern translation
+
+The provider has special translations for certain patterns of operations. These pattern-based translations are more susceptible to client-evaluation than standard translations. 
+
+The following sections describe the patterns that are currently translated. If you find that one of these patterns is being evaluated on the client, please open an issue.
+
+
+## LIKE ANY
+
+## LIKE ALL
+
+## EXISTS  
+
+## @>
+
+## &&

src/EFCore.PG/Extensions/NpgsqlArrayExtensions.cs

@@ -0,0 +1,154 @@
+#region License
+
+// The PostgreSQL License
+//
+// Copyright (C) 2016 The Npgsql Development Team
+//
+// Permission to use, copy, modify, and distribute this software and its
+// documentation for any purpose, without fee, and without a written
+// agreement is hereby granted, provided that the above copyright notice
+// and this paragraph and the following two paragraphs appear in all copies.
+//
+// IN NO EVENT SHALL THE NPGSQL DEVELOPMENT TEAM BE LIABLE TO ANY PARTY
+// FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES,
+// INCLUDING LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS
+// DOCUMENTATION, EVEN IF THE NPGSQL DEVELOPMENT TEAM HAS BEEN ADVISED OF
+// THE POSSIBILITY OF SUCH DAMAGE.
+//
+// THE NPGSQL DEVELOPMENT TEAM SPECIFICALLY DISCLAIMS ANY WARRANTIES,
+// INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
+// AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS
+// ON AN "AS IS" BASIS, AND THE NPGSQL DEVELOPMENT TEAM HAS NO OBLIGATIONS
+// TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
+
+#endregion
+
+using System;
+using System.Collections.Generic;
+using System.Runtime.CompilerServices;
+using JetBrains.Annotations;
+
+// ReSharper disable once CheckNamespace
+namespace Microsoft.EntityFrameworkCore
+{
+    /// <summary>
+    /// Provides extension methods for <see cref="List{T}"/> supporting PostgreSQL translation.
+    /// </summary>
+    public static class NpgsqlArrayExtensions
+    {
+        /// <summary>
+        /// Concatenates elements using the supplied delimiter.
+        /// </summary>
+        /// <param name="_">The DbFunctions instance.</param>
+        /// <param name="array">The list to convert to a string in which to locate the value.</param>
+        /// <param name="delimiter">The value used to delimit the elements.</param>
+        /// <typeparam name="T">The type of the elements of <paramref name="array"/>.</typeparam>
+        /// <returns>
+        /// The string concatenation of the elements with the supplied delimiter.
+        /// </returns>
+        /// <exception cref="ClientEvaluationNotSupportedException">
+        /// This method is only intended for use via SQL translation as part of an EF Core LINQ query.
+        /// </exception>
+        public static string ArrayToString<T>([CanBeNull] this DbFunctions _, [NotNull] T[] array, [CanBeNull] string delimiter)
+            => throw ClientEvaluationNotSupportedException();
+
+        /// <summary>
+        /// Concatenates elements using the supplied delimiter.
+        /// </summary>
+        /// <param name="_">The DbFunctions instance.</param>
+        /// <param name="list">The list to convert to a string in which to locate the value.</param>
+        /// <param name="delimiter">The value used to delimit the elements.</param>
+        /// <typeparam name="T">The type of the elements of <paramref name="list"/>.</typeparam>
+        /// <returns>
+        /// The string concatenation of the elements with the supplied delimiter.
+        /// </returns>
+        /// <exception cref="ClientEvaluationNotSupportedException">
+        /// This method is only intended for use via SQL translation as part of an EF Core LINQ query.
+        /// </exception>
+        public static string ArrayToString<T>([CanBeNull] this DbFunctions _, [NotNull] List<T> list, [CanBeNull] string delimiter)
+            => throw ClientEvaluationNotSupportedException();
+
+        /// <summary>
+        /// Concatenates elements using the supplied delimiter and the string representation for null elements.
+        /// </summary>
+        /// <param name="_">The DbFunctions instance.</param>
+        /// <param name="array">The list to convert to a string in which to locate the value.</param>
+        /// <param name="delimiter">The value used to delimit the elements.</param>
+        /// <param name="nullString">The value used to represent a null value.</param>
+        /// <typeparam name="T">The type of the elements of <paramref name="array"/>.</typeparam>
+        /// <returns>
+        /// The string concatenation of the elements with the supplied delimiter and null string.
+        /// </returns>
+        /// <exception cref="ClientEvaluationNotSupportedException">
+        /// This method is only intended for use via SQL translation as part of an EF Core LINQ query.
+        /// </exception>
+        public static string ArrayToString<T>([CanBeNull] this DbFunctions _, [NotNull] T[] array, [CanBeNull] string delimiter, [CanBeNull] string nullString)
+            => throw ClientEvaluationNotSupportedException();
+
+        /// <summary>
+        /// Concatenates elements using the supplied delimiter and the string representation for null elements.
+        /// </summary>
+        /// <param name="_">The DbFunctions instance.</param>
+        /// <param name="list">The list to convert to a string in which to locate the value.</param>
+        /// <param name="delimiter">The value used to delimit the elements.</param>
+        /// <param name="nullString">The value used to represent a null value.</param>
+        /// <typeparam name="T">The type of the elements of <paramref name="list"/>.</typeparam>
+        /// <returns>
+        /// The string concatenation of the elements with the supplied delimiter and null string.
+        /// </returns>
+        /// <exception cref="ClientEvaluationNotSupportedException">
+        /// This method is only intended for use via SQL translation as part of an EF Core LINQ query.
+        /// </exception>
+        public static string ArrayToString<T>([CanBeNull] this DbFunctions _, [NotNull] List<T> list, [CanBeNull] string delimiter, [CanBeNull] string nullString)
+            => throw ClientEvaluationNotSupportedException();
+
+        /// <summary>
+        /// Converts the input string into an array using the supplied delimiter and the string representation for null elements.
+        /// </summary>
+        /// <param name="_">The DbFunctions instance.</param>
+        /// <param name="input">The input string of delimited values.</param>
+        /// <param name="delimiter">The value that delimits the elements.</param>
+        /// <param name="nullString">The value that represents a null value.</param>
+        /// <typeparam name="T">The type of the elements in the resulting array.</typeparam>
+        /// <returns>
+        /// The array resulting from splitting the input string based on the supplied delimiter and null string.
+        /// </returns>
+        /// <exception cref="ClientEvaluationNotSupportedException">
+        /// This method is only intended for use via SQL translation as part of an EF Core LINQ query.
+        /// </exception>
+        public static T[] StringToArray<T>([CanBeNull] this DbFunctions _, [NotNull] string input, [CanBeNull] string delimiter, [CanBeNull] string nullString)
+            => throw ClientEvaluationNotSupportedException();
+
+        /// <summary>
+        /// Converts the input string into a <see cref="List{T}"/> using the supplied delimiter and the string representation for null elements.
+        /// </summary>
+        /// <param name="_">The DbFunctions instance.</param>
+        /// <param name="input">The input string of delimited values.</param>
+        /// <param name="delimiter">The value that delimits the elements.</param>
+        /// <param name="nullString">The value that represents a null value.</param>
+        /// <typeparam name="T">The type of the elements in the resulting array.</typeparam>
+        /// <returns>
+        /// The list resulting from splitting the input string based on the supplied delimiter and null string.
+        /// </returns>
+        /// <exception cref="ClientEvaluationNotSupportedException">
+        /// This method is only intended for use via SQL translation as part of an EF Core LINQ query.
+        /// </exception>
+        public static List<T> StringToList<T>([CanBeNull] this DbFunctions _, [NotNull] string input, [CanBeNull] string delimiter, [CanBeNull] string nullString)
+            => throw ClientEvaluationNotSupportedException();
+
+        #region Utilities
+
+        /// <summary>
+        /// Helper method to throw a <see cref="NotSupportedException"/> with the name of the throwing method.
+        /// </summary>
+        /// <param name="method">The method that throws the exception.</param>
+        /// <returns>
+        /// A <see cref="NotSupportedException"/>.
+        /// </returns>
+        [NotNull]
+        static NotSupportedException ClientEvaluationNotSupportedException([CallerMemberName] string method = default)
+            => new NotSupportedException($"{method} is only intended for use via SQL translation as part of an EF Core LINQ query.");
+
+        #endregion
+    }
+}

src/EFCore.PG/Extensions/NpgsqlServiceCollectionExtensions.cs

@@ -51,6 +51,7 @@
 // ReSharper disable once CheckNamespace
 namespace Microsoft.Extensions.DependencyInjection
 {
+    // ReSharper disable once UnusedMember.Global
     public static class NpgsqlEntityFrameworkServicesBuilderExtensions
     {
         /// <summary>
@@ -106,6 +107,8 @@ public static IServiceCollection AddEntityFrameworkNpgsql([NotNull] this IServic
                 .TryAdd<IQueryCompilationContextFactory, NpgsqlQueryCompilationContextFactory>()
                 .TryAdd<IMemberTranslator, NpgsqlCompositeMemberTranslator>()
                 .TryAdd<ICompositeMethodCallTranslator, NpgsqlCompositeMethodCallTranslator>()
+                .TryAdd<IExpressionFragmentTranslator, NpgsqlCompositeExpressionFragmentTranslator>()
+                .TryAdd<IQueryOptimizer, NpgsqlQueryOptimizer>()
                 .TryAdd<IQuerySqlGeneratorFactory, NpgsqlQuerySqlGeneratorFactory>()
                 .TryAdd<ISqlTranslatingExpressionVisitorFactory, NpgsqlSqlTranslatingExpressionVisitorFactory>()
                 .TryAdd<ISingletonOptions, INpgsqlOptions>(p => p.GetService<INpgsqlOptions>())