fix(cli/file-import): improvements and config(s) hashes or files (#1056)

elasticms-admin/environment-variables.md

@@ -401,14 +401,26 @@ Define backend elasticms url. CommonBundle provides a CoreApi instance.
 
 Define backend authentication token. The commonBundle coreApi instance becomes authenticated.
 
-### EMS_BACKEND_API_TIMEOUT
+### EMS_CORE_API_HEADERS
 
-Adjust the API client's timeout. By default is set to `30` seconds, if you API request may take longueur (e.g. during migration) you can increase the timeout :
+Define extra headers for the API client, helpful for adding cookie header for xdebug.
 
 ```dotenv
-EMS_ELASTICSEARCH_HOSTS='300'
+EMS_CORE_API_HEADERS='{"Cookie":"XDEBUG_SESSION=PHPSTORM"}'
 ```
 
+### EMS_CORE_API_MAX_CONNECTIONS
+
+Define the max connections to the API client, when using async calls. By default it is set to `6`.
+
+### EMS_BACKEND_API_TIMEOUT
+
+Adjust the API client's timeout. By default is set to `30` seconds, if you API request may take longueur (e.g. during migration) you can increase the timeout :
+
+### EMS_BACKEND_API_VERIFY
+
+Define the `verify_host` and `verify_peer` for the api client, by default it is set to `true`.
+
 ### EMS_CACHE
 
 Define the ems cache type. Default value `file_system`.

elasticms-cli/commands.md

@@ -1,18 +1,22 @@
 # EMS (CommonBundle) Commands
 
 <!-- TOC -->
+
 * [EMS Commands](#ems-commonbundle-commands)
-  * [Admin](#admin)
-    * [Backup](#backup)
+    * [Admin](#admin)
+        * [Backup](#backup)
+
 <!-- TOC -->
 
 ### Admin
 
 #### Backup
 
-The command downloads the configuration (JSON files for content types, environments, ...) and documents (JSON files) for all managed content types.
+The command downloads the configuration (JSON files for content types, environments, ...) and documents (JSON files) for
+all managed content types.
 
-Be cautious that the document are downloaded from the elasticsearch's default indexes. So ensure that your elasticsearch's indexes are well synchronized. Only the last finalized revision will be archived.
+Be cautious that the document are downloaded from the elasticsearch's default indexes. So ensure that your
+elasticsearch's indexes are well synchronized. Only the last finalized revision will be archived.
 
 ```bash
 Usage:
@@ -25,8 +29,8 @@ Options:
       --documents-folder[=DOCUMENTS-FOLDER]  Export documents folder
 ```
 
-The environment variable [`EMS_EXCLUDED_CONTENT_TYPES`](parameters.md#ems_excluded_content_types) can be used in order to exclude documents from a list content types.
-
+The environment variable [`EMS_EXCLUDED_CONTENT_TYPES`](parameters.md#ems_excluded_content_types) can be used in order
+to exclude documents from a list content types.
 
 #### Command
 
@@ -44,4 +48,83 @@ Example:
 
 ```bash
 php bin/console ems:admin:command 'ems:env:rebuild preview'
+```
+
+## File Reader
+
+### Import
+
+With this command you can upload a folder to a content-type.
+If the merge options is set to `false`, the rawData will be replaced.
+It will send async index requests, the responses are parsed when the flush-size is reached (default 100).
+
+```bash
+Description:
+  Import an Excel file or a CSV file, one document per row
+
+Usage:
+  emscli:file-reader:import [options] [--] <file> <content-type>
+
+Arguments:
+  file                  File path (xlsx or csv)
+  content-type          Content type target
+
+Options:
+      --config=CONFIG          Config(s) json, file path or hash (multiple values allowed)
+      --dry-run                Just do a dry run
+      --merge=MERGE            Perform a merge or replace [default: true]
+      --flush-size=FLUSH-SIZE  Flush size for the queue [default: 100]
+      --limit=LIMIT            Limit the rows
+```
+
+### Config
+
+You pass multiple config options as string, file path or hash.
+They will be merged together.
+
+```bash
+php bin/console emscli:file-reader:import pages.csv page \
+--config='{"generate_hash": true}' \
+--config='e509a485f786583351cb81911f49ed6a78e28262' \
+--config='./var/files/config.json'
+```
+
+* `delimiter`: ?string (default=null) 
+  * Define the csv delimiter, default 
+* `default_data`: array (default=[])
+  * Data array will be merged with row data
+* `delete_missing_documents`: bool (default=false)
+  * The command will delete content type document that are missing in the import file
+* `encoding`: ?string (default=null)
+  * Define the input file encoding
+* `exclude_rows`: int[] (default=[])
+  * Pass an array of row positions to exclude (0 is first row, -1 is last row)
+* `generate_hash`: bool (default=false)
+  * Use the OUUID column and the content type name in order to generate a "better" ouuid
+* `ouuid_expression`: ?string (default="row['ouuid']")
+  * Expression language apply to excel rows in order to identify the document by its ouuid. If equal to
+    null new document will be created
+* `ouuid_prefix`: ?string (default=null)
+  * The ouuid will prefix with this value before hashing
+
+### Example
+
+I.e.: `ems:file:impo --config='{"ouuid_expression":null}' /home/dockerce/documents/promo/features.xlsx feature`
+
+During the import an associate array containing the Excel row is available in the source `_sync_metadata`.
+
+Example of a field post processing to import a release data:
+
+```twig
+{% if finalize and rootObject._sync_metadata["release"] is defined %}
+    {{ rootObject._sync_metadata["release"]|json_encode|raw }}
+{% endif %}
+```
+
+Example to import data into a multiplexed title field:
+
+```twig
+{% if finalize and rootObject._sync_metadata["title_#{form.parent.name}"] is defined %}
+    {{ rootObject._sync_metadata["title_#{form.parent.name}"]|json_encode|raw }}
+{% endif %}
 ```

elasticms-cli/index.md

@@ -10,4 +10,3 @@ This is a command client to interact with and ElasticMS Admin with some helper c
 | [emscli:documents:update](/elasticms-cli/documents.md)    | Update documents form excel or csv with custom configuration |   
 | [emscli:web:audit](/elasticms-cli/audit.md)               | Audit a website                                              |
 | [emscli:media-library:sync](/elasticms-cli/media-file.md) | Sync a folder to a media-file content typ                    |
-| [emscli:file-reader:import](/elasticms-cli/file-reader.md) | Sync a folder to a media-file content typ                    |

elasticms-cli/parameters.md

@@ -84,14 +84,26 @@ Define backend elasticms url. CommonBundle provides a CoreApi instance.
 
 Define backend authentication token. The commonBundle coreApi instance becomes authenticated.
 
-### EMS_BACKEND_API_TIMEOUT
+### EMS_CORE_API_HEADERS
 
-Adjust the API client's timeout. By default is set to `30` seconds, if you API request may take longueur (e.g. during migration) you can increase the timeout :
+Define extra headers for the API client, helpful for adding cookie header for xdebug.
 
 ```dotenv
-EMS_ELASTICSEARCH_HOSTS='300'
+EMS_CORE_API_HEADERS='{"Cookie":"XDEBUG_SESSION=PHPSTORM"}'
 ```
 
+### EMS_CORE_API_MAX_CONNECTIONS
+
+Define the max connections to the API client, when using async calls. By default it is set to `6`.
+
+### EMS_BACKEND_API_TIMEOUT
+
+Adjust the API client's timeout. By default is set to `30` seconds, if you API request may take longueur (e.g. during migration) you can increase the timeout :
+
+### EMS_BACKEND_API_VERIFY
+
+Define the `verify_host` and `verify_peer` for the api client, by default it is set to `true`.
+
 ### EMS_CACHE
 
 Define the ems cache type. Default value `file_system`.

elasticms-web/parameters.md

@@ -212,14 +212,26 @@ Define backend elasticms url. CommonBundle provides a CoreApi instance.
 
 Define backend authentication token. The commonBundle coreApi instance becomes authenticated.
 
-### EMS_BACKEND_API_TIMEOUT
+### EMS_CORE_API_HEADERS
 
-Adjust the API client's timeout. By default is set to `30` seconds, if you API request may take longueur (e.g. during migration) you can increase the timeout :
+Define extra headers for the API client, helpful for adding cookie header for xdebug.
 
 ```dotenv
-EMS_ELASTICSEARCH_HOSTS='300'
+EMS_CORE_API_HEADERS='{"Cookie":"XDEBUG_SESSION=PHPSTORM"}'
 ```
 
+### EMS_CORE_API_MAX_CONNECTIONS
+
+Define the max connections to the API client, when using async calls. By default it is set to `6`.
+
+### EMS_BACKEND_API_TIMEOUT
+
+Adjust the API client's timeout. By default is set to `30` seconds, if you API request may take longueur (e.g. during migration) you can increase the timeout :
+
+### EMS_BACKEND_API_VERIFY
+
+Define the `verify_host` and `verify_peer` for the api client, by default it is set to `true`.
+
 ### EMS_CACHE
 
 Define the ems cache type. Default value `file_system`.