Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

Improve DELETE functionality. #131

Closed
abbashus opened this issue Jul 24, 2019 · 2 comments
Closed

Improve DELETE functionality. #131

abbashus opened this issue Jul 24, 2019 · 2 comments
Assignees

Comments

@abbashus
Copy link
Contributor

abbashus commented Jul 24, 2019

Currently, the SQL DELETE query does not give proper results to the user.

Data:

PUT bank/_doc/1
{
  "name" : "Abbas",
  "state" : "WA"
}

PUT bank/_doc/2
{
  "name" : "John",
  "state" : "CA" 
}

Vanilla DELETE query:

POST _opendistro/_sql/_explain?
{
    "query" : "DELETE FROM bank" 
}

==== Explain: ====
{"size":1000,"query":{"match_all":{"boost":1.0}},"_source":false}

==== ES raw output: ====
{
  "took" : 84,
  "timed_out" : false,
  "total" : 2,
  "updated" : 0,
  "created" : 0,
  "deleted" : 2, 
  "batches" : 1,
  "version_conflicts" : 0,
  "noops" : 0,
  "retries" : {
    "bulk" : 0,
    "search" : 0
  },
  "throttled_millis" : 0,
  "requests_per_second" : -1.0,
  "throttled_until_millis" : 0,
  "failures" : [ ]
}

==== _opendistro/_sql?_format=jdbc ====
{
  "error": {
    "reason": "There was internal problem at backend",
    "details": "class com.amazon.opendistroforelasticsearch.sql.domain.Delete cannot be cast to class com.amazon.opendistroforelasticsearch.sql.domain.Select (com.amazon.opendistroforelasticsearch.sql.domain.Delete and com.amazon.opendistroforelasticsearch.sql.domain.Select are in unnamed module of loader java.net.FactoryURLClassLoader @46963479)",
    "type": "ClassCastException"
  },
  "status": 500
}

==== _opendistro/_sql?format=csv ====
null 

DELETE query with condition(s):

POST _opendistro/_sql/_explain?
{
    "query" : "DELETE FROM bank WHERE name = 'Abbas' " 
}

{
  "size": 1000,
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "name": {
              "value": "Abbas",
              "boost": 1
            }
          }
        }
      ],
      "adjust_pure_negative": true,
      "boost": 1
    }
  },
  "_source": false
}

Observation:

  • Even though the actual data is deleted , output of format=jdbc emits exception and for format=csv it just outputs null which is not very informative.
  • For DELETE query with WHERE clause, for fields with type text + keyword, the ES DSL is incorrect and thus data is not deleted as expected. It should be name.keyword instead of name in Term query.

Proposed Solution:
For format=jdbc, result should be something like this:

{
  "schema": [
    {
      "name": "deleted_rows",
      "type": "text"
    }
  ],
  "total": 1,
  "datarows": [
    [
      2
    ]
  ],
  "size": 1,
  "status": 200
}

For format=csv, result should be something like this to be consistent with jdbc.

deleted_rows
2

For incorrect ES DSL , make it go through TermFieldRewriter.

@abbashus
Copy link
Contributor Author

Closing #96 in lieu of this.

@dai-chen
Copy link
Member

The DELETE for JDBC (default) format has been fixed. Will create separate issue if CSV needs this support.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants