Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Qubole] - Adding support to process Quantum query types. #4066

Merged
merged 14 commits into from
Aug 29, 2019
Merged

[Qubole] - Adding support to process Quantum query types. #4066

Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
9cabdd1
[Qubole] - Adding support to process Quantum query types.
sandeepV2 Aug 13, 2019
6e6af57
-Making quantum as defult query.
sandeepV2 Aug 19, 2019
7a38a45
keeping pep8 standards
sandeepV2 Aug 19, 2019
2789a2b
Maintainig pep8 std
sandeepV2 Aug 20, 2019
e5d0015
Use latest version of qds-sdk
sandeepV2 Aug 20, 2019
42b704c
Use qds-sdk v1.13.0
sandeepV2 Aug 20, 2019
ad98cb4
Use qds-sdk v1.12.0
sandeepV2 Aug 20, 2019
44058cf
Use qds-sdk v1.13.0
sandeepV2 Aug 20, 2019
b5dce0f
Updating SDK with verified version
sandeepV2 Aug 26, 2019
2144c02
hive as default query type
sandeepV2 Aug 28, 2019
c7333b4
Merge branch 'master' into qubole_quantum
sandeepV2 Aug 29, 2019
8f552b4
qds-sdk : Locking most recent release version
sandeepV2 Aug 29, 2019
964c089
qds-sdk : Locking recent release version
sandeepV2 Aug 29, 2019
e68aa49
falling back to original version of qds-sdk
sandeepV2 Aug 29, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 15 additions & 10 deletions redash/query_runner/qubole.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@
try:
import qds_sdk
from qds_sdk.qubole import Qubole as qbol
from qds_sdk.commands import Command, HiveCommand, PrestoCommand
from qds_sdk.commands import Command, HiveCommand
from qds_sdk.commands import SqlCommand, PrestoCommand
arikfr marked this conversation as resolved.
Show resolved Hide resolved
enabled = True
except ImportError:
enabled = False
Expand All @@ -24,6 +25,11 @@ def configuration_schema(cls):
return {
"type": "object",
"properties": {
"query_type": {
"type": "string",
"title": "Query Type (quantum / presto / hive)",
"default": "hive"
},
"endpoint": {
"type": "string",
"title": "API Endpoint",
Expand All @@ -37,15 +43,10 @@ def configuration_schema(cls):
"type": "string",
"title": "Cluster Label",
"default": "default"
},
"query_type": {
"type": "string",
"title": "Query Type (hive or presto)",
"default": "hive"
}
},
"order": ["endpoint", "token", "cluster"],
"required": ["endpoint", "token", "cluster"],
"order": ["query_type", "endpoint", "token", "cluster"],
"required": ["endpoint", "token"],
"secret": ["token"]
}

Expand All @@ -67,8 +68,12 @@ def run_query(self, query, user):
api_url='%s/api' % self.configuration['endpoint'])

try:
cls = PrestoCommand if(self.configuration['query_type'] == 'presto') else HiveCommand
cmd = cls.create(query=query, label=self.configuration['cluster'])
if self.configuration['query_type'] == 'quantum':
sandeepV2 marked this conversation as resolved.
Show resolved Hide resolved
cmd = SqlCommand.create(query=query)
else:
cls = PrestoCommand if(self.configuration['query_type'] == 'presto') else HiveCommand
cmd = cls.create(query=query, label=self.configuration['cluster'])

logging.info("Qubole command created with Id: %s and Status: %s", cmd.id, cmd.status)

while not Command.is_done(cmd.status):
Expand Down