[CT-410] Handle host name with trailing backslash

[JCZuurmond]

Describe the feature

Support complete host name in profile, that is include the https:// before the host.

now: adb-<organisation id>.azuredatabricks.net
complete host: https://adb-<organisation id>.azuredatabricks.net

Describe alternatives you've considered

Keeping it as is.

Additional context

When Databricks refers to DATABRICKS_HOST it means the complete host name, including the https://.

Who will this benefit?

It's more consistent with the Databricks documentation, therefore easier to understand. Also, I insert the host and token as environment variable, if the complete host name is supported, then the Databricks cli works with those environment variables too.

Are you interested in contributing this feature?

Sure. It is a good first issue, I am willing to coach someone too.

[VersusFacit]

hey @JCZuurmond Appreciate the explanation and generous offer to contribute/coach someone else.

Before I can triage this well, could you do me a favor and let me know where I can learn more about this method documentation-wise. It's not entirely clear to me at the outset that we should match databrick's documentation or that this is the superior method, or if specifying https:// would prevent other users without that http scheme from being explicitly listed.

Looking forward to some pointers from you :)

[jtcohen6]

I think we may already have logic to handle this, added in #165:

dbt-spark/dbt/adapters/spark/connections.py

Lines 366 to 369 in d92a4e5

	# Prepend https:// if it is missing
	host = creds.host
	if not host.startswith('https://'):
	host = 'https://' + creds.host

That is, you can supply the host with or without https://, it should work either way.

[JCZuurmond]

Interesting, I'll have a look, I probably made a mistake somewhere

[jtcohen6]

I have noticed that a simple copy-paste of the Databricks host URL often includes a trailing backslash (https://adb-<organisation id>.azuredatabricks.net/), and dbt-spark doesn't do anything to handle that

[JCZuurmond]

@jtcohen6 : Could you make this a good-first-issue?

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.

[tim-steinkuhler]

@JCZuurmond, do you still think this is needed? Looking at @jtcohen6's comment, it seems like providing the full host name is already supported.

[Fokko]

@tim-steinkuhler Implemented this in 2021 in #165 :) I think trimming the trailing slash would be a great addition. Would be nice if you have a Databricks instance laying around to test this?

[tim-steinkuhler]

wow, that's a quick response fokko :-), that sounds doable!

[JCZuurmond]

@tim-steinkuhler : Will you implement this issue?

[tim-steinkuhler]

I think I have the correct code, but just having some trouble setting everything up spark correctly, so haven't been able to test it yet :-D