Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DPE-1794 Implement COS integration #93

Merged
merged 40 commits into from
Mar 19, 2024
Merged
Show file tree
Hide file tree
Changes from 36 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
84af432
WIP: Initial implementation of COS with a lot of rough edges
shayancanonical Nov 20, 2023
a71cbd1
WIP: rough working prototype of cos integration
shayancanonical Nov 21, 2023
fb501f7
Replace secrets charm lib with published one
shayancanonical Nov 27, 2023
26c99d4
Merge branch 'main' into feature/cos_integration
shayancanonical Nov 27, 2023
bb7f341
WIP
shayancanonical Nov 27, 2023
79a5d90
Fix bug where credentials were passed to router options instead of th…
shayancanonical Nov 28, 2023
efea884
Address PR feedback
shayancanonical Dec 1, 2023
4a20fa7
Address PR feedback + update data_interfaces and cos_agent charm libs
shayancanonical Dec 1, 2023
927cffe
Merge branch 'main' into feature/cos_integration
shayancanonical Dec 1, 2023
05302d9
Add integration test for the exporter endpoint
shayancanonical Dec 4, 2023
feaa8ad
Increase timeout for exporter tests to avoid unnecessary timeout erro…
shayancanonical Dec 4, 2023
426bddf
Skip exporter tests on focal
shayancanonical Dec 4, 2023
88da3a9
Delete extra ) from CI workflow file
shayancanonical Dec 4, 2023
7a241ac
Try removing curly braces from integration test condition in ci.yaml
shayancanonical Dec 4, 2023
5cb2d56
Try using single quotes for matrix variable values in integration tes…
shayancanonical Dec 4, 2023
743604a
Try adding curly braces back + use single quotes instead of double qu…
shayancanonical Dec 4, 2023
0418ba5
Try to exclude matrix entry for exporter tests in focal
shayancanonical Dec 5, 2023
234bf0b
Add comment explaining why we skip exporter tests on focal
shayancanonical Dec 5, 2023
44f00cf
Import latest version of data_secrets lib and test focal compatibilit…
shayancanonical Mar 4, 2024
8027248
Merge branch 'main' into feature/cos_integration
shayancanonical Mar 4, 2024
3235040
Update outdated charm libs
shayancanonical Mar 4, 2024
b215877
Specify series for subordinate charms to avoid conflict with principa…
shayancanonical Mar 5, 2024
6689a35
Merge branch 'main' into feature/cos_integration
shayancanonical Mar 5, 2024
b16525e
Fix typos and issues from merge conflict resolution
shayancanonical Mar 6, 2024
a7e6c77
Reconcile all workloads (router, exporter, tls) in one method
shayancanonical Mar 7, 2024
908a011
Run code format
shayancanonical Mar 7, 2024
8ef349b
Address minor PR feedback
shayancanonical Mar 11, 2024
b0cba95
Update outdated charm libs
shayancanonical Mar 11, 2024
1776402
Address PR feedback
shayancanonical Mar 13, 2024
9d03646
Minor leftover improvements + fix bugs
shayancanonical Mar 13, 2024
d45df31
Fix typo in method call
shayancanonical Mar 13, 2024
635d807
Fix bugs + remove usage of data_secrets and use dynamic usage of secr…
shayancanonical Mar 14, 2024
d5b41ce
Another round of feedback + move abstracted secrets code to a separat…
shayancanonical Mar 14, 2024
bdb40a9
Address PR feedback
shayancanonical Mar 15, 2024
ade2bff
Leftovers cleanup
shayancanonical Mar 15, 2024
917d965
Fix bug introduced during refactor
shayancanonical Mar 15, 2024
6e76961
Address feedback + make database test more resilient by using block_u…
shayancanonical Mar 15, 2024
50044f0
Avoid using reconcile_services wrapper in workload
shayancanonical Mar 18, 2024
9665f3e
Make properties in abstract charm private as they will not be invoked…
shayancanonical Mar 18, 2024
7e49cd7
Instantiate database_provides in abstract_charm for type hinting
shayancanonical Mar 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,434 changes: 998 additions & 436 deletions lib/charms/data_platform_libs/v0/data_interfaces.py

Large diffs are not rendered by default.

806 changes: 806 additions & 0 deletions lib/charms/grafana_agent/v0/cos_agent.py

Large diffs are not rendered by default.

68 changes: 58 additions & 10 deletions lib/charms/operator_libs_linux/v2/snap.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@

# Increment this PATCH version before using `charmcraft publish-lib` or reset
# to 0 if you are raising the major API version
LIBPATCH = 3
LIBPATCH = 4


# Regex to locate 7-bit C1 ANSI sequences
Expand Down Expand Up @@ -214,7 +214,7 @@ class Snap(object):
- state: a `SnapState` representation of its install status
- channel: "stable", "candidate", "beta", and "edge" are common
- revision: a string representing the snap's revision
- confinement: "classic" or "strict"
- confinement: "classic", "strict", or "devmode"
"""

def __init__(
Expand Down Expand Up @@ -475,6 +475,8 @@ def _install(
args = []
if self.confinement == "classic":
args.append("--classic")
if self.confinement == "devmode":
args.append("--devmode")
if channel:
args.append('--channel="{}"'.format(channel))
if revision:
Expand All @@ -489,6 +491,7 @@ def _refresh(
channel: Optional[str] = "",
cohort: Optional[str] = "",
revision: Optional[str] = None,
devmode: bool = False,
leave_cohort: Optional[bool] = False,
) -> None:
"""Refresh a snap.
Expand All @@ -497,6 +500,7 @@ def _refresh(
channel: the channel to install from
cohort: optionally, specify a cohort.
revision: optionally, specify the revision of the snap to refresh
devmode: optionally, specify devmode confinement
leave_cohort: leave the current cohort.
"""
args = []
Expand All @@ -506,6 +510,9 @@ def _refresh(
if revision:
args.append('--revision="{}"'.format(revision))

if devmode:
args.append("--devmode")

if not cohort:
cohort = self._cohort

Expand All @@ -530,6 +537,7 @@ def ensure(
self,
state: SnapState,
classic: Optional[bool] = False,
devmode: bool = False,
channel: Optional[str] = "",
cohort: Optional[str] = "",
revision: Optional[str] = None,
Expand All @@ -539,6 +547,7 @@ def ensure(
Args:
state: a `SnapState` to reconcile to.
classic: an (Optional) boolean indicating whether classic confinement should be used
devmode: an (Optional) boolean indicating whether devmode confinement should be used
channel: the channel to install from
cohort: optional. Specify the key of a snap cohort.
revision: optional. the revision of the snap to install/refresh
Expand All @@ -549,7 +558,15 @@ def ensure(
Raises:
SnapError if an error is encountered
"""
self._confinement = "classic" if classic or self._confinement == "classic" else ""
if classic and devmode:
raise ValueError("Cannot set both classic and devmode confinement")

if classic or self._confinement == "classic":
self._confinement = "classic"
elif devmode or self._confinement == "devmode":
self._confinement = "devmode"
else:
self._confinement = ""

if state not in (SnapState.Present, SnapState.Latest):
# We are attempting to remove this snap.
Expand All @@ -566,7 +583,7 @@ def ensure(
self._install(channel, cohort, revision)
else:
# The snap is installed, but we are changing it (e.g., switching channels).
self._refresh(channel, cohort, revision)
self._refresh(channel=channel, cohort=cohort, revision=revision, devmode=devmode)

self._update_snap_apps()
self._state = state
Expand Down Expand Up @@ -892,6 +909,7 @@ def add(
state: Union[str, SnapState] = SnapState.Latest,
channel: Optional[str] = "",
classic: Optional[bool] = False,
devmode: bool = False,
cohort: Optional[str] = "",
revision: Optional[str] = None,
) -> Union[Snap, List[Snap]]:
Expand All @@ -904,6 +922,8 @@ def add(
channel: an (Optional) channel as a string. Defaults to 'latest'
classic: an (Optional) boolean specifying whether it should be added with classic
confinement. Default `False`
devmode: an (Optional) boolean specifying whether it should be added with devmode
confinement. Default `False`
cohort: an (Optional) string specifying the snap cohort to use
revision: an (Optional) string specifying the snap revision to use

Expand All @@ -920,7 +940,7 @@ def add(
if isinstance(state, str):
state = SnapState(state)

return _wrap_snap_operations(snap_names, state, channel, classic, cohort, revision)
return _wrap_snap_operations(snap_names, state, channel, classic, devmode, cohort, revision)


@_cache_init
Expand All @@ -936,8 +956,13 @@ def remove(snap_names: Union[str, List[str]]) -> Union[Snap, List[Snap]]:
snap_names = [snap_names] if isinstance(snap_names, str) else snap_names
if not snap_names:
raise TypeError("Expected at least one snap to add, received zero!")

return _wrap_snap_operations(snap_names, SnapState.Absent, "", False)
return _wrap_snap_operations(
snap_names=snap_names,
state=SnapState.Absent,
channel="",
classic=False,
devmode=False,
)


@_cache_init
Expand All @@ -946,6 +971,7 @@ def ensure(
state: str,
channel: Optional[str] = "",
classic: Optional[bool] = False,
devmode: bool = False,
cohort: Optional[str] = "",
revision: Optional[int] = None,
) -> Union[Snap, List[Snap]]:
Expand All @@ -957,6 +983,8 @@ def ensure(
channel: an (Optional) channel as a string. Defaults to 'latest'
classic: an (Optional) boolean specifying whether it should be added with classic
confinement. Default `False`
devmode: an (Optional) boolean specifying whether it should be added with devmode
confinement. Default `False`
cohort: an (Optional) string specifying the snap cohort to use
revision: an (Optional) integer specifying the snap revision to use

Expand All @@ -970,7 +998,15 @@ def ensure(
channel = "latest"

if state in ("present", "latest") or revision:
return add(snap_names, SnapState(state), channel, classic, cohort, revision)
return add(
snap_names=snap_names,
state=SnapState(state),
channel=channel,
classic=classic,
devmode=devmode,
cohort=cohort,
revision=revision,
)
else:
return remove(snap_names)

Expand All @@ -980,6 +1016,7 @@ def _wrap_snap_operations(
state: SnapState,
channel: str,
classic: bool,
devmode: bool,
cohort: Optional[str] = "",
revision: Optional[str] = None,
) -> Union[Snap, List[Snap]]:
Expand All @@ -995,7 +1032,12 @@ def _wrap_snap_operations(
snap.ensure(state=SnapState.Absent)
else:
snap.ensure(
state=state, classic=classic, channel=channel, cohort=cohort, revision=revision
state=state,
classic=classic,
devmode=devmode,
channel=channel,
cohort=cohort,
revision=revision,
)
snaps["success"].append(snap)
except SnapError as e:
Expand All @@ -1014,13 +1056,17 @@ def _wrap_snap_operations(


def install_local(
filename: str, classic: Optional[bool] = False, dangerous: Optional[bool] = False
filename: str,
classic: Optional[bool] = False,
devmode: Optional[bool] = False,
dangerous: Optional[bool] = False,
) -> Snap:
"""Perform a snap operation.

Args:
filename: the path to a local .snap file to install
classic: whether to use classic confinement
devmode: whether to use devmode confinement
dangerous: whether --dangerous should be passed to install snaps without a signature

Raises:
Expand All @@ -1033,6 +1079,8 @@ def install_local(
]
if classic:
args.append("--classic")
if devmode:
args.append("--devmode")
if dangerous:
args.append("--dangerous")
try:
Expand Down
8 changes: 5 additions & 3 deletions metadata.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,9 @@ provides:
shared-db:
interface: mysql-shared
scope: container
cos-agent:
interface: cos_agent
limit: 1
requires:
backend-database:
interface: mysql_client
Expand All @@ -36,14 +39,13 @@ requires:
interface: juju-info
scope: container
peers:
cos:
interface: cos
upgrade-version-a:
# Relation versioning scheme:
# DA056 - Upgrading in-place upgrade protocol
# https://docs.google.com/document/d/1H7qy5SAwLiCOKO9xMQJbbQP5_-jGV6Lhi-mJOk4gZ08/edit
interface: upgrade
# TODO TLS VM: re-enable peer relation
# mysql-router-peers:
# interface: mysql_router_peers
# DEPRECATED shared-db: Workaround for legacy "mysql-shared" interface using unit databags instead of app databag
deprecated-shared-db-credentials:
interface: _deprecated_shared_db_peers
Loading
Loading