Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DPE-1794 Implement COS integration #93

Merged
merged 40 commits into from
Mar 19, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
84af432
WIP: Initial implementation of COS with a lot of rough edges
shayancanonical Nov 20, 2023
a71cbd1
WIP: rough working prototype of cos integration
shayancanonical Nov 21, 2023
fb501f7
Replace secrets charm lib with published one
shayancanonical Nov 27, 2023
26c99d4
Merge branch 'main' into feature/cos_integration
shayancanonical Nov 27, 2023
bb7f341
WIP
shayancanonical Nov 27, 2023
79a5d90
Fix bug where credentials were passed to router options instead of th…
shayancanonical Nov 28, 2023
efea884
Address PR feedback
shayancanonical Dec 1, 2023
4a20fa7
Address PR feedback + update data_interfaces and cos_agent charm libs
shayancanonical Dec 1, 2023
927cffe
Merge branch 'main' into feature/cos_integration
shayancanonical Dec 1, 2023
05302d9
Add integration test for the exporter endpoint
shayancanonical Dec 4, 2023
feaa8ad
Increase timeout for exporter tests to avoid unnecessary timeout erro…
shayancanonical Dec 4, 2023
426bddf
Skip exporter tests on focal
shayancanonical Dec 4, 2023
88da3a9
Delete extra ) from CI workflow file
shayancanonical Dec 4, 2023
7a241ac
Try removing curly braces from integration test condition in ci.yaml
shayancanonical Dec 4, 2023
5cb2d56
Try using single quotes for matrix variable values in integration tes…
shayancanonical Dec 4, 2023
743604a
Try adding curly braces back + use single quotes instead of double qu…
shayancanonical Dec 4, 2023
0418ba5
Try to exclude matrix entry for exporter tests in focal
shayancanonical Dec 5, 2023
234bf0b
Add comment explaining why we skip exporter tests on focal
shayancanonical Dec 5, 2023
44f00cf
Import latest version of data_secrets lib and test focal compatibilit…
shayancanonical Mar 4, 2024
8027248
Merge branch 'main' into feature/cos_integration
shayancanonical Mar 4, 2024
3235040
Update outdated charm libs
shayancanonical Mar 4, 2024
b215877
Specify series for subordinate charms to avoid conflict with principa…
shayancanonical Mar 5, 2024
6689a35
Merge branch 'main' into feature/cos_integration
shayancanonical Mar 5, 2024
b16525e
Fix typos and issues from merge conflict resolution
shayancanonical Mar 6, 2024
a7e6c77
Reconcile all workloads (router, exporter, tls) in one method
shayancanonical Mar 7, 2024
908a011
Run code format
shayancanonical Mar 7, 2024
8ef349b
Address minor PR feedback
shayancanonical Mar 11, 2024
b0cba95
Update outdated charm libs
shayancanonical Mar 11, 2024
1776402
Address PR feedback
shayancanonical Mar 13, 2024
9d03646
Minor leftover improvements + fix bugs
shayancanonical Mar 13, 2024
d45df31
Fix typo in method call
shayancanonical Mar 13, 2024
635d807
Fix bugs + remove usage of data_secrets and use dynamic usage of secr…
shayancanonical Mar 14, 2024
d5b41ce
Another round of feedback + move abstracted secrets code to a separat…
shayancanonical Mar 14, 2024
bdb40a9
Address PR feedback
shayancanonical Mar 15, 2024
ade2bff
Leftovers cleanup
shayancanonical Mar 15, 2024
917d965
Fix bug introduced during refactor
shayancanonical Mar 15, 2024
6e76961
Address feedback + make database test more resilient by using block_u…
shayancanonical Mar 15, 2024
50044f0
Avoid using reconcile_services wrapper in workload
shayancanonical Mar 18, 2024
9665f3e
Make properties in abstract charm private as they will not be invoked…
shayancanonical Mar 18, 2024
7e49cd7
Instantiate database_provides in abstract_charm for type hinting
shayancanonical Mar 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
143 changes: 143 additions & 0 deletions lib/charms/data_platform_libs/v0/data_secrets.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
"""Secrets related helper classes/functions."""
# Copyright 2023 Canonical Ltd.
# See LICENSE file for licensing details.

from typing import Dict, Literal, Optional

from ops import Secret, SecretInfo
from ops.charm import CharmBase
from ops.model import SecretNotFoundError

# The unique Charmhub library identifier, never change it
LIBID = "d77fb3d01aba41ed88e837d0beab6be5"

# Increment this major API version when introducing breaking changes
LIBAPI = 0

# Increment this PATCH version before using `charmcraft publish-lib` or reset
# to 0 if you are raising the major API version
LIBPATCH = 1


APP_SCOPE = "app"
UNIT_SCOPE = "unit"
Scopes = Literal["app", "unit"]


class DataSecretsError(Exception):
"""A secret that we want to create already exists."""


class SecretAlreadyExistsError(DataSecretsError):
"""A secret that we want to create already exists."""


def generate_secret_label(charm: CharmBase, scope: Scopes) -> str:
"""Generate unique group_mappings for secrets within a relation context.

Defined as a standalone function, as the choice on secret labels definition belongs to the
Application Logic. To be kept separate from classes below, which are simply to provide a
(smart) abstraction layer above Juju Secrets.
"""
members = [charm.app.name, scope]
return f"{'.'.join(members)}"


# Secret cache


class CachedSecret:
"""Abstraction layer above direct Juju access with caching.

The data structure is precisely re-using/simulating Juju Secrets behavior, while
also making sure not to fetch a secret multiple times within the same event scope.
"""
shayancanonical marked this conversation as resolved.
Show resolved Hide resolved

def __init__(self, charm: CharmBase, label: str, secret_uri: Optional[str] = None):
self._secret_meta = None
self._secret_content = {}
self._secret_uri = secret_uri
self.label = label
self.charm = charm

def add_secret(self, content: Dict[str, str], scope: Scopes) -> Secret:
"""Create a new secret."""
if self._secret_uri:
raise SecretAlreadyExistsError(
"Secret is already defined with uri %s", self._secret_uri
)

if scope == APP_SCOPE:
secret = self.charm.app.add_secret(content, label=self.label)
else:
secret = self.charm.unit.add_secret(content, label=self.label)
self._secret_uri = secret.id
self._secret_meta = secret
return self._secret_meta

@property
def meta(self) -> Optional[Secret]:
"""Getting cached secret meta-information."""
if self._secret_meta:
return self._secret_meta

if not (self._secret_uri or self.label):
return

try:
self._secret_meta = self.charm.model.get_secret(label=self.label)
except SecretNotFoundError:
if self._secret_uri:
self._secret_meta = self.charm.model.get_secret(
id=self._secret_uri, label=self.label
)
return self._secret_meta

def get_content(self) -> Dict[str, str]:
"""Getting cached secret content."""
if not self._secret_content:
if self.meta:
self._secret_content = self.meta.get_content()
return self._secret_content

def set_content(self, content: Dict[str, str]) -> None:
"""Setting cached secret content."""
if self.meta:
self.meta.set_content(content)
self._secret_content = content

def get_info(self) -> Optional[SecretInfo]:
"""Wrapper function for get the corresponding call on the Secret object if any."""
if self.meta:
return self.meta.get_info()


class SecretCache:
"""A data structure storing CachedSecret objects."""

def __init__(self, charm):
self.charm = charm
self._secrets: Dict[str, CachedSecret] = {}

def get(self, label: str, uri: Optional[str] = None) -> Optional[CachedSecret]:
"""Getting a secret from Juju Secret store or cache."""
if not self._secrets.get(label):
secret = CachedSecret(self.charm, label, uri)

# Checking if the secret exists, otherwise we don't register it in the cache
if secret.meta:
self._secrets[label] = secret
return self._secrets.get(label)

def add(self, label: str, content: Dict[str, str], scope: Scopes) -> CachedSecret:
"""Adding a secret to Juju Secret."""
if self._secrets.get(label):
raise SecretAlreadyExistsError(f"Secret {label} already exists")

secret = CachedSecret(self.charm, label)
secret.add_secret(content, scope)
self._secrets[label] = secret
return self._secrets[label]


# END: Secret cache
Loading
Loading