Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ROX-16735: Add vector.dev Helm chart #1038

Merged
merged 11 commits into from
Jun 21, 2023

Conversation

parametalol
Copy link
Contributor

@parametalol parametalol commented May 16, 2023

Description

Checklist (Definition of Done)

  • Unit and integration tests added
  • Added test description under Test manual
  • Documentation added if necessary (i.e. changes to dev setup, test execution, ...)
  • CI and all relevant tests are passing
  • Add the ticket number to the PR title if available, i.e. ROX-12345: ...
  • Discussed security and business related topics privately. Will move any security and business related topics that arise to private communication channel.
  • Add secret to app-interface Vault or Secrets Manager if necessary

Test manual

$ helm dependencies list 
NAME         	VERSION	REPOSITORY             	STATUS  
cloudwatch   	0.1.0  	                      	unpacked
observability	0.1.0  	                      	unpacked
logging      	0.1.0  	                      	unpacked
vector       	0.21.1 	https://helm.vector.dev	ok      
$ helm template --name-template test ... | yq 'select(.metadata.name | contains "rhacs-vector") | .kind'
ServiceAccount
---
Secret
---
ConfigMap
---
Service
---
Service
---
StatefulSet

Persistent volumes:

$ oc get pvc
NAME                  STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
data-rhacs-vector-0   Bound    pvc-0c019b18-9a02-449b-a11c-552b252a4983   10Gi       RWO            gp3            5h49m
data-rhacs-vector-1   Bound    pvc-607a44d8-6c75-4888-9407-0a06c7be9306   10Gi       RWO            gp3            5h37m
data-rhacs-vector-2   Bound    pvc-969469d6-a2ad-4b83-8ff3-f0f993e2b75a   10Gi       RWO            gp3            5h37m

Vector deployment container:

      containers:
        - name: vector
          image: "timberio/vector:0.29.1-distroless-libc"
          imagePullPolicy: IfNotPresent
          args:
            - --config-dir
            - /etc/vector/
          env:
            - name: AWS_ACCESS_KEY_ID
              valueFrom:
                secretKeyRef:
                  key: aws_access_key_id
                  name: cloudwatch
            - name: AWS_SECRET_ACCESS_KEY
              valueFrom:
                secretKeyRef:
                  key: aws_secret_access_key
                  name: cloudwatch
...

HTTPS config:

$ curl -v ...
...
* Server certificate:
*  subject: CN=rhacs-vector.default.svc
*  start date: May 22 08:06:58 2023 GMT
*  expire date: May 21 08:06:59 2025 GMT
*  issuer: CN=openshift-service-serving-signer@1684507557
...

Test logs: https://s3.console.aws.amazon.com/s3/buckets/acs-audit-logs--us-east-1--acs-dev-dp-01?prefix=test-tenant-id/2023-05-22/&region=us-east-1

@openshift-ci
Copy link
Contributor

openshift-ci bot commented May 16, 2023

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@parametalol parametalol temporarily deployed to development May 16, 2023 15:52 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development May 16, 2023 15:52 — with GitHub Actions Inactive
@parametalol
Copy link
Contributor Author

Current dependencies on/for this PR:

This comment was auto-generated by Graphite.

@parametalol parametalol temporarily deployed to development May 16, 2023 15:54 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development May 16, 2023 15:54 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development May 16, 2023 16:14 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development May 16, 2023 16:14 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development May 16, 2023 16:14 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development May 17, 2023 08:18 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development May 17, 2023 08:18 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development May 17, 2023 08:18 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development May 17, 2023 08:18 — with GitHub Actions Inactive
@parametalol parametalol requested a review from mtodor May 17, 2023 08:19
@parametalol parametalol temporarily deployed to development May 17, 2023 08:20 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development May 17, 2023 08:20 — with GitHub Actions Inactive
@parametalol parametalol marked this pull request as ready for review May 17, 2023 08:54
@parametalol parametalol temporarily deployed to development May 17, 2023 08:54 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development May 17, 2023 08:54 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development May 17, 2023 08:54 — with GitHub Actions Inactive
Copy link
Contributor

@mtodor mtodor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should adjust a few configurations for the vector aggregator.

dp-terraform/helm/rhacs-terraform/terraform_cluster.sh Outdated Show resolved Hide resolved
dp-terraform/helm/rhacs-terraform/values.yaml Show resolved Hide resolved
dp-terraform/helm/rhacs-terraform/values.yaml Outdated Show resolved Hide resolved
scripts/lib/helm.sh Show resolved Hide resolved
@parametalol parametalol temporarily deployed to development May 17, 2023 16:20 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development May 17, 2023 16:20 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development May 17, 2023 16:20 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development May 22, 2023 14:04 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development May 22, 2023 14:04 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development May 22, 2023 14:04 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development May 22, 2023 14:04 — with GitHub Actions Inactive
Copy link
Contributor

@mtodor mtodor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice progress! I've added a few additional suggestions related to the configuration of Vector. Please look at it by yourself if something can be even more improved.

Maybe there are some good suggestions here: https://vector.dev/docs/setup/going-to-prod/arch/aggregator/

dp-terraform/helm/rhacs-terraform/terraform_cluster.sh Outdated Show resolved Hide resolved
dp-terraform/helm/rhacs-terraform/values.yaml Outdated Show resolved Hide resolved
dp-terraform/helm/rhacs-terraform/values.yaml Outdated Show resolved Hide resolved
@parametalol parametalol temporarily deployed to development June 5, 2023 07:38 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development June 5, 2023 07:38 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development June 5, 2023 07:39 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development June 5, 2023 07:39 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development June 5, 2023 07:40 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development June 5, 2023 07:40 — with GitHub Actions Inactive
Copy link
Contributor

@mtodor mtodor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we still need a minor tweak for matchExpressions.

dp-terraform/helm/rhacs-terraform/values.yaml Outdated Show resolved Hide resolved
@parametalol parametalol temporarily deployed to development June 15, 2023 09:58 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development June 15, 2023 09:58 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development June 15, 2023 09:58 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development June 15, 2023 09:58 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development June 15, 2023 09:59 — with GitHub Actions Inactive
@parametalol parametalol temporarily deployed to development June 15, 2023 09:59 — with GitHub Actions Inactive
@parametalol
Copy link
Contributor Author

/retest

@parametalol parametalol requested a review from mtodor June 19, 2023 12:05
Copy link
Contributor

@mtodor mtodor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can go forward and merge this. The next step will be to adjust some settings and enable vector.

@openshift-ci openshift-ci bot added the lgtm label Jun 21, 2023
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 21, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: 0x656b694d, mtodor

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@parametalol parametalol merged commit 3e70db0 into main Jun 21, 2023
@parametalol parametalol deleted the michael/ROX-16735-vector-aggregator branch June 21, 2023 11:54
@mtodor mtodor mentioned this pull request Jun 21, 2023
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants