Terraform MongoDB Self Hosted

[tenaciousdlg]

This is an example of setting up MongoDB Self Hosted and a Teleport agent via Terraform. It is based on the guide from Teleport's documentation. Additional tools used are Vault and AWS. A self hosted Vault server is used for the Teleport Secrets (certificates) for MongoDB. They are mounted to the Vault kv2 engine at kv/data/mongodb. AWS is used for hosting.

Notes

• tctl auth sign --format=mongodb --host=localhost --out=mongo --ttl=2190h has been run and the Secrets mounted to Vault. I updated --host to localhost since the Teleport agent is running on the same server.
• My teleport user has access to a Role that allows access to all database names as the Database users alice and bob.

spec:
  allow:
    db_names:
    - '*'
    db_users:
    - alice
    - bob

• An identity file is used as the authentication mechanism for Teleport with Terraform. More information can be found here on this.
• The Teleport SSH service is also enabled on the agent

Project Directory Structure

.
├── config
│   └── userdata
├── main.tf
├── terraform-identity
├── terraform.tfvars
└── variables.tf

Terraform Configuration Files

variables.tf:

##################################################################################
# VARIABLES
##################################################################################
# teleport
##################################################################################
variable "proxy_service_address" {
  type        = string
  description = "hostname of the Teleport Proxy Service (cluster)"
}

variable "teleport_version" {
  type        = string
  description = "version of Teleport to install on the agent"
}
##################################################################################
# aws
##################################################################################
variable "aws_region" {
  type        = string
  description = "region in which to deploy AWS resources"
}

variable "instance_hostname" {
  type        = string
  description = "hostname of instance and how it is identified in AWS"
}

variable "ssh_key_name" {
  type        = string
  description = "name of key used with AWS"
}

variable "ssh_pub" {
  type        = string
  description = "public key value of AWS SSH key pair"
}
##################################################################################

main.tf:

##################################################################################
# PROVIDERS
##################################################################################
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.16"
    }
    teleport = {
      source  = "terraform.releases.teleport.dev/gravitational/teleport"
      version = "14.0.0"
    }
  }
  required_version = ">= 1.2.0"
}
provider "aws" {
  region = var.aws_region
}
provider "teleport" {
  # Update addr to point to your Teleport tenant URL's host:port (e.g. teleport.example.com)
  addr               = "${var.proxy_service_address}:443"
  # An identity file is used to authenticate against the Terraform user impersonation and role of the Teleport cluster
  identity_file_path = "terraform-identity"
}
# Vault is used for the certificate trust. It holds the initial certs between the MongoDB install and the Teleport cluster
provider "vault" {
  address = "http://127.0.0.1:8200"
}
# Used for generating an auth token for the Teleport MongoDB service
provider "random" {
}
##################################################################################
# DATA SOURCES
##################################################################################
# Data source to pull the latest ubuntu image
data "aws_ami" "ubuntu" {
  most_recent = true
  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-*-22.04-amd64-server-*"]
  }
  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }
  # Value can be pulled with the following command:
  # aws ec2 describe-images --image-ids ami-024e6efaf93d85776 --output json | jq '.Images[] | {Platform, OwnerId}'
  owners = ["099720109477"] 
}
# Data source for pulling the Teleport Secrets (certificates) for MongoDB
data "vault_kv_secret_v2" "mongo" {
  mount = "kv"
  name  = "mongodb"
}
##################################################################################
# RESOURCES
##################################################################################
resource "random_string" "token" {
  length = 32
}
# Creates a Teleport auth token allowing SSH (Node) and MongoDB (Db) access to the Teleport cluster
resource "teleport_provision_token" "mongo" {
  spec = {
    roles = [
      "Db",
      "Node",
    ]
    name = random_string.token.result
  }
  metadata = {
    # TTL of auth token
    expires = timeadd(timestamp(), "1h")
  }
}

resource "aws_key_pair" "main" {
  key_name   = var.ssh_key_name
  public_key = var.ssh_pub
}

resource "aws_instance" "main" {
  ami                         = data.aws_ami.ubuntu.id
  instance_type               = "t3.micro"
  associate_public_ip_address = true
  key_name                    = aws_key_pair.main.key_name
  user_data = templatefile("./config/userdata", {
    token  = teleport_provision_token.mongo.metadata.name
    tele   = var.proxy_service_address
    host   = var.instance_hostname
    # cas is the name of the key on the Vault mount
    moncas = base64decode(data.vault_kv_secret_v2.mongo.data["cas"]) 
    # same with crt
    moncrt = base64decode(data.vault_kv_secret_v2.mongo.data["crt"]) 
  })
  metadata_options {
    http_endpoint = "enabled"
    http_tokens   = "required"
  }
  root_block_device {
    encrypted = true
  }
  # Prevents resource being recreated for minor versions of AMI 
  lifecycle {
    ignore_changes = [ami, user_data]
  }
  tags = {
    Name = var.instance_hostname
  }
}

##################################################################################
# OUTPUT
##################################################################################
output "login" {
  value = "Login with `tsh db ls` or via ${var.proxy_service_address}"
}
##################################################################################

config/userdata:

#!/bin/bash
#########################################################
# Self hosted mongodb install with Teleport
# Based on https://goteleport.com/docs/database-access/guides/mongodb-self-hosted/
#########################################################
sudo hostnamectl set-hostname ${host}
sudo apt update && sudo apt upgrade -y 
sudo apt install -y net-tools cowsay jq
########################################################
## Teleport installation
#########################################################
## Community Edition
#curl https://goteleport.com/static/install.sh | bash -s $${teleport_version}
## ENT Edition
sudo curl https://apt.releases.teleport.dev/gpg -o /usr/share/keyrings/teleport-archive-keyring.asc
source /etc/os-release
echo "deb [signed-by=/usr/share/keyrings/teleport-archive-keyring.asc] \
https://apt.releases.teleport.dev/$${ID?} $${VERSION_CODENAME?} stable/rolling" \
| sudo tee /etc/apt/sources.list.d/teleport.list > /dev/null
sudo apt-get update
sudo apt-get install teleport-ent
echo ${token} > /var/lib/teleport/token
#########################################################
## Teleport configuration
#########################################################
# creates the Teleport agent's configuration file using a heredoc
sudo cat <<-EOF > /etc/teleport.yaml
version: v3
teleport:
  data_dir: "/var/lib/teleport"
  proxy_server: "${tele}:443"
  auth_token: "/var/lib/teleport/token"
  log:
    output: stderr
    severity: INFO
    format:
      output: text
db_service:
  enabled: true
  databases:
  - name: "mongodb"
    protocol: "mongodb"
    uri: "localhost:27017"
    static_labels:
      "environment": "stage"
      "env": "stage"
      "os": "ubuntu"
      "cloud": "aws" 
auth_service:
  enabled: "no"
ssh_service:
  enabled: "yes"
  commands:
  - name: hostname
    command: [hostname]
    period: 1m0s
  labels:
    environment: stage
    env: stage
    os: ubuntu
    cloud: aws
  enhanced_recording:
    enabled: true
proxy_service:
  enabled: "no"
app_service:
  enabled: "no"
EOF
systemctl enable teleport
systemctl restart teleport
#########################################################
# MongoDB server install and teleport configuration
# adds a user named alice that has access to everything in the database and another user named bob with read access
#########################################################
curl -fsSL https://pgp.mongodb.com/server-7.0.asc | \
   sudo gpg -o /usr/share/keyrings/mongodb-server-7.0.gpg \
   --dearmor
echo "deb [ arch=amd64,arm64 signed-by=/usr/share/keyrings/mongodb-server-7.0.gpg ] https://repo.mongodb.org/apt/ubuntu jammy/mongodb-org/7.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-7.0.list
sudo apt-get update
sudo apt-get install -y mongodb-org
# certificate installation 
sudo mkdir -p /etc/certs
sudo touch /etc/certs/mongo.crt
sudo touch /etc/certs/mongo.cas
sudo echo "${moncas}" > /etc/certs/mongo.cas
sudo echo "${moncrt}" > /etc/certs/mongo.crt
sudo systemctl enable mongod
sudo systemctl start mongod
# mongodb user creation for Teleport
sudo cat <<-EOF > /home/ubuntu/user.js
// user.js
db.getSiblingDB("\$external").runCommand(
  {
    createUser: "CN=alice",
    roles: [
      { role: "readWriteAnyDatabase", db: "admin" }
    ]
  }
)

db.getSiblingDB("\$external").runCommand(
  {
    createUser: "CN=bob",
    roles: [
      { role: "read", db: "admin" }
    ]
  }
)
EOF

sleep 1 
sudo mongosh "127.0.0.1" /home/ubuntu/user.js
sleep 4
sudo mv /etc/mongod.conf{,.orig}
sudo cat <<-EOF > /etc/mongod.conf
storage:
  dbPath: /var/lib/mongodb
security:
  authorization: enabled
systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log
net:
  port: 27017
  bindIp: 127.0.0.1
  tls:
    # Using allowTLS instead of requireTLS to allow non-TLS connections 
    mode: allowTLS
    certificateKeyFile: /etc/certs/mongo.crt
    CAFile: /etc/certs/mongo.cas
    # Added for app/local connections in addition to TLS connections
    allowConnectionsWithoutCertificates: true
processManagement:
  timeZoneInfo: /usr/share/zoneinfo
EOF
sudo systemctl restart mongod
#########################################################

Example terraform.tfvars file:

instance_hostname     = "self-mongodb"
proxy_service_address = "teleport.example.com"
teleport_version      = "14.2.0"
aws_region            = "us-east-2"
ssh_pub               = "ssh-rsa AAAGJHAKJ$HG$#KJG$KJ#QHG$Kxxxxx+sJRKYDXryPT/TU/VC0="
ssh_key_name          = "my_key"

Usage

1. Confirm Vault is running and accessible

❯ vault kv get -mount=kv mongo
== Secret Path ==
kv/data/mongodb

======= Metadata =======
Key                Value
---                -----
...
=== Data ===
Key    Value
---    -----
cas    0d0hBWURWUVFLR....

2. Confirm active AWS and Teleport credentials

❯ tsh status
> Profile URL:        https://teleport.example.com:443
...
  Valid until:        2023-11-29 21:00:06 -0600 CST [valid for 3h37m0s]
...

3. Execute Terraform configuration (this assumes terraform init has been previously ran

terraform plan --out=test.plan
terraform apply "test.plan"

4. Login to the database

❯ tsh db ls
Name                               Description Allowed Users Labels                                          Connect                
---------------------------------- ----------- ------------- ----------------------------------------------- ---------------------- 
> mongodb (user: alice, db: acm...             [alice bob]   cloud=aws,env=stage,environment=stage,os=ubuntu tsh db connect mongodb 

❯ tsh db connect --db-user=alice mongodb --db-name=acme
Current Mongosh Log ID: 658d0bb01a61fb6f4f0f8a73
Connecting to:          mongodb://localhost:62081/acme?...appName=mongosh+2.1.1
Using MongoDB:          7.0.4
Using Mongosh:          2.1.1

For mongosh info see: https://docs.mongodb.com/mongodb-shell/

acme> show dbs
admin   100.00 KiB
config   12.00 KiB
local    40.00 KiB
acme> 

[rifler]

Hello, thanks for this guide!

I have only one question - how do you handle secrets renewal and expiration monitoring?

I mean what will happen after 2190 hours?

[tenaciousdlg]

Hi @rifler great question and after 2190 hours the cert would be invalid and the database access would break. This is detailed in the TTL excerpt of the TLS section on the Self Hosted MongoDB guide. You would want to rotate the certificate out with a new valid certificate ahead of the 2190 hour TTL. Some folks do this by hand and others via automation (like Puppet/Ansible).

For a production use case I would look into using Machine ID with Database output instead. Machine ID is not shown in this particular discussion as I wanted this doc to mirror the Self Hosted guide.