diff --git a/roles/ess-pro/README.md b/roles/ess-pro/README.md new file mode 100644 index 0000000..07a2d2e --- /dev/null +++ b/roles/ess-pro/README.md @@ -0,0 +1,221 @@ +# Ansible Role: ess_pro + +Deploys Element Server Suite Pro on a single-node K3s cluster, using the +official `oci://registry.element.io/matrix-stack` Helm chart. + +Follows the conventions of the other `digitalboard.core` roles +(bookstack, opnform, homarr): the role itself is secrets-agnostic; +sensitive values are supplied via `group_vars/ess_servers.yml` as +`community.hashi_vault` lookups against OpenBao. + +Replaces the previously-planned `coturn` + `nextcloud-talk-hpb` +(spreed-signaling + Janus) stack with a fully-fledged Matrix backend +(Synapse Pro, MAS, Element Web, Element Admin, Element Call / LiveKit). + +--- + +## Hostnames + +| Component | Default hostname | +| --------------------- | ----------------------------- | +| Matrix `serverName` | `digitalboard.ch` | +| Synapse | `matrix.digitalboard.ch` | +| MAS | `mas.digitalboard.ch` | +| Element Web | `chat.digitalboard.ch` | +| Element Admin Panel | `admin.digitalboard.ch` | +| Matrix RTC / LiveKit | `rtc.digitalboard.ch` | +| `.well-known` apex | `digitalboard.ch` | + +Note: MAS uses `mas.*` because `auth.digitalboard.ch` is already owned +by Keycloak in the reference infrastructure. Override the whole map via +`ess_pro_hostnames` if needed (e.g. for `wksbern.ch`). + +--- + +## Architecture + +``` + ┌────────────────────────────────────────────┐ +Internet ──HTTPS──▶│ DMZ Traefik (reference-ansible) │ + │ chat.* mas.* matrix.* admin.* rtc.* │ + └───────────────────┬─────────────────────────┘ + │ HTTP (TLS terminated) + ▼ + ┌────────────────────────────────────────────┐ + │ ess host (Debian bookworm + K3s) │ + │ ┌──────────────────────────────────────┐ │ + │ │ ess namespace │ │ + │ │ • synapse-pro │ │ + │ │ • matrix-authentication-service │ │ + │ │ • element-web │ │ + │ │ • element-admin │ │ + │ │ • matrix-rtc (lk-jwt + LiveKit SFU) │ │ + │ │ • haproxy / well-known │ │ + │ └──────────────────────────────────────┘ │ + └────────────────────────────────────────────┘ + │ UDP 50000–60000 + ▼ + LiveKit ICE candidates +``` + +Integration with the existing reference-ansible stack: + +- **DMZ Traefik** terminates TLS, forwards HTTP to the K3s node. +- **Keycloak** on `auth.digitalboard.ch` (Realm `Digitalboard`) is MAS' + upstream OIDC provider — same SSO story as bookstack/opnform/homarr. +- **Garage** (S3-compatible) hosts the Synapse media store via the + `ess-media` bucket. +- **OpenBao** on the same path layout (`kv/digitalboard/`). +- **Cluster lives on the same VM** that was previously planned for + coturn/HPB, because it has the right DMZ NAT topology for SFU UDP. + +--- + +## Prerequisites + +1. Ansible collections on the control node: + + ```bash + ansible-galaxy collection install \ + kubernetes.core community.general community.hashi_vault + pip install kubernetes pyyaml hvac + ``` + +2. ESS Pro subscription credentials in OpenBao at + `kv/digitalboard/ess-pro` (KV v2, flat keys): + + ```bash + bao kv put kv/digitalboard/ess-pro \ + username='ess-customer-xxx' \ + token='paste-from-customer.element.io' \ + client_secret='from-keycloak' \ + s3_access_key='...' \ + s3_secret_key='...' + ``` + + See `examples/openbao-bootstrap.sh` for an interactive helper. + +3. Keycloak OIDC client `ess-mas` in the `Digitalboard` realm with + redirect URI + `https://mas.digitalboard.ch/upstream/callback/01J0KCK0DNNNDIGITALBOARDKC01`. + +4. Garage bucket `ess-media` with a dedicated access key. + +5. DNS A/AAAA records for `matrix.`, `mas.`, `chat.`, `admin.`, `rtc.` + and the apex `digitalboard.ch`, pointing at the DMZ Traefik. + +6. DMZ firewall NAT-forwards UDP `50000-60000` (configurable) and TCP + `7881` to the K3s node — LiveKit's media ports. + +--- + +## Required variables + +| Variable | Notes | +| --------------------------------- | ------------------------------------ | +| `ess_pro_registry_username` | OpenBao lookup — see example | +| `ess_pro_registry_token` | OpenBao lookup | +| `ess_pro_oidc_client_secret` | OpenBao lookup (when OIDC enabled) | +| `ess_pro_s3_access_key` | OpenBao lookup (when S3 enabled) | +| `ess_pro_s3_secret_key` | OpenBao lookup (when S3 enabled) | +| `ess_pro_rtc_external_ip` | DMZ public IP for LiveKit ICE | + +See `defaults/main.yml` for everything else. The `examples/` directory +contains a ready-to-use `group_vars/ess_servers.yml` with all the +OpenBao lookups pre-wired. + +--- + +## Example playbook + +```yaml +- name: Deploy Element Server Suite Pro + hosts: ess_servers + become: true + roles: + - digitalboard.core.k3s + - digitalboard.core.ess_pro +``` + +With inventory variables (`group_vars/ess_servers.yml`): + +```yaml +ess_pro_server_name: "digitalboard.ch" +ess_pro_oidc_enabled: true +ess_pro_s3_media_enabled: true +ess_pro_rtc_external_ip: "203.0.113.42" + +ess_pro_registry_username: "{{ lookup('community.hashi_vault.vault_kv2_get', + 'digitalboard/ess-pro', + mount_point='kv').data.data.username }}" +ess_pro_registry_token: "{{ lookup('community.hashi_vault.vault_kv2_get', + 'digitalboard/ess-pro', + mount_point='kv').data.data.token }}" +ess_pro_oidc_client_secret: "{{ lookup('community.hashi_vault.vault_kv2_get', + 'digitalboard/ess-pro', + mount_point='kv').data.data.client_secret }}" +ess_pro_s3_access_key: "{{ lookup('community.hashi_vault.vault_kv2_get', + 'digitalboard/ess-pro', + mount_point='kv').data.data.s3_access_key }}" +ess_pro_s3_secret_key: "{{ lookup('community.hashi_vault.vault_kv2_get', + 'digitalboard/ess-pro', + mount_point='kv').data.data.s3_secret_key }}" +``` + +--- + +## Post-deploy + +1. Get the bootstrap admin password: + + ```bash + kubectl -n ess get secrets/ess-generated \ + -o jsonpath='{.data.ADMIN_USER_PASSWORD}' | base64 -d + ``` + +2. Log in to `https://admin.digitalboard.ch` as + `@localadmin:digitalboard.ch`. + +3. Create users via the Admin Panel or via MAS: + + ```bash + kubectl -n ess exec -it deploy/ess-matrix-authentication-service -- \ + mas-cli manage register-user + ``` + +4. If OIDC is enabled, users can also log in directly via Keycloak from + the Element Web client. + +--- + +## Operations + +- **Re-deploy / config change**: re-run the playbook. `kubernetes.core.helm` + performs `helm upgrade --install` — idempotent. +- **Upgrade chart version**: bump `ess_pro_chart_version`, re-run. +- **Rotate the Element token**: update it in OpenBao, re-run the + playbook. The role re-creates the image pull secret and re-authenticates + the Helm CLI. +- **Rendered values.yaml** on the host: `/etc/ess/values.yaml`. +- **Tear down**: + + ```bash + helm uninstall -n ess ess && kubectl delete ns ess + ``` + +--- + +## Known caveats + +- The bundled in-cluster Postgres is **not for production** — point at + an external Postgres VM before going live. +- TLS termination on the DMZ Traefik means well-known delegation and + Element Call ICE rely on the upstream proxy sending correct + `X-Forwarded-Proto`. Synapse is configured with `x_forwarded: true`; + verify with `curl https://digitalboard.ch/.well-known/matrix/server`. +- ESS Pro Helm chart field names track upstream — if a future chart + version renames a field (e.g. `matrixRTC.sfu.additional`), update + `templates/values.yaml.j2` accordingly. Run + `helm show values oci://registry.element.io/matrix-stack` after + major upgrades. +- The `serverName` is **immutable** after first deploy. diff --git a/roles/ess-pro/defaults/main.yml b/roles/ess-pro/defaults/main.yml new file mode 100644 index 0000000..e555c05 --- /dev/null +++ b/roles/ess-pro/defaults/main.yml @@ -0,0 +1,129 @@ +# SPDX-License-Identifier: MIT-0 +--- +# ============================================================================= +# ess-pro role — defaults +# ============================================================================= +# Secrets (registry token, OIDC client secret, S3 keys, Postgres passwords) +# are intentionally left empty here. Provide them via +# `group_vars/ess_servers.yml` — either as plain values (PoC), via +# ansible-vault, or as OpenBao lookups, matching the pattern used by the +# other digitalboard.core roles (bookstack, opnform, homarr). +# +# Example OpenBao lookup: +# ess_pro_registry_token: "{{ lookup('community.hashi_vault.vault_kv2_get', +# 'digitalboard/ess-pro', +# mount_point='kv').data.data.token }}" + +# ----------------------------------------------------------------------------- +# Helm release +# ----------------------------------------------------------------------------- +ess_pro_namespace: "ess" +ess_pro_release_name: "ess" +ess_pro_chart_ref: "oci://registry.element.io/matrix-stack" +# Pin a chart version in production. Leave empty to track the latest stable. +ess_pro_chart_version: "" +ess_pro_helm_timeout: "15m" +ess_pro_helm_wait: true + +# Where to store rendered values.yaml on the target host. +ess_pro_config_dir: "/etc/ess" + +# ----------------------------------------------------------------------------- +# Matrix server identity +# ----------------------------------------------------------------------------- +# The Matrix serverName forms the user ID domain (@user:serverName) and the +# federation key. It cannot be changed after the first deploy without data +# loss. Override for production deployments (e.g. wksbern.ch). +ess_pro_server_name: "digitalboard.ch" + +# Per-service hostnames. The DMZ Traefik (reference-ansible) terminates TLS +# for these and forwards to the K3s node. +# +# Convention follows the other digitalboard.core roles: +# wiki.digitalboard.ch (bookstack) +# forms.digitalboard.ch (opnform) +# home.digitalboard.ch (homarr) +# auth.digitalboard.ch (keycloak) <- already taken — MAS uses `mas.` +# chat.digitalboard.ch (this role, Element Web) +ess_pro_hostnames: + synapse: "matrix.{{ ess_pro_server_name }}" + mas: "mas.{{ ess_pro_server_name }}" + element_web: "chat.{{ ess_pro_server_name }}" + element_admin: "admin.{{ ess_pro_server_name }}" + matrix_rtc: "rtc.{{ ess_pro_server_name }}" + +# ----------------------------------------------------------------------------- +# Element image registry credentials (from customer.element.io) +# ----------------------------------------------------------------------------- +ess_pro_registry_url: "registry.element.io" +ess_pro_registry_username: "" # set in group_vars/ess_servers.yml +ess_pro_registry_token: "" # set in group_vars/ess_servers.yml + +# ----------------------------------------------------------------------------- +# Ingress / TLS strategy +# ----------------------------------------------------------------------------- +# The reference-ansible pattern terminates TLS on the DMZ Traefik. Inside +# K3s, workloads serve plain HTTP. +ess_pro_tls_terminate_externally: true +ess_pro_ingress_class: "traefik" + +# ----------------------------------------------------------------------------- +# PostgreSQL +# ----------------------------------------------------------------------------- +# Chart-internal Postgres for demo. For production, point at an external DB. +ess_pro_postgres_external: false +ess_pro_postgres_host: "" +ess_pro_postgres_port: 5432 +ess_pro_postgres_sslmode: "prefer" +ess_pro_postgres_synapse_db: "synapse" +ess_pro_postgres_synapse_user: "synapse" +ess_pro_postgres_synapse_password: "" +ess_pro_postgres_mas_db: "mas" +ess_pro_postgres_mas_user: "mas" +ess_pro_postgres_mas_password: "" + +# ----------------------------------------------------------------------------- +# Delegated authentication via the digitalboard Keycloak +# ----------------------------------------------------------------------------- +# Create a confidential OIDC client `ess-mas` in the `Digitalboard` realm +# with redirect_uri +# https://{{ ess_pro_hostnames.mas }}/upstream/callback/01J0KCK0DNNNDIGITALBOARDKC01 +ess_pro_oidc_enabled: false +ess_pro_oidc_issuer: "https://auth.digitalboard.ch/realms/Digitalboard" +ess_pro_oidc_client_id: "ess-mas" +ess_pro_oidc_client_secret: "" +ess_pro_oidc_provider_name: "Digitalboard" +ess_pro_oidc_scopes: + - "openid" + - "profile" + - "email" + +# ----------------------------------------------------------------------------- +# Media storage (Garage S3) +# ----------------------------------------------------------------------------- +ess_pro_s3_media_enabled: false +ess_pro_s3_endpoint: "https://s3.digitalboard.ch" +ess_pro_s3_region: "garage" +ess_pro_s3_bucket: "ess-media" +ess_pro_s3_access_key: "" +ess_pro_s3_secret_key: "" + +# ----------------------------------------------------------------------------- +# Matrix RTC Backend / Element Call (LiveKit SFU + lk-jwt-service) +# ----------------------------------------------------------------------------- +ess_pro_rtc_enabled: true +ess_pro_rtc_udp_port_range_start: 50000 +ess_pro_rtc_udp_port_range_end: 60000 +# Externally reachable IP for LiveKit ICE candidates (DMZ public IP). +ess_pro_rtc_external_ip: "" + +# ----------------------------------------------------------------------------- +# Initial admin (chart creates @localadmin: by default) +# ----------------------------------------------------------------------------- +ess_pro_create_initial_admin: true + +# ----------------------------------------------------------------------------- +# Helm CLI install +# ----------------------------------------------------------------------------- +ess_pro_helm_version: "v3.16.4" +ess_pro_helm_install_dir: "/usr/local/bin" diff --git a/roles/ess-pro/examples/group_vars-ess_servers.yml b/roles/ess-pro/examples/group_vars-ess_servers.yml new file mode 100644 index 0000000..aef83ba --- /dev/null +++ b/roles/ess-pro/examples/group_vars-ess_servers.yml @@ -0,0 +1,92 @@ +# SPDX-License-Identifier: MIT-0 +--- +# inventory/group_vars/ess_servers.yml +# Public configuration for the ESS Pro deployment. All secrets are pulled +# from OpenBao at runtime — same pattern as bookstack/opnform/homarr. + +# ---- Matrix identity ---------------------------------------------------- +ess_pro_server_name: "digitalboard.ch" + +# Hostnames default to: +# matrix.digitalboard.ch (Synapse) +# mas.digitalboard.ch (Matrix Authentication Service) +# chat.digitalboard.ch (Element Web) +# admin.digitalboard.ch (Element Admin Panel) +# rtc.digitalboard.ch (Matrix RTC / LiveKit) +# `auth.digitalboard.ch` is intentionally avoided — Keycloak already owns it. + +# ---- DMZ Traefik terminates TLS ----------------------------------------- +ess_pro_tls_terminate_externally: true + +# ---- External Postgres -------------------------------------------------- +# Disable for first PoC iteration (uses chart-internal Postgres). +ess_pro_postgres_external: false +# ess_pro_postgres_host: "postgres.svc.digitalboard.ch" + +# ---- Delegated auth via the Digitalboard Keycloak ----------------------- +ess_pro_oidc_enabled: true +ess_pro_oidc_issuer: "https://auth.digitalboard.ch/realms/Digitalboard" +ess_pro_oidc_client_id: "ess-mas" +ess_pro_oidc_provider_name: "Digitalboard" + +# ---- Garage S3 media store ---------------------------------------------- +ess_pro_s3_media_enabled: true +ess_pro_s3_endpoint: "https://s3.digitalboard.ch" +ess_pro_s3_bucket: "ess-media" + +# ---- Matrix RTC / LiveKit ----------------------------------------------- +# Public-facing IP of the DMZ NAT so LiveKit publishes the right ICE +# candidates. Use the same address that the DMZ Traefik lives behind. +ess_pro_rtc_external_ip: "203.0.113.42" # placeholder — set for your env + +# ============================================================================= +# Secrets — sourced from OpenBao via community.hashi_vault, same as the +# other digitalboard.core roles. +# +# OpenBao paths (KV v2, mount `kv`): +# +# digitalboard/ess-pro +# ├── username (Element customer.element.io username) +# ├── token (Element customer.element.io token) +# ├── client_secret (Keycloak ess-mas OIDC client secret) +# ├── s3_access_key (Garage access key for ess-media bucket) +# ├── s3_secret_key (Garage secret key) +# ├── synapse_db_password (only if postgres_external: true) +# └── mas_db_password (only if postgres_external: true) +# +# Bootstrap once with: +# bao kv put kv/digitalboard/ess-pro \ +# username='ess-customer-xxx' \ +# token='paste-from-customer-portal' \ +# client_secret='from-keycloak' \ +# s3_access_key='...' s3_secret_key='...' +# ============================================================================= + +ess_pro_registry_username: "{{ lookup('community.hashi_vault.vault_kv2_get', + 'digitalboard/ess-pro', + mount_point='kv').data.data.username }}" + +ess_pro_registry_token: "{{ lookup('community.hashi_vault.vault_kv2_get', + 'digitalboard/ess-pro', + mount_point='kv').data.data.token }}" + +ess_pro_oidc_client_secret: "{{ lookup('community.hashi_vault.vault_kv2_get', + 'digitalboard/ess-pro', + mount_point='kv').data.data.client_secret }}" + +ess_pro_s3_access_key: "{{ lookup('community.hashi_vault.vault_kv2_get', + 'digitalboard/ess-pro', + mount_point='kv').data.data.s3_access_key }}" + +ess_pro_s3_secret_key: "{{ lookup('community.hashi_vault.vault_kv2_get', + 'digitalboard/ess-pro', + mount_point='kv').data.data.s3_secret_key }}" + +# Uncomment when ess_pro_postgres_external is true: +# ess_pro_postgres_synapse_password: "{{ lookup('community.hashi_vault.vault_kv2_get', +# 'digitalboard/ess-pro', +# mount_point='kv').data.data.synapse_db_password }}" +# +# ess_pro_postgres_mas_password: "{{ lookup('community.hashi_vault.vault_kv2_get', +# 'digitalboard/ess-pro', +# mount_point='kv').data.data.mas_db_password }}" diff --git a/roles/ess-pro/examples/openbao-bootstrap.sh b/roles/ess-pro/examples/openbao-bootstrap.sh new file mode 100755 index 0000000..9bfc9b4 --- /dev/null +++ b/roles/ess-pro/examples/openbao-bootstrap.sh @@ -0,0 +1,26 @@ +#!/usr/bin/env bash +# Bootstrap the OpenBao secret needed by the ess-pro Ansible role. +# Single KV v2 entry at kv/digitalboard/ess-pro with all keys flat +# (same layout as digitalboard/bookstack, digitalboard/opnform, etc.). +# +# Requires: `bao` CLI in PATH, `BAO_ADDR` exported, authenticated. + +set -euo pipefail + +MOUNT="${MOUNT:-kv}" +PATH_="${PATH_:-digitalboard/ess-pro}" + +read -p "Element registry username (from customer.element.io): " REG_USER +read -s -p "Element registry token: " REG_TOKEN; echo +read -s -p "Keycloak ess-mas client secret: " OIDC_SECRET; echo +read -p "Garage S3 access key: " S3_AK +read -s -p "Garage S3 secret key: " S3_SK; echo + +bao kv put "${MOUNT}/${PATH_}" \ + username="${REG_USER}" \ + token="${REG_TOKEN}" \ + client_secret="${OIDC_SECRET}" \ + s3_access_key="${S3_AK}" \ + s3_secret_key="${S3_SK}" + +echo "Done. Verify with: bao kv get ${MOUNT}/${PATH_}" diff --git a/roles/ess-pro/examples/site.yml b/roles/ess-pro/examples/site.yml new file mode 100644 index 0000000..28394ba --- /dev/null +++ b/roles/ess-pro/examples/site.yml @@ -0,0 +1,14 @@ +# SPDX-License-Identifier: MIT-0 +--- +# Example play, mirroring the convention used by the other digitalboard.core +# roles (digitalboard.core.bookstack, digitalboard.core.opnform, ...). +# +# Place the ess-pro role into your digitalboard.core collection alongside +# the others, then reference it as `digitalboard.core.ess_pro`. + +- name: Deploy Element Server Suite Pro + hosts: ess_servers + become: true + roles: + - digitalboard.core.k3s + - digitalboard.core.ess_pro diff --git a/roles/ess-pro/handlers/main.yml b/roles/ess-pro/handlers/main.yml new file mode 100644 index 0000000..1aff5a3 --- /dev/null +++ b/roles/ess-pro/handlers/main.yml @@ -0,0 +1,4 @@ +--- +# No handlers needed in normal operation — `kubernetes.core.helm` reconciles +# state declaratively. Kept as a placeholder for future hooks (e.g. restarting +# a sidecar Traefik when host names change). diff --git a/roles/ess-pro/meta/main.yml b/roles/ess-pro/meta/main.yml new file mode 100644 index 0000000..94848fe --- /dev/null +++ b/roles/ess-pro/meta/main.yml @@ -0,0 +1,18 @@ +--- +galaxy_info: + role_name: ess_pro + author: digitalboard + description: Deploy Element Server Suite Pro via its official Helm chart + license: MIT + min_ansible_version: "2.14" + platforms: + - name: Debian + versions: + - bookworm + +dependencies: + - role: k3s + +collections: + - kubernetes.core + - community.general diff --git a/roles/ess-pro/tasks/credentials.yml b/roles/ess-pro/tasks/credentials.yml new file mode 100644 index 0000000..fb9d367 --- /dev/null +++ b/roles/ess-pro/tasks/credentials.yml @@ -0,0 +1,41 @@ +# SPDX-License-Identifier: MIT-0 +--- +# Helm needs to authenticate against registry.element.io to pull both the +# matrix-stack chart AND the Pro container images. We do both: +# 1. `helm registry login` so the chart pull works. +# 2. A docker-registry Secret in the namespace so pods can pull images. + +- name: Log in to Element Helm/OCI registry + ansible.builtin.command: + cmd: >- + {{ ess_pro_helm_install_dir }}/helm registry login {{ ess_pro_registry_url }} + --username {{ ess_pro_registry_username | quote }} + --password-stdin + stdin: "{{ ess_pro_registry_token }}" + register: helm_login + changed_when: "'Login Succeeded' in (helm_login.stdout + helm_login.stderr)" + no_log: true + +- name: Create image pull Secret for the ESS namespace + kubernetes.core.k8s: + kubeconfig: "{{ ess_pro_kubeconfig }}" + state: present + definition: + apiVersion: v1 + kind: Secret + type: kubernetes.io/dockerconfigjson + metadata: + name: "{{ ess_pro_image_pull_secret_name }}" + namespace: "{{ ess_pro_namespace }}" + labels: + app.kubernetes.io/managed-by: ansible + data: + .dockerconfigjson: "{{ _dockerconfig | to_json | b64encode }}" + vars: + _dockerconfig: + auths: + "{{ ess_pro_registry_url }}": + username: "{{ ess_pro_registry_username }}" + password: "{{ ess_pro_registry_token }}" + auth: "{{ (ess_pro_registry_username ~ ':' ~ ess_pro_registry_token) | b64encode }}" + no_log: true diff --git a/roles/ess-pro/tasks/deploy.yml b/roles/ess-pro/tasks/deploy.yml new file mode 100644 index 0000000..be08eab --- /dev/null +++ b/roles/ess-pro/tasks/deploy.yml @@ -0,0 +1,63 @@ +--- +- name: Render ESS values.yaml + ansible.builtin.template: + src: values.yaml.j2 + dest: "{{ ess_pro_values_file }}" + owner: root + group: root + mode: "0640" + +- name: Deploy / upgrade ESS Pro Helm release + kubernetes.core.helm: + kubeconfig: "{{ ess_pro_kubeconfig }}" + name: "{{ ess_pro_release_name }}" + chart_ref: "{{ ess_pro_chart_ref }}" + chart_version: "{{ ess_pro_chart_version | default(omit, true) }}" + release_namespace: "{{ ess_pro_namespace }}" + create_namespace: false + values_files: + - "{{ ess_pro_values_file }}" + wait: "{{ ess_pro_helm_wait | bool }}" + wait_timeout: "{{ ess_pro_helm_timeout }}" + atomic: false + state: present + register: helm_release + +- name: Show release status + ansible.builtin.debug: + msg: "{{ helm_release.status | default('no status returned') }}" + when: helm_release is defined + +- name: Wait for Synapse pod to be Ready + kubernetes.core.k8s_info: + kubeconfig: "{{ ess_pro_kubeconfig }}" + kind: Pod + namespace: "{{ ess_pro_namespace }}" + label_selectors: + - "app.kubernetes.io/name=synapse" + register: synapse_pods + until: + - synapse_pods.resources | length > 0 + - synapse_pods.resources[0].status.containerStatuses is defined + - (synapse_pods.resources[0].status.containerStatuses | selectattr('ready', 'equalto', true) | list | length) > 0 + retries: 30 + delay: 10 + +- name: Fetch the localadmin bootstrap password (one-shot, only printed in verbose runs) + kubernetes.core.k8s_info: + kubeconfig: "{{ ess_pro_kubeconfig }}" + kind: Secret + namespace: "{{ ess_pro_namespace }}" + name: "{{ ess_pro_release_name }}-generated" + register: ess_generated_secret + when: ess_pro_create_initial_admin | bool + no_log: true + +- name: Show how to retrieve the localadmin password + ansible.builtin.debug: + msg: | + ESS Pro is up. To get the localadmin password: + kubectl -n {{ ess_pro_namespace }} get secrets/{{ ess_pro_release_name }}-generated \ + -o jsonpath='{.data.ADMIN_USER_PASSWORD}' | base64 -d + Login at https://{{ ess_pro_hostnames.element_admin }} as @localadmin:{{ ess_pro_server_name }} + when: ess_pro_create_initial_admin | bool diff --git a/roles/ess-pro/tasks/main.yml b/roles/ess-pro/tasks/main.yml new file mode 100644 index 0000000..463a84b --- /dev/null +++ b/roles/ess-pro/tasks/main.yml @@ -0,0 +1,51 @@ +# SPDX-License-Identifier: MIT-0 +--- +- name: Validate required variables + ansible.builtin.assert: + that: + - ess_pro_server_name | length > 0 + - ess_pro_registry_username | length > 0 + - ess_pro_registry_token | length > 0 + fail_msg: >- + ess_pro_server_name, ess_pro_registry_username and ess_pro_registry_token + must be set. Provide them in group_vars/ess_servers.yml (typically as + OpenBao lookups, following the digitalboard.core convention). + quiet: true + +- name: Validate OIDC variables when OIDC is enabled + ansible.builtin.assert: + that: + - ess_pro_oidc_issuer | length > 0 + - ess_pro_oidc_client_secret | length > 0 + fail_msg: ess_pro_oidc_issuer and ess_pro_oidc_client_secret must be set when OIDC is enabled. + quiet: true + when: ess_pro_oidc_enabled | bool + +- name: Validate S3 variables when S3 media is enabled + ansible.builtin.assert: + that: + - ess_pro_s3_endpoint | length > 0 + - ess_pro_s3_access_key | length > 0 + - ess_pro_s3_secret_key | length > 0 + fail_msg: S3 endpoint, access key and secret key must be set when S3 media is enabled. + quiet: true + when: ess_pro_s3_media_enabled | bool + +- name: Validate external Postgres variables + ansible.builtin.assert: + that: + - ess_pro_postgres_host | length > 0 + - ess_pro_postgres_synapse_password | length > 0 + - ess_pro_postgres_mas_password | length > 0 + fail_msg: External Postgres host and per-component passwords must be set when ess_pro_postgres_external is true. + quiet: true + when: ess_pro_postgres_external | bool + +- name: Run prerequisite tasks (Helm CLI, namespace) + ansible.builtin.import_tasks: prerequisites.yml + +- name: Authenticate against Element image registry and create pull secret + ansible.builtin.import_tasks: credentials.yml + +- name: Render values.yaml and deploy the Helm release + ansible.builtin.import_tasks: deploy.yml diff --git a/roles/ess-pro/tasks/prerequisites.yml b/roles/ess-pro/tasks/prerequisites.yml new file mode 100644 index 0000000..7be0b93 --- /dev/null +++ b/roles/ess-pro/tasks/prerequisites.yml @@ -0,0 +1,66 @@ +--- +- name: Ensure required OS packages are present + ansible.builtin.apt: + name: + - python3-kubernetes + - python3-yaml + - ca-certificates + - curl + state: present + update_cache: true + +- name: Check whether Helm is already installed + ansible.builtin.stat: + path: "{{ ess_pro_helm_install_dir }}/helm" + register: helm_binary + +- name: Check installed Helm version + ansible.builtin.command: "{{ ess_pro_helm_install_dir }}/helm version --short" + register: helm_version_check + changed_when: false + failed_when: false + when: helm_binary.stat.exists + +- name: Download Helm tarball + ansible.builtin.get_url: + url: "https://get.helm.sh/helm-{{ ess_pro_helm_version }}-linux-amd64.tar.gz" + dest: "/tmp/helm-{{ ess_pro_helm_version }}.tar.gz" + mode: "0644" + when: not helm_binary.stat.exists or (ess_pro_helm_version not in (helm_version_check.stdout | default(''))) + +- name: Unpack Helm + ansible.builtin.unarchive: + src: "/tmp/helm-{{ ess_pro_helm_version }}.tar.gz" + dest: /tmp/ + remote_src: true + creates: "/tmp/linux-amd64/helm" + when: not helm_binary.stat.exists or (ess_pro_helm_version not in (helm_version_check.stdout | default(''))) + +- name: Install Helm binary + ansible.builtin.copy: + src: /tmp/linux-amd64/helm + dest: "{{ ess_pro_helm_install_dir }}/helm" + remote_src: true + mode: "0755" + when: not helm_binary.stat.exists or (ess_pro_helm_version not in (helm_version_check.stdout | default(''))) + +- name: Ensure ESS config directory exists + ansible.builtin.file: + path: "{{ ess_pro_config_dir }}" + state: directory + mode: "0750" + owner: root + group: root + +- name: Ensure ESS namespace exists + kubernetes.core.k8s: + kubeconfig: "{{ ess_pro_kubeconfig }}" + state: present + definition: + apiVersion: v1 + kind: Namespace + metadata: + name: "{{ ess_pro_namespace }}" + labels: + app.kubernetes.io/managed-by: ansible + app.kubernetes.io/part-of: digitalboard diff --git a/roles/ess-pro/templates/values.yaml.j2 b/roles/ess-pro/templates/values.yaml.j2 new file mode 100644 index 0000000..8719354 --- /dev/null +++ b/roles/ess-pro/templates/values.yaml.j2 @@ -0,0 +1,208 @@ +--- +# ============================================================================= +# Element Server Suite Pro — values.yaml +# Rendered by Ansible role `ess-pro` for {{ inventory_hostname }}. +# DO NOT EDIT MANUALLY — re-run the playbook instead. +# ============================================================================= + +serverName: {{ ess_pro_server_name }} + +imagePullSecrets: + - name: {{ ess_pro_image_pull_secret_name }} + +# ----------------------------------------------------------------------------- +# Synapse (Matrix homeserver) +# ----------------------------------------------------------------------------- +synapse: + enabled: true + ingress: + host: {{ ess_pro_hostnames.synapse }} +{% if ess_pro_tls_terminate_externally %} + tlsSecret: "" + annotations: + # DMZ Traefik in front handles TLS termination. Inside K3s the workloads + # serve plain HTTP. Traefik (if used as IngressClass) will then route on + # host header only. + traefik.ingress.kubernetes.io/router.entrypoints: web +{% else %} + ingressClassName: {{ ess_pro_ingress_class }} +{% endif %} +{% if ess_pro_postgres_external %} + postgres: + host: {{ ess_pro_postgres_host }} + port: {{ ess_pro_postgres_port }} + sslMode: {{ ess_pro_postgres_sslmode }} + database: {{ ess_pro_postgres_synapse_db }} + user: {{ ess_pro_postgres_synapse_user }} + password: + value: {{ ess_pro_postgres_synapse_password }} +{% endif %} + additional: + digitalboard-config: + config: | + # Trust X-Forwarded-Proto from the upstream DMZ Traefik so Synapse + # generates HTTPS URLs (well-known, federation) correctly. + x_forwarded: true + admin_contact: "mailto:admin@{{ ess_pro_server_name }}" + # Registration is disabled by default in ESS Pro — keep it that way + # and provision users via Keycloak / mas-cli. + enable_registration: false +{% if ess_pro_s3_media_enabled %} + media_storage_providers: + - module: s3_storage_provider.S3StorageProviderBackend + store_local: true + store_remote: true + store_synchronous: true + config: + bucket: {{ ess_pro_s3_bucket }} + region_name: {{ ess_pro_s3_region }} + endpoint_url: {{ ess_pro_s3_endpoint }} + access_key_id: {{ ess_pro_s3_access_key }} + secret_access_key: {{ ess_pro_s3_secret_key }} +{% endif %} + +# ----------------------------------------------------------------------------- +# Matrix Authentication Service (MAS) — handles all login +# ----------------------------------------------------------------------------- +matrixAuthenticationService: + enabled: true + ingress: + host: {{ ess_pro_hostnames.mas }} +{% if ess_pro_tls_terminate_externally %} + annotations: + traefik.ingress.kubernetes.io/router.entrypoints: web +{% else %} + ingressClassName: {{ ess_pro_ingress_class }} +{% endif %} +{% if ess_pro_postgres_external %} + postgres: + host: {{ ess_pro_postgres_host }} + port: {{ ess_pro_postgres_port }} + sslMode: {{ ess_pro_postgres_sslmode }} + database: {{ ess_pro_postgres_mas_db }} + user: {{ ess_pro_postgres_mas_user }} + password: + value: {{ ess_pro_postgres_mas_password }} +{% endif %} +{% if ess_pro_oidc_enabled %} + additional: + digitalboard-oidc: + config: | + # Delegate login to Keycloak via OIDC. MAS becomes a thin RP that + # imports users on first login. Local accounts (localadmin) keep + # working alongside. + upstream_oauth2: + providers: + - id: "01J0KCK0DNNNDIGITALBOARDKC01" + human_name: "{{ ess_pro_oidc_provider_name }}" + issuer: "{{ ess_pro_oidc_issuer }}" + client_id: "{{ ess_pro_oidc_client_id }}" + client_secret: "{{ ess_pro_oidc_client_secret }}" + token_endpoint_auth_method: "client_secret_basic" + scope: "{{ ess_pro_oidc_scopes | join(' ') }}" + claims_imports: + subject: + template: "{% raw %}{{ user.sub }}{% endraw %}" + localpart: + action: require + template: "{% raw %}{{ user.preferred_username }}{% endraw %}" + displayname: + action: suggest + template: "{% raw %}{{ user.name }}{% endraw %}" + email: + action: suggest + template: "{% raw %}{{ user.email }}{% endraw %}" + set_email_verification: always +{% endif %} + +# ----------------------------------------------------------------------------- +# Element Web (web client) +# ----------------------------------------------------------------------------- +elementWeb: + enabled: true + ingress: + host: {{ ess_pro_hostnames.element_web }} +{% if ess_pro_tls_terminate_externally %} + annotations: + traefik.ingress.kubernetes.io/router.entrypoints: web +{% else %} + ingressClassName: {{ ess_pro_ingress_class }} +{% endif %} + +# ----------------------------------------------------------------------------- +# Element Admin (ESS Pro admin panel) +# ----------------------------------------------------------------------------- +elementAdmin: + enabled: true + ingress: + host: {{ ess_pro_hostnames.element_admin }} +{% if ess_pro_tls_terminate_externally %} + annotations: + traefik.ingress.kubernetes.io/router.entrypoints: web +{% else %} + ingressClassName: {{ ess_pro_ingress_class }} +{% endif %} + +# ----------------------------------------------------------------------------- +# Matrix RTC Backend (Element Call → LiveKit SFU + lk-jwt-service) +# This replaces the previous coturn + spreed-signaling + Janus stack. +# ----------------------------------------------------------------------------- +matrixRTC: + enabled: {{ ess_pro_rtc_enabled | bool }} + ingress: + host: {{ ess_pro_hostnames.matrix_rtc }} +{% if ess_pro_tls_terminate_externally %} + annotations: + traefik.ingress.kubernetes.io/router.entrypoints: web +{% else %} + ingressClassName: {{ ess_pro_ingress_class }} +{% endif %} +{% if ess_pro_rtc_enabled %} + sfu: + additional: | + # LiveKit configuration. UDP range must be opened on the DMZ firewall + # and forwarded to the K3s node IP. + rtc: + port_range_start: {{ ess_pro_rtc_udp_port_range_start }} + port_range_end: {{ ess_pro_rtc_udp_port_range_end }} + use_external_ip: false +{% if ess_pro_rtc_external_ip %} + node_ip: {{ ess_pro_rtc_external_ip }} +{% endif %} + tcp_port: 7881 +{% endif %} + +# ----------------------------------------------------------------------------- +# Well-known delegation — exposes /.well-known/matrix/{client,server} +# This is what makes federation and client discovery work for +# @user:{{ ess_pro_server_name }} when the homeserver lives on a subdomain. +# ----------------------------------------------------------------------------- +wellKnownDelegation: + enabled: true + ingress: + host: {{ ess_pro_server_name }} +{% if ess_pro_tls_terminate_externally %} + annotations: + traefik.ingress.kubernetes.io/router.entrypoints: web +{% else %} + ingressClassName: {{ ess_pro_ingress_class }} +{% endif %} + +# ----------------------------------------------------------------------------- +# Internal Postgres (only used when ess_pro_postgres_external is false) +# ----------------------------------------------------------------------------- +{% if not ess_pro_postgres_external %} +postgres: + enabled: true + # Demo-grade DB. Move to an external Postgres before going live. +{% else %} +postgres: + enabled: false +{% endif %} + +# ----------------------------------------------------------------------------- +# Initial admin (ESS chart creates @localadmin:{{ ess_pro_server_name }}) +# Password is in Secret "{{ ess_pro_release_name }}-generated", key ADMIN_USER_PASSWORD. +# ----------------------------------------------------------------------------- +initSecrets: + enabled: true diff --git a/roles/ess-pro/vars/main.yml b/roles/ess-pro/vars/main.yml new file mode 100644 index 0000000..d857295 --- /dev/null +++ b/roles/ess-pro/vars/main.yml @@ -0,0 +1,6 @@ +--- +# Internal computed paths and labels. Don't override in inventory. +ess_pro_kubeconfig: "/etc/rancher/k3s/k3s.yaml" +ess_pro_values_file: "{{ ess_pro_config_dir }}/values.yaml" +ess_pro_credentials_file: "{{ ess_pro_config_dir }}/ess-credentials.yaml" +ess_pro_image_pull_secret_name: "ess-image-pull" diff --git a/roles/k3s/README.md b/roles/k3s/README.md new file mode 100644 index 0000000..81e841a --- /dev/null +++ b/roles/k3s/README.md @@ -0,0 +1,29 @@ +# Role: k3s + +Installs a single-node K3s cluster on Debian bookworm. Used as the runtime for +the `ess-pro` role. + +## Design choices + +- **Traefik disabled inside K3s** because the project's DMZ Traefik already + fronts the cluster. Routing happens via NodePort/ClusterIP through the + external Traefik. If you want K3s' bundled Traefik as the ingress + controller, remove `traefik` from `k3s_disable_components` and adjust the + upstream Traefik to route by host headers only. +- **servicelb (Klipper) disabled** for the same reason — no LoadBalancer + services needed in the PoC. + +## Variables + +See `defaults/main.yml`. Override `k3s_version` to pin a specific K3s +release. The cluster/service CIDRs default to K3s' standard ranges; only +change if they clash with your libvirt networks. + +## Usage + +```yaml +- hosts: vdmzess01 + roles: + - role: k3s + - role: ess-pro +``` diff --git a/roles/k3s/defaults/main.yml b/roles/k3s/defaults/main.yml new file mode 100644 index 0000000..09d6931 --- /dev/null +++ b/roles/k3s/defaults/main.yml @@ -0,0 +1,27 @@ +--- +# K3s installation defaults +# See https://docs.k3s.io/installation/configuration for all options. + +k3s_version: "v1.31.5+k3s1" +k3s_install_script_url: "https://get.k3s.io" + +# Disable K3s' built-in Traefik because the project's DMZ Traefik is already +# in front and we don't want two competing ingress controllers. +# Also disable servicelb (Klipper) since we route via the K3s node IP directly. +k3s_disable_components: + - traefik + - servicelb + +# Bind kubeconfig readable for the deploy user (default vagrant). +# In production tighten this back to 600 and copy explicitly. +k3s_write_kubeconfig_mode: "0644" + +# Channel selection. Use stable for PoC, lock to k3s_version above for prod. +k3s_channel: "stable" + +# Cluster CIDRs (rarely need touching, set if conflicting with libvirt nets). +k3s_cluster_cidr: "10.42.0.0/16" +k3s_service_cidr: "10.43.0.0/16" + +# Extra args appended to INSTALL_K3S_EXEC. +k3s_extra_args: [] diff --git a/roles/k3s/handlers/main.yml b/roles/k3s/handlers/main.yml new file mode 100644 index 0000000..1556bb6 --- /dev/null +++ b/roles/k3s/handlers/main.yml @@ -0,0 +1,5 @@ +--- +- name: Restart k3s + ansible.builtin.systemd: + name: k3s + state: restarted diff --git a/roles/k3s/meta/main.yml b/roles/k3s/meta/main.yml new file mode 100644 index 0000000..463f15b --- /dev/null +++ b/roles/k3s/meta/main.yml @@ -0,0 +1,12 @@ +--- +galaxy_info: + role_name: k3s + author: digitalboard + description: Install single-node K3s suitable for hosting ESS Pro + license: MIT + min_ansible_version: "2.14" + platforms: + - name: Debian + versions: + - bookworm +dependencies: [] diff --git a/roles/k3s/tasks/main.yml b/roles/k3s/tasks/main.yml new file mode 100644 index 0000000..c0871de --- /dev/null +++ b/roles/k3s/tasks/main.yml @@ -0,0 +1,86 @@ +--- +# Install K3s as a single-node Kubernetes cluster. +# This role is intentionally minimal: it installs K3s, waits for the API, +# and makes kubectl + the kubeconfig usable for the downstream ess-pro role. + +- name: Check whether K3s is already installed + ansible.builtin.stat: + path: /usr/local/bin/k3s + register: k3s_binary + +- name: Ensure curl is installed + ansible.builtin.apt: + name: curl + state: present + update_cache: true + when: not k3s_binary.stat.exists + +- name: Download K3s install script + ansible.builtin.get_url: + url: "{{ k3s_install_script_url }}" + dest: /tmp/k3s-install.sh + mode: "0755" + when: not k3s_binary.stat.exists + +- name: Build INSTALL_K3S_EXEC string + ansible.builtin.set_fact: + k3s_exec_args: >- + {{ + (['--write-kubeconfig-mode=' ~ k3s_write_kubeconfig_mode] + + (k3s_disable_components | map('regex_replace', '^(.*)$', '--disable=\\1') | list) + + ['--cluster-cidr=' ~ k3s_cluster_cidr, + '--service-cidr=' ~ k3s_service_cidr] + + k3s_extra_args) | join(' ') + }} + +- name: Install K3s + ansible.builtin.command: + cmd: /tmp/k3s-install.sh + environment: + INSTALL_K3S_VERSION: "{{ k3s_version }}" + INSTALL_K3S_CHANNEL: "{{ k3s_channel }}" + INSTALL_K3S_EXEC: "{{ k3s_exec_args }}" + args: + creates: /usr/local/bin/k3s + notify: Restart k3s + +- name: Ensure k3s service is started and enabled + ansible.builtin.systemd: + name: k3s + state: started + enabled: true + +- name: Wait for kubeconfig to appear + ansible.builtin.wait_for: + path: /etc/rancher/k3s/k3s.yaml + state: present + timeout: 60 + +- name: Wait for Kubernetes API to respond + ansible.builtin.command: kubectl --kubeconfig /etc/rancher/k3s/k3s.yaml get --raw=/readyz + register: k3s_ready + retries: 30 + delay: 5 + until: k3s_ready.rc == 0 + changed_when: false + +- name: Create symlink for kubectl + ansible.builtin.file: + src: /usr/local/bin/k3s + dest: /usr/local/bin/kubectl + state: link + force: false + failed_when: false + +- name: Ensure ~/.kube exists for root + ansible.builtin.file: + path: /root/.kube + state: directory + mode: "0700" + +- name: Provide kubeconfig at /root/.kube/config + ansible.builtin.copy: + src: /etc/rancher/k3s/k3s.yaml + dest: /root/.kube/config + remote_src: true + mode: "0600"