Digitalboard/digitalboard.core
6
2
Fork 0
Code Pull requests 1 Releases Packages Activity Actions

feat: services bundle and collection documentation #8

Merged
Tobias-Wuest merged 14 commits from feat/services-bundle-and-docs into main 2026-06-04 09:17:43 +00:00
Conversation 0 Commits 14 Files changed 86 +3100 -836
Simon commented 2026-05-27 21:11:43 +00:00
Owner
Copy link

Bundles the self-hosted service roles from the GymBurgdorf PoC and brings the whole collection's documentation up to standard.

Service roles

Multi-domain routing, split-horizon host rewrites, OIDC bootstrap/hardening, optional authentik ForwardAuth, idempotent provisioning and ACME-DNS knobs across the deployed services.

Collection documentation (latest commit)

Multi-agent review of every role README against its defaults, tasks and templates. Replaces ansible-galaxy init placeholders and corrects docs that had drifted from the code.

  • README: role table for all 16 roles, requirements, role-ordering
  • galaxy.yml: declare community.docker / community.general deps; real description, tags, urls
  • meta/runtime.yml: requires_ansible '>=2.15.0'
  • plugins/README: document homarr_layout filter + garage_credentials lookup
  • per-role meta/main.yml and README for all placeholder roles

Correctness fixes found during review:

  • keycloak: wrong domain default, drop invented keycloak_cert_resolver, document the provisioning feature
  • garage: root_domain is .s3., not the bare domain
  • opnform: jwt/front_api secrets use openssl rand -hex 32 (incl. validation fail_msg)
  • send: S3 example references garage_s3_domains[0] (was singular)
  • opencloud: document required opencloud_wopi_domain

License normalized to MIT-0 across galaxy.yml, role meta and READMEs to match the SPDX headers.

Bundles the self-hosted service roles from the GymBurgdorf PoC and brings the whole collection's documentation up to standard. ## Service roles Multi-domain routing, split-horizon host rewrites, OIDC bootstrap/hardening, optional authentik ForwardAuth, idempotent provisioning and ACME-DNS knobs across the deployed services. ## Collection documentation (latest commit) Multi-agent review of every role README against its defaults, tasks and templates. Replaces ansible-galaxy init placeholders and corrects docs that had drifted from the code. - README: role table for all 16 roles, requirements, role-ordering - galaxy.yml: declare community.docker / community.general deps; real description, tags, urls - meta/runtime.yml: requires_ansible '>=2.15.0' - plugins/README: document homarr_layout filter + garage_credentials lookup - per-role meta/main.yml and README for all placeholder roles Correctness fixes found during review: - keycloak: wrong domain default, drop invented keycloak_cert_resolver, document the provisioning feature - garage: root_domain is .s3.<first-entry>, not the bare domain - opnform: jwt/front_api secrets use openssl rand -hex 32 (incl. validation fail_msg) - send: S3 example references garage_s3_domains[0] (was singular) - opencloud: document required opencloud_wopi_domain License normalized to MIT-0 across galaxy.yml, role meta and READMEs to match the SPDX headers.
Simon added 24 commits 2026-05-27 21:11:44 +00:00 
- Refactor: collapse `*_domain` + `*_extra_domains` into a single
  `*_domains` list across authentik, collabora, garage and nextcloud
  roles. First entry is the canonical FQDN (used for OVERWRITEHOST,
  BASE_URL, notify_push setup and garage root_domain).
- Authentik blueprint: guard the OAuth sources block so an empty
  `authentik_login_sources` no longer renders an invalid YAML key.
- Nextcloud: introduce `nextcloud_collabora_public_domain` and set
  Collabora's `public_wopi_url` separately from the server-to-server
  `wopi_url` so browsers can reach Collabora via the public name while
  Nextcloud still talks to it on the internal one.
- Nextcloud: URL-encode the postgres user/password in DATABASE_URL.
- Drop `recreate: always` from collabora/drawio/homarr/opencloud/traefik
  handlers and the authentik_outpost_ldap start task. `up -d` with
  `state: present` already recreates exactly the services whose
  compose definition changed; the blanket recreate was forcing
  restarts even when nothing relevant moved.
- Rewrite the `*_domains` Traefik Host loop to the `Host(\`a\`) ||
  Host(\`b\`)` form across authentik/collabora/garage/nextcloud so the
  rule still matches when traefik can't normalize the comma-form into
  the same canonical shape.
- Traefik: add `traefik_acme_tcp_only` (sets LEGO_EXPERIMENTAL_DNS_TCP_ONLY)
  and `traefik_acme_disable_ans_checks` (disables lego's authoritative-NS
  propagation check) for environments where the DNS path between the
  traefik container and the zone's nameservers is constrained.
- Traefik DMZ collector: two-step merge so a `traefik_dmz_exposed_services`
  entry that sets its own `backend_host` wins over the host fallback;
  lets a route target an internal FQDN covered by the backend cert's
  SANs instead of the raw IP.
- Nextcloud: add `nextcloud_notify_push_domain` override for the
  `occ notify_push:setup` call so the setup check can hit an internal
  FQDN instead of hairpinning through the DMZ. Push router now matches
  every entry in `nextcloud_domains`.
- Nextcloud: also %2F-escape slashes in the postgres user/password
  inside the notify_push DATABASE_URL.
Add `traefik_extra_hosts` (list of `host:ip`) that maps straight into
the traefik container's compose `extra_hosts`. Needed when a downstream
middleware (e.g. ForwardAuth to authentik on a sibling LAN) has to
resolve a public FQDN to an internal IP because the DMZ doesn't hairpin
the public address back inside.

Empty by default; behaviour unchanged for existing inventories.
* `authentik_host_rewrite_domains`: extra hostnames that reach the
  authentik container but make it generate URLs (OIDC issuer, reset
  links) as if requested from the canonical `authentik_domains[0]`.
  Each entry gets its own traefik router and a URL-based loadbalancer
  service that disables passHostHeader and pins X-Forwarded-Host via
  middleware, so server-to-server calls on internal FQDNs keep traffic
  in the LAN while the iss claim stays aligned with the public host.
  Uses a network alias on the canonical FQDN so traefik (sharing the
  network) resolves the URL upstream to this very container.

* proxy-app blueprint:
  - `mode` (default `forward_single`) lets callers pick between proxy,
    forward_single and forward_domain providers in one template.
  - `allowed_groups`: when set, emit one PolicyBinding per group on
    the application; authentik OR-evaluates bindings, so users in any
    listed group pass and others are denied.

Existing inventories with an empty list see no behavioural change.
Add `*_authentik_forward_auth` + `*_authentik_forward_auth_url` knobs to
both roles. When enabled:

* drawio: traefik attaches a ForwardAuth middleware pointing at the
  authentik embedded outpost; unauthenticated requests get redirected
  to log in and downstream sees X-Authentik-* identity headers.

* garage WebUI: same ForwardAuth wiring, and `AUTH_USER_PASS` is dropped
  from the container env so authentik is the only gate. Tasks now key
  the htpasswd hash workflow off `_garage_webui_htpasswd_active`
  (`webui_enabled AND NOT authentik_forward_auth`); when authentik
  fronts the UI we skip hashing entirely. htpasswd hash is also now
  cached on disk and re-verified via `htpasswd -vbB` so unchanged
  passwords stop showing as `changed=true` on every run.

Both knobs default to `false`, preserving existing htpasswd/plain behaviour.
* bootstrap: `garage layout show` truncates node IDs to 16 chars, but
  the membership check compared against the full hex. After the first
  successful join, subsequent runs no longer found the short ID in
  `layout show` and re-issued `layout assign`, marking the task
  changed every time. Compare against both the truncated and the full
  form so a configured node stays detected. Also tag the read-only
  `garage node id` / `layout show` probes with `changed_when: false`.

* provision keys: the old parser sliced `stdout_lines[1:]` to drop the
  header but missed that INFO log lines and ANSI escapes can interleave
  with table rows. Replace with an explicit `^GK[0-9a-fA-F]+` filter
  after stripping ANSI, so probe-output noise no longer corrupts the
  existing-keys set and triggers spurious `key new` calls.

* provision buckets: same class of fix — match `^[0-9a-f]{16}\s` data
  rows instead of slicing `[2:]`, which broke when the table header
  wasn't exactly two lines.

* provision permissions: pre-read `bucket info` for each (key, bucket)
  pair and only run `bucket allow` when the current `RWO` flag set for
  that key ID doesn't already match the desired permissions. Previously
  `bucket allow` ran unconditionally and reported changed every play.

* `changed_when: false` on all read-only probes (`key list`, `key info`,
  `bucket list`).
Every `occ config:app:set` / `ldap:set-config` / `notify_push:setup`
call previously fired on every play, marking changed even when the
stored value already matched. Now we read the current value first and
only invoke the setter when it differs:

* richdocuments (collabora): pre-read wopi_url, public_wopi_url,
  disable_certificate_verification, wopi_allowlist into a fact map;
  guard each `config:app:set` and tag `richdocuments:activate-config`
  with `changed_when: false` since it's a discovery refresh.

* drawio: same pattern for DrawioUrl, DrawioTheme, DrawioOffline,
  comparing as strings (occ stores booleans as "1"/"0").

* user_ldap: pre-read `ldap:show-config s01 --output=json`, parse JSON
  defensively (occ logs interleave on stderr), and skip per-key
  `ldap:set-config` calls when the stored value already equals the
  desired one.

* notify_push: skip `notify_push:setup` when the stored base_endpoint
  already matches the computed URL.

* plugins: `app:install`/`app:enable` were treating "already installed/
  enabled" output as a change. Add the negative match to `changed_when`
  so re-runs of a fully-provisioned site report ok rather than changed.
nextcloud/server#59629: under PHP 8.x with OPcache,
UserConfig::getValueBool() passes a non-string from getTypedValue()
straight into strtolower(), throwing a TypeError on every authenticated
request once user_ldap is involved. Fix landed in master (PR #59646)
but no stable33 backport made it into 33.0.4.

Discover all compose-managed nextcloud containers, check whether the
`strtolower((string)` cast is already present, and `sed` it into
`lib/private/Config/UserConfig.php` on the ones that still ship the
broken version. Idempotent via grep guard so re-runs are no-ops.

Remove this block once the deployed image >= 33.0.4 ships the upstream fix.
Add drawio_extra_domains (list, default empty). The traefik Host rule
on the drawio router now expands to Host(<canonical>) || Host(<extra>)
... so the same container can answer on additional FQDNs — e.g. an
internal *.int.* name so a DMZ reverse-proxy can reach drawio via a
backend hostname covered by the local traefik cert.

Empty by default; behaviour unchanged for existing inventories.
Each of the five roles touched in this branch now ships:

* meta/argument_specs.yml: typed schema for every variable in
  defaults/main.yml plus the optional inputs surfaced via this
  branch (traefik_extra_hosts, authentik_host_rewrite_domains,
  authentik_proxy_apps.mode / .allowed_groups, drawio_extra_domains,
  drawio_authentik_forward_auth*, garage_webui_authentik_forward_auth*).
  All five specs load cleanly through ansible-core's
  ArgumentSpecValidator.

* README.md: replaces the ansible-galaxy boilerplate (where it was
  still in place) with a focused write-up — service vars, required
  secrets, ForwardAuth/idempotency notes, dependencies, and a working
  example playbook. authentik and garage READMEs are rewritten to cover
  the new knobs while preserving their existing content.
Rename roles/OpnForm → roles/opnform so the role resolves as
  digitalboard.core.opnform (Ansible collection convention is
  lowercase). Update tests/test.yml reference accordingly.

  Add automated admin user creation via POST /api/register, gated on
  opnform_admin_email + opnform_admin_password. Idempotent through a
  prior login probe. Without these vars the manual setup page flow is
  preserved.

  Add automated OIDC IdentityConnection setup via the per-workspace
  /api/open/workspaces/{id}/oidc-connections endpoint, gated on
  opnform_oidc_enabled. Hard-coupled to the admin bootstrap (the API
  requires an authenticated admin token); validation block fails fast
  if OIDC is enabled without admin credentials. Supports both an
  explicit opnform_oidc_group_role_mappings list and a fallback
  opnform_oidc_admin_group convenience var.

  Convert opnform_oidc_scopes from space-separated string to YAML list
  to match OpnForm's API expectation. Rewrite README "First login" and
  "OIDC setup" sections to reflect that self-hosted OpnForm does not
  ship a pre-seeded admin and to document the new bootstrap paths.
  BREAKING CHANGE: opnform_oidc_scopes changed from space-separated
  string to YAML list. Inventories that override it must update from
  "openid profile email" to [openid, profile, email].
* vars/main.yml: header was 'vars file for homarr' (copy-paste from the
  homarr role). Fixed to 'vars file for opnform'. File body is empty.
* meta/main.yml: replace ansible-galaxy init boilerplate with real
  metadata — author, description, license (MIT-0), min_ansible_version
  set to '2.15' as a string (galaxy schema requires str), galaxy_tags
  for discovery, and an empty dependencies list.

The third inline finding (dead roles/opnform/templates/compose.yml.j2)
is resolved by dropping the WIP commit a6f301e during the rebase rather
than removing it in a separate commit — the file no longer exists in
the rebased history.
opnform_app_key, opnform_jwt_secret, opnform_front_api_secret and
opnform_db_password shipped as real base64 strings in defaults — they
look like production secrets that just happen to be public. Set all
four to '' and rely on the existing Validate task (and the new
argument_specs marking them required) to fail fast when an inventory
forgets to override them.

Mirror the docstring comment to show how to generate each one with
openssl.
50 typed options covering the full defaults file plus the OIDC subschema
(group_role_mappings with idp_group + role choices). Required secrets
(app_key, jwt_secret, front_api_secret, db_password) marked
required: true so ansible refuses the play with a clear error before
the validate task even runs.

Loads cleanly through ansible-core's ArgumentSpecValidator. Matches the
spec convention introduced for traefik, authentik, drawio, garage and
nextcloud.
Deploy BookStack with linuxserver.io images behind Traefik, including
Entra ID OIDC SSO support and a daily backup timer.

Stack:
- lscr.io/linuxserver/bookstack:version-v26.03.3
- lscr.io/linuxserver/mariadb:11.4.9
- Traefik labels for websecure entrypoint on internal network
- Healthcheck via mariadb-admin ping (LSIO image lacks healthcheck.sh)

Features:
- Persistent APP_KEY generated on first run, stored in volume dir
- Optional OIDC SSO via Microsoft Entra ID (configurable per-instance)
- Idempotent admin user creation with DB-based existence check
- Daily systemd timer backup (DB dump + uploads tar + APP_KEY)
  with configurable retention

Implementation notes:
- DB queries use --protocol=tcp with the app user because root@localhost
  uses unix_socket auth in the LSIO MariaDB image (no password) and
  root@% does not exist
- docker_container_exec uses argv: (list) instead of command: (string)
  to avoid argument-splitting issues
- Migration-wait task ensures users table exists before admin check,
  since /login returns 200 before Laravel migrations complete
- no_log: true on all tasks that reference DB or admin passwords
- artisan absolute path (/app/www/artisan) because LSIO image WORKDIR
  is not the app directory

Adds bookstack route to DMZ Traefik service registry.
47 typed options covering the full defaults file plus the OIDC and
backup-timer subsystems. The three secrets the role asserts on
(db_root_password, db_password, admin_password) are marked
required: true so ansible refuses the play with a clear error before
the validate task even runs.

Loads cleanly through ansible-core's ArgumentSpecValidator with 100%
defaults/spec coverage. Matches the spec convention used by traefik,
authentik, drawio, garage, nextcloud, opnform, coturn, talk and send.
Deploys timvisee/send with a Redis backend behind Traefik. Supports
local-disk or S3 storage (e.g. via the garage role). Uses the shared
`*_domains` list convention so the router can accept internal *.int.*
names alongside the canonical BASE_URL host.
* tasks/main.yml: prefix all builtin modules with ansible.builtin
  (file, template) — silences ansible-lint fqcn[action-core] and
  matches the convention used by the other roles in this collection.

* meta/main.yml: change min_ansible_version from the float 2.14 to
  the string '2.14'. ansible-galaxy's schema requires a string here
  (ansible-lint schema[meta] complains otherwise — same fix I just
  applied to the opnform role).
The router rule joined send_domains with ', ' which is the v2 syntax
('Host(`a`, `b`)'). Traefik v3 expects each Host() to be its own
matcher joined with the explicit '||' OR operator. With v3 the comma
form is silently ignored — only the first host actually matches.

Match the pattern already used in the authentik, drawio and nextcloud
roles in this collection.
29 typed options with full defaults coverage (no required: true marks —
the role works with an empty S3 config when storage_backend=local).
Documents the send_domains list convention, the local-vs-s3 storage
choice, the timing/size limits and the Traefik / network wiring.

Loads through ansible-core's ArgumentSpecValidator. Matches the spec
convention used by the other roles in this collection.
When send_storage_backend=s3 the role previously deployed the container
with whatever was in send_s3_* (often empty strings from the defaults).
The container would then start, accept uploads, and fail to persist
anything silently. Same pattern as the validate blocks in coturn,
talk, bookstack and opnform: fail fast at task time with a clear error
that points at the four missing variables.

Skipped entirely when send_storage_backend=local (the default).
Bundle of cross-role changes for the gymb services deployment:

- Traefik routers: OR-combine opnform/homarr/bookstack Host rules with new
  *_extra_domains (internal *.int.* FQDNs for a DMZ reverseproxy), and emit
  tls.certresolver only when traefik_cert_mode == acme (drawio, homarr,
  opnform, send).
- Split-horizon: bookstack_extra_hosts / opnform_extra_hosts add container
  /etc/hosts overrides so containers reach the IdP public FQDN over the LAN.
- bookstack: assert the OIDC issuer resolves concretely (reject "//v2.0"),
  allowing non-Entra IdPs that override bookstack_oidc_issuer.
- homarr: derive the bcrypt salt from the password digest so the admin hash
  is idempotent — no spurious template changes / container restarts.
- opnform: PATCH an existing OIDC connection instead of skipping (applies
  corrected inventory on re-run); add OIDC_FORCE_LOGIN (enabled only after
  bootstrap) and an optional direct-SSO ingress entrypoint.

Docs: READMEs and meta/argument_specs.yml updated for all new variables.
Replace ansible-galaxy init placeholders across the collection and
correct documentation that drifted from the code, after a multi-agent
review of every role README against its defaults, tasks and templates.

Collection level:
- README: role table for all 16 roles, requirements and role-ordering
- galaxy.yml: declare community.docker and community.general deps,
  real description/tags/urls; normalize license to MIT-0
- meta/runtime.yml: requires_ansible '>=2.15.0'
- plugins/README: document the homarr_layout filter and
  garage_credentials lookup instead of scaffold boilerplate

Per-role meta/main.yml and README for the placeholder roles
(389ds, authentik, authentik_outpost_ldap, base, collabora, drawio,
garage, homarr, httpbin, keycloak, nextcloud, opencloud, traefik).

Correctness fixes found during review:
- keycloak: wrong domain default, drop invented keycloak_cert_resolver,
  document the provisioning feature
- garage: root_domain is .s3.<first-entry>, not the bare domain
- opnform: jwt/front_api secrets use `openssl rand -hex 32`; align the
  validation fail_msg in tasks/main.yml accordingly
- send: S3 example references garage_s3_domains[0] (was singular)
- opencloud: document required opencloud_wopi_domain

License normalized to MIT-0 across galaxy.yml, role meta and READMEs to
match the SPDX headers.
Simon changed title from test to feat: services bundle and collection documentation 2026-05-27 21:16:23 +00:00
Simon force-pushed feat/services-bundle-and-docs from 03bf0efe44 to 3236ca332f 2026-05-27 21:18:32 +00:00 Compare
requested review from Tobias-Wuest 2026-05-28 09:18:32 +00:00
Simon added 1 commit 2026-06-02 12:00:24 +00:00 
- authentik: address the rewrite service by compose service name instead
  of a network alias on the public FQDN, which shadowed extra_hosts pins
  and broke OIDC discovery for c-ares-based (Node) resolvers
- homarr: add homarr_extra_hosts to pin the IdP FQDN to a LAN IP so OIDC
  discovery stays in-network while the issuer matches the browser-facing URL
- opnform: add opnform_oidc_sso_redirect_root to 302 the root URL to the
  SSO path (deep-links untouched, /login?bypass=1 break-glass); restart
  ingress via container restart so envsubst re-renders nginx.conf
- nextcloud: make the UserConfig sed workaround fail loud on upstream
  drift instead of silently skipping (nextcloud/server#59629)
- gitignore: exclude the local .ansible/ collection cache
Simon added 1 commit 2026-06-02 15:05:47 +00:00 
The api service now also receives FRONT_API_SECRET so AuthenticateJWT
accepts the UI's server-side JWT forwards instead of blacklisting them
on UA mismatch. On the ui service the var is renamed FRONT_API_SECRET ->
NUXT_API_SECRET so Nuxt's runtimeConfig.apiSecret is actually populated
(NUXT_<key> convention) and injected as x-api-secret, short-circuiting
the UA-fingerprint check that otherwise 401s every reload.
Tobias-Wuest merged commit a8954f525c into main 2026-06-04 09:17:43 +00:00
Tobias-Wuest deleted branch feat/services-bundle-and-docs 2026-06-04 09:17:44 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Tobias-Wuest
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Digitalboard/digitalboard.core#8
No description provided.
Delete branch "feat/services-bundle-and-docs"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?