Skip to content

Operations and scaling

Hodor is a self-contained .NET 10 web application that self-hosts via Kestrel on port 5200 and serves the pre-built Angular app from the same process. It can therefore run in a container (OpenShift/Kubernetes), as a Linux service, or on a Windows server — with no code changes. The metadata database is always an external SQL Server or PostgreSQL.

Only three things make up a run's state:

State Where it lives Must survive a restart
Schedules, run history, run locks Metadata database (SQL Server / Postgres) Yes — external database
Pipelines (.hodor) Hodor:PipelinesFolder (volume /data) Yes
Data Protection keys Hodor:KeysPath (volume /data) Yes — otherwise encrypted secrets are lost

What happens if the process/pod dies?

Nothing is lost, and an orchestrator (OpenShift, systemd, Windows) restarts it automatically.

  • Schedules survive. The database is the source of truth. On startup Hodor reads all enabled cron schedules back from the database and re-registers them.
  • Missed runs are caught up. If the process was down when a nightly load was due, one catch-up run is triggered on startup (comparing the last run against the cron). Disable with Hodor:Scheduler:CatchUpMissedRuns=false.
  • Interrupted runs are reconciled. A run left on Running when the process died is marked Interrupted and its run lock is cleared, so the pipeline is immediately startable again. Loads run in a transaction with rollback — no half-loaded data, just an interrupted run you can restart.
  • Health probes. /healthz (liveness) and /readyz (readiness, checks the database connection). Use them as probes so traffic isn't routed in until the app is ready.

Scaling in OpenShift / Kubernetes

A full Helm chart is available under charts/hodor.

One replica (default)

The defaults run one replica with a ReadWriteOnce PVC for /data and the Recreate deployment strategy. This is the recommended setup for most organisations: one instance per organisation.

The Recreate strategy causes brief downtime on deploy

Because the /data volume is ReadWriteOnce, the old pod must terminate before the new one can mount the volume. An update therefore involves a short pause — the price of a single-writer volume.

Multiple replicas

The application is built for multiple replicas: a RunLocks table with heartbeats in the database ensures two pods never double-run the same pipeline (one takes the lock, the other skips the run). A dead lock is cleared once its heartbeat goes stale, so a pipeline never stays stuck locked.

To actually run more than one replica, /data (pipelines + Data Protection keys) must be moved to a ReadWriteMany volume so all pods can mount it at once:

# values.yaml
replicaCount: 2
persistence:
  accessModes:
    - ReadWriteMany
  storageClassName: <an RWX class, e.g. NFS/CephFS>

The health probes (/healthz, /readyz) plus a RollingUpdate strategy then give zero-downtime deploys, since multiple pods can mount the volume at once.


Deploying without a container (Linux, systemd)

No Docker required — publish and run as an ordinary service:

# on the build machine
dotnet publish Hodor.Api -c Release -o ./publish

Copy ./publish to the server and run it as a systemd service:

# /etc/systemd/system/hodor.service
[Unit]
Description=Hodor ETL
After=network.target

[Service]
WorkingDirectory=/opt/hodor
ExecStart=/usr/bin/dotnet /opt/hodor/Hodor.Api.dll
Restart=always
RestartSec=5
User=hodor
Environment=ASPNETCORE_URLS=http://127.0.0.1:5200
Environment=Database__Provider=Postgres
Environment=Database__ConnectionString=Host=...;Database=hodor;Username=hodor;Password=...
Environment=Jwt__Key=at-least-32-characters-secret-key
Environment=Hodor__KeysPath=/var/lib/hodor/keys

[Install]
WantedBy=multi-user.target
sudo systemctl enable --now hodor

Put a reverse proxy (nginx, Traefik) in front of port 5200 for HTTPS/TLS. systemd restarts the service automatically if it dies (Restart=always).

Avoid installing the .NET runtime on the server

Publish self-contained to bundle the runtime into the folder: dotnet publish Hodor.Api -c Release --self-contained -r linux-x64 -o ./publish


Windows Server

One of Hodor's best fits, since it already recommends Windows authentication against SQL Server (Trusted_Connection=True) — .hodor files then contain no secrets at all. Two standard paths:

  1. Install the ASP.NET Core Hosting Bundle (.NET 10) on the server.
  2. dotnet publish Hodor.Api -c Release -o C:\inetpub\hodor.
  3. Create an IIS site pointing at the publish folder. IIS becomes the reverse proxy to Kestrel (via AspNetCoreModuleV2) and handles TLS/certificates in Windows.
  4. Set configuration (database, Jwt:Key, Hodor:KeysPath) via environment variables on the app pool or in appsettings.Production.json.

Windows Service

dotnet publish Hodor.Api -c Release -o C:\hodor
New-Service -Name Hodor -BinaryPathName "C:\Program Files\dotnet\dotnet.exe C:\hodor\Hodor.Api.dll" -StartupType Automatic
Start-Service Hodor

Kestrel listens on port 5200; put IIS/ARR or the firewall in front for TLS.

SQL Server on the same box

A SQL Server on the same Windows server (or domain) is a natural combination — run the Hodor service under a service account with database access and use Trusted_Connection=True in the connection string.