> For the complete documentation index, see [llms.txt](https://docs.syntho.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.syntho.ai/deploy-syntho/deploy-syntho-using-kubernetes.md). # Kubernetes (Helm) Deploy Syntho on Kubernetes using Helm charts.
What Helm deploys This deployment typically includes: * Frontend (UI) * Backend * Core API * Ray (separate chart) * PostgreSQL (metadata, optional if you use an external DB) * Redis (optional if you use an external Redis)
Ray sizing and storage (optional) Ray capacity depends on your data size and throughput goals. If you didn’t get sizing guidance yet, ask Syntho Support. Also ensure your cluster can provision **shared (RWX) storage** for Ray workers when the chart enables shared volumes. If your StorageClass does not support `ReadWriteMany`, Ray pods can fail to schedule or start. OpenShift / CRI-O note: some clusters default to low process limits. Ray can hit those limits when scaling.
Offline deployment Offline Kubernetes deployments follow the same steps below. You just need to stage artifacts on a connected machine first. 1. **Stage Helm charts** * Download the Helm chart bundles (or clone `deployment-tools`). * Transfer the chart bundles to the offline environment where you run `helm upgrade --install`. 2. **Stage container images** Your cluster nodes must be able to fetch images. * **Recommended:** mirror images into an internal registry reachable by the cluster nodes. * **Alternative:** preload images onto every node (works, but is harder to operate). After mirroring, update image references in both `helm/ray/values.yaml` and `helm/syntho-ui/values.yaml` to point to your internal registry (and keep using a specific version tag). 3. **Registry authentication step changes** * If you use an **internal registry**, create the ImagePullSecret for that registry and reference it via `imagePullSecrets`. * If you **preload images** on nodes, ensure the chart uses an `imagePullPolicy` that doesn’t force pulls (commonly `IfNotPresent`). {% hint style="info" %} Upgrades are the same procedure. Repeat the staging step for the new `APPLICATION_VERSION`. {% endhint %}
### Deploy {% stepper %} {% step %} **Prerequisites** Make sure you meet the [prerequisites](/deploy-syntho/deploy-syntho-using-kubernetes/prerequisites.md) before deploying. {% endstep %} {% step %} **Get the Helm charts** Use the `deployment-tools` repository: * [Ray Helm chart](https://github.com/syntho-ai/deployment-tools/tree/main/helm/ray) * [Syntho Helm chart](https://github.com/syntho-ai/deployment-tools/tree/main/helm/syntho-ui) If you prefer release artifacts, use the `deployment-tools` releases matching the version that you are deploying: * Download: * `ray-helm-chart.tar.gz` * `syntho-ui-helm-chart.tar.gz` {% endstep %} {% step %} **Create a namespace** ```bash kubectl create namespace syntho ``` {% endstep %} {% step %} **Create an ImagePullSecret** ```bash kubectl create secret docker-registry syntho-cr-secret \ --namespace syntho \ --docker-server= \ --docker-username= \ --docker-password= ``` In both charts, reference it: ```yaml imagePullSecrets: - name: syntho-cr-secret ``` {% hint style="info" %} Ensure that `syntho.azurecr.io` is added to your firewall allowlist to enable image pulls {% endhint %} {% endstep %} {% step %} **Deploy Ray** Ray provides distributed execution for heavy jobs. The repository's `helm/ray/chart/` contains the chart, including the KubeRay operator and RayCluster dependencies. `helm/ray/crds/` contains the corresponding RayCluster, RayJob, and RayService CustomResourceDefinition manifests for cluster administrators. Use the CRDs supplied with your chart version; CRD installation requires cluster-level permissions. Minimal settings in your Ray values file (commonly `helm/ray/values.yaml`): Use a specific `` for production. Avoid `latest` unless Syntho tells you to. ```yaml SynthoLicense: kuberay-operator: imagePullSecrets: - name: syntho-cr-secret ray-operator: imagePullSecrets: - name: syntho-cr-secret ray-cluster: imagePullSecrets: - name: syntho-cr-secret image: tag: ``` Deploy: ```bash helm upgrade --install ray-cluster ./helm/ray/chart \ --values helm/ray/values.yaml \ --namespace syntho ``` {% hint style="info" %} If your bundle uses a different values filename or path, use that path in `--values`. {% endhint %} {% hint style="info" %} The Ray head service is typically `ray-cluster-ray-head`. Use it as `ray_address` in the Syntho chart. {% endhint %}
Ray troubleshooting (ArgoCD, permissions) If Ray pods fail due to volume permissions, set a permissive security context: ```yaml ray-cluster: head: securityContext: runAsUser: 0 runAsGroup: 0 worker: securityContext: runAsUser: 0 runAsGroup: 0 ```
{% endstep %} {% step %} **Deploy Syntho** Copy `helm/syntho-ui/values.yaml` to your own values file and configure it for your deployment. Keep that file separately and pass its path to Helm with `--values`. The commands below use `helm/syntho-ui/values.yaml`; substitute your own file's path. {% hint style="info" %} Production recommendation: use a hosted / managed PostgreSQL. See [Back up PostgreSQL](/deploy-syntho/deploy-syntho-using-kubernetes/back-up-postgresql.md). {% endhint %} **Set images** Set image repositories and tags to the versions provided by Syntho. Use a specific `` for production. Avoid `latest` unless Syntho tells you to. ```yaml frontend: image: repository: synthoregistry.azurecr.io/syntho-core-frontend tag: backend: image: repository: synthoregistry.azurecr.io/syntho-core-backend tag: core: image: repository: synthoregistry.azurecr.io/syntho-core-api tag: ``` **Set license key** Set the license key in the Syntho chart values: ```yaml SynthoLicense: ``` Use the same license key value as in `helm/ray/values.yaml`. **Configure external PostgreSQL (recommended)**
Example values Create an external PostgreSQL host and two databases (common setup): * Backend metadata DB * Core API metadata DB Then configure the Syntho Helm values to use them and disable the embedded databases. Common values: ```yaml backend: database_enabled: false db: host: port: 5432 user: password: name: core: database_enabled: false db: host: port: 5432 username: password: name: ```
**Configure Redis** By default, the chart deploys Redis and exposes it as `redis-svc`. Redis uses persistent storage and AOF for queues and ongoing job logs. Review [Redis persistence and sizing](#redis-persistence-and-sizing) before deployment. Choose the data volume size before the first installation; the default is `1Gi`. If you use an external Redis, update these values: ```yaml backend: redis: host: redis-svc port: 6379 db: 0 core: redis: host: redis-svc port: 6379 db: 1 ```
Using the chart-managed PostgreSQL/Redis (optional) Keep the embedded dependencies enabled if you don’t run external services. Disable embedded PostgreSQL by setting `database_enabled: false` (as shown above). Minimal example: ```yaml backend: database_enabled: true db: host: database port: 5432 core: database_enabled: true db: host: database port: 5432 ``` If you keep embedded PostgreSQL enabled, the in-cluster service is typically: * Host: `database` * Port: `5432`
Frontend URL and Ingress: ```yaml frontend_url: frontend_protocol: https # or http ingress: enabled: true className: nginx hosts: - host: paths: - path: / pathType: Prefix ``` The Ingress also routes backend traffic using path-based routing (typically `GET /api/*`). You don’t need a separate Ingress for the backend.
Ingress annotations and TLS (nginx + cert-manager example) ```yaml ingress: enabled: true name: frontend-ingress className: nginx annotations: cert-manager.io/cluster-issuer: "" # when using cert-manager nginx.ingress.kubernetes.io/proxy-buffer-size: "32k" nginx.ingress.kubernetes.io/affinity: "cookie" nginx.ingress.kubernetes.io/use-regex: "true" nginx.ingress.kubernetes.io/proxy-connect-timeout: "600" nginx.ingress.kubernetes.io/proxy-read-timeout: "600" nginx.ingress.kubernetes.io/proxy-send-timeout: "600" nginx.ingress.kubernetes.io/proxy-body-size: "512m" hosts: - host: paths: - path: / pathType: Prefix tls: # remove this section when not using TLS - hosts: - secretName: frontend-tls ```
Admin user: ```yaml backend: user: username: admin email: admin@company.com password: ``` Backend secret key: ```yaml backend: secret_key: ``` Core API encryption key: ```bash python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())" ``` ```yaml core: secret_key: ``` Ray address: ```yaml ray_address: ray-cluster-ray-head ```
Use externally managed Kubernetes Secrets (optional) By default, the chart creates `backend-secret` and `core-secret` from your Helm values. If you want to bring your own Secrets (for example via an external secrets manager), set: * `backend.manualSecretName: ` * `core.manualSecretName: ` Create the Secrets in the same namespace. Backend Secret must include: ```yaml apiVersion: v1 kind: Secret metadata: name: type: Opaque stringData: backend.db.password: "" backend.secret_key: "" backend.user.password: "" ``` Core Secret must include: ```yaml apiVersion: v1 kind: Secret metadata: name: type: Opaque stringData: core.secret_key: "" core.database_url: "postgresql+asyncpg://:@:5432/" license_key: "" ```
Deploy: ```bash helm upgrade --install syntho-ui ./helm/syntho-ui \ --values helm/syntho-ui/values.yaml \ --namespace syntho ``` {% hint style="info" %} If your bundle uses a different values filename or path, use that path in `--values`. {% endhint %}
HTTPS and cookies If TLS is terminated at a reverse proxy / load balancer / Ingress, Syntho must be configured for `https`. If Syntho is configured for `http`, browsers can reject cookies. Symptoms include login loops or sessions not sticking. Fix: * Set the frontend protocol to `https`. * Disable secure cookies.
{% endstep %} {% step %} **Verify deployment** Check pods: ```bash kubectl get pods -n syntho ``` Open the UI at your `frontend_url`. If pods are not Ready, check logs: ```bash kubectl logs -n syntho ``` {% hint style="info" %} If the UI doesn’t resolve, check DNS and the Ingress external address first. {% endhint %} If this does not work, see [Troubleshooting](/deploy-syntho/deploy-syntho-using-kubernetes/troubleshooting.md). {% endstep %} {% step %} **Back up PostgreSQL** [Back up PostgreSQL](/deploy-syntho/deploy-syntho-using-kubernetes/back-up-postgresql.md) Syntho application databases before first use to validate the process. {% endstep %} {% endstepper %} ### Redis persistence and sizing The chart deploys a single Redis primary (`redis.replicaCount: 1`), not a high-availability Redis cluster. Redis holds queued tasks and complete ongoing job logs until their final archive is saved in PostgreSQL. Provision memory for all unarchived jobs, including long-running jobs and archive retries, plus queues and other application data. Redis writes an **Append-Only File (AOF)** to the `redis-data` PVC mounted at `/data`. It uses `appendfsync everysec`, keeps fsync enabled during rewrites, and reloads persisted data when its pod is replaced using the same volume. The separate `redis-claim` PVC holds configuration, not job data. | Helm value | Default | Purpose | | ------------------------------------------ | --------------------------------------------------- | -------------------------------------------------------------------------- | | `redis.maxmemory` | `512mb` | Memory budget for Redis data, not total process memory | | `redis.persistence.size` | `1Gi` | Data PVC request for a new installation | | `redis.persistence.storageClassName` | `null` | Inherit `redis.storageClassName`; set explicitly to select a storage class | | `redis.persistence.appendonly` | `true` | Enable AOF persistence | | `redis.persistence.appendfsync` | `everysec` | Flush policy: `everysec`, `always`, or `no` | | `redis.persistence.noAppendfsyncOnRewrite` | `false` | Continue fsync during rewrites | | `redis.resources.requests` | CPU `100m`, memory `512Mi` | Pod scheduling requests | | `redis.resources.limits` | CPU `1`, memory `2Gi` | Container limits | | `redis.startupProbe` | Period `10s`, timeout `5s`, failure threshold `180` | Allow up to 30 minutes for initial data loading | | `redis.readinessProbe` | Period `10s`, timeout `5s`, failure threshold `3` | Remove an unready Redis pod from Service endpoints | | `redis.livenessProbe` | Period `30s`, timeout `5s`, failure threshold `5` | Restart an unresponsive Redis after startup | Only `redis.replicaCount: 1` is supported; the chart rejects other replica counts. `redis.persistence.storageClassName` takes precedence over `redis.storageClassName`, which defaults to `"default"`. Storage-class selection applies to both the data and configuration PVCs. For statically provisioned volumes, set the storage-class values to an empty string and configure `redis.pvLabelKey` to match the PV's `pv-label-key` label. Provide suitable volumes for both claims. The `noeviction` policy prevents memory pressure from silently evicting queue or log keys. At the memory budget, Redis rejects writes. Leave substantial container memory headroom: AOF rewrites can approach twice the dataset's memory usage, plus process overhead. Keep resource requests at or below limits. The default **1Gi data PVC is not a workload sizing recommendation**. Allow space for the current AOF, rewritten files, concurrent writes, and any RDB snapshots. AOF files can be larger than the in-memory dataset. For example, to request 10Gi on a **new installation**, add this to your values file: ```yaml redis: persistence: size: 10Gi ``` Select a StorageClass available in your cluster. Pod recovery on another node depends on the provisioner and volume topology; local/node-bound storage cannot simply follow the pod. Startup probes allow up to 30 minutes for Redis to load persisted data by default. For larger datasets or slower storage, adjust `redis.startupProbe.failureThreshold` and `redis.startupProbe.periodSeconds` to allow enough recovery time. All Redis probes require an exact `PONG` response, so a `LOADING` response is not readiness. The startup probe suppresses readiness and liveness checks until it succeeds. A checksum of the Redis ConfigMap triggers a pod restart when Redis configuration changes, and the init container copies the updated configuration to the configuration PVC. This checksum does not track unrelated chart settings or application Secrets. {% hint style="warning" %} Persistence is not high availability or a backup. A crash can lose roughly the latest second of writes with `everysec`; storage failures can lose more. `always` trades higher write latency for more frequent fsync, while `no` leaves flushing to the operating system. Disabling AOF or fsync during rewrites weakens durability. Protect volumes and check their reclaim policies: volume deletion or storage loss can destroy queues and unarchived logs. {% endhint %} For an **existing installation**, changing `redis.persistence.size` does not resize its PVC and can cause Helm to reject an immutable StatefulSet change. Follow [Resize the Redis data PVC](/deploy-syntho/deploy-syntho-using-kubernetes/operations.md#resize-the-redis-data-pvc) instead. For monitoring, see [Redis health and capacity](/deploy-syntho/deploy-syntho-using-kubernetes/operations.md#redis-health-and-capacity). If you use external Redis, configure equivalent persistence, `noeviction`, capacity, and recovery on that service. These Helm values configure only the chart-managed Redis. ### Next steps * Day-to-day commands: [Operations](/deploy-syntho/deploy-syntho-using-kubernetes/operations.md) * Upgrade procedure: [Upgrade](/deploy-syntho/deploy-syntho-using-kubernetes/upgrade.md) * Common issues: [Troubleshooting](/deploy-syntho/deploy-syntho-using-kubernetes/troubleshooting.md) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.syntho.ai/deploy-syntho/deploy-syntho-using-kubernetes.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.