Deploy CosmicAC
Deploy the CosmicAC Docker Compose stack on your host machine.
Deploy the CosmicAC stack on your host machine with Docker Compose. For how CosmicAC connects to your cluster and runs jobs, see Deployment architecture.
Stack
The stack runs seven services, plus Redis and Caddy:
- cosmicac-wrk-ork
- cosmicac-app-node
- cosmicac-proxy-inference-http
- cosmicac-proxy-inference-hrpc
- cosmicac-wrk-server-k8s-nvidia
- cosmicac-ui
- cosmicac-wrk-monitor
- redis
- caddy
The CosmicAC services use private images published under ghcr.io/tetherto/, in the form ghcr.io/tetherto/<repo>:<tag>. Redis and Caddy use their official public images.
Prerequisites
The recommended host is Ubuntu 22.04 or 24.04 on x86_64. On that host, you need the following:
- Docker Engine and Docker Compose v2.
- Task.
- The jq, node, and kubectl command-line tools.
- Access to the private CosmicAC deployment repository, which holds the Compose files and deploy scripts. Request access from the CosmicAC team.
- GitHub Container Registry (GHCR) credentials for private images.
- Your GitHub username.
- A classic GitHub personal access token with the read:packages scope so it can pull the private ghcr.io/tetherto images. If the tetherto org enforces single sign-on, approve the token for the org. See Managing your personal access tokens.
- A GPU Kubernetes cluster that already meets the Requirements. CosmicAC connects to this cluster but doesn't set it up.
- A valid, readable kubeconfig for the GPU Kubernetes cluster.
Steps
Verify the prerequisites
Enable Docker, then confirm you have everything listed in Prerequisites:
sudo systemctl enable --now docker
docker compose version
task --version
node --version
kubectl version --clientSet up the environment
Clone the deployment repository and change into it:
git clone <deployment-repo-url>
cd <deployment-repo>Create the .env file from the example:
cp .env.example .envIn .env, set the variables your deployment needs. See Required before the first deployment for the ones you must set, and the deployment configuration for the rest.
If GITHUB_PAT and GITHUB_USER aren't set in .env, bootstrap prompts for them so it can log in to GHCR and pull the private images.
Add the kubeconfig
Get the kubeconfig from your cluster administrator. It must meet the Kubeconfig requirements.
Open a file on the host, paste the kubeconfig, and save it:
nano ~/kubeconfigPrint the file's absolute path:
realpath ~/kubeconfigSet KUBECONFIG_SRC in .env to that path:
KUBECONFIG_SRC=/home/<user>/kubeconfigConfirm the kubeconfig is readable and reaches your cluster:
test -s ~/kubeconfig && echo ok
kubectl --kubeconfig ~/kubeconfig config current-context
kubectl --kubeconfig ~/kubeconfig cluster-infoRun the first-time bootstrap
task bootstrap deploys the whole stack with one command, using TAG from .env. For what it runs and the other deployment commands, see Task deployment commands.
Print the deployment directory and copy the path:
pwdOpen a root shell:
sudo -iA root shell starts in root's home directory, so return to the copied path:
cd <copied-path>Run the bootstrap:
task bootstrapVerify the deployment
Check the services and the API:
task ps
curl -s4 -o /dev/null -w '%{http_code}\n' http://127.0.0.1:5173/
curl -s4 "http://127.0.0.1:5173/api/auth/servers?overwrite_cache=true"
curl -s4 "http://127.0.0.1:5173/api/auth/pricing?location=IN&type=gpu&hrs=2"
curl -s4 "http://127.0.0.1:5173/api/auth/jobs?page=1&pageSize=10"The calls carry no token because a default deployment runs with authentication turned off.
The deployment is healthy when task ps shows every service Up and http://127.0.0.1:5173/ returns 200. The servers call lists your GPUs, and the pricing call returns a total cost rather than an error.
Open the web interface
Open http://<server-ip>:5173 in your browser. If you changed UI_PORT, use that port instead.
A default deployment runs with authentication turned off, so anyone who reaches this port has full access without signing in. Restrict network access to the deployment. See Deployment configuration for the authentication defaults.
Help and troubleshooting
Bootstrap fails with rm: cannot remove ...: Permission denied
Bootstrap resets the HyperMQ stores under ./services/*/store before it deploys. The stack's containers create those files as root, so a non-root user can't delete them.
Print the deployment directory and copy the path:
pwdOpen a root shell:
sudo -iReturn to the copied path:
cd <copied-path>Rerun the bootstrap:
task bootstrapBootstrap fails with error from registry: denied
Error response from daemon: error from registry: denied
[ERROR] Required image is unavailable: ghcr.io/tetherto/cosmicac-wrk-ork:release-1.1.0
Docker stores GHCR credentials separately for each user, in that user's ~/.docker/config.json. A user who hasn't logged in to GHCR can't pull the private ghcr.io/tetherto images, even when another user on the same host can.
Print the deployment directory and copy the path:
pwdOpen a root shell:
sudo -iReturn to the copied path:
cd <copied-path>Rerun the bootstrap:
task bootstrapNext steps
With CosmicAC running, create your first job or learn how to operate the deployment: