Installation overview

This page describes the machines, external dependencies, and prerequisites involved in a first Kibotos deployment.

Install the components in this order:

1. Install ctrl and issue the first Operator enrollment key.
2. Install node and join it to the cluster.
3. Optional: install the Exposure gateway, install the E2B gateway, and connect instance metrics storage.

For reinstalling and upgrading, see Reinstall and upgrade. For uninstalling, see Uninstall. To evaluate on a single machine first, see Try Kibotos on a single machine.

OverviewCopy Anchor Link

A deployment involves the following kinds of machines. Each role has its own page.

You can also run ctrl, the E2B gateway, and the Exposure gateway on Kubernetes or Docker from the published container images. See Run in containers. node runs only on hosts. See Install node.

External dependencies:

• The deployer runs PostgreSQL and the S3-compatible artifact store, including their availability and backups.
• All of ctrl's reads and writes must reach the PostgreSQL primary. A connection proxy or connection pool can sit in between; kibotos-ctrl migrate must connect to the primary directly. See Connecting to the database.
• For writing and evaluating times (publishing, expiration, and Exposure deadlines), ctrl uses only its own host's clock. The PostgreSQL server's clock plays no part. Clock skew on a managed database does not affect ctrl, but the hosts of ctrl and every node must have time synchronization enabled. See Time synchronization.
• Public DNS and an HTTPS reverse proxy are needed only for the E2B SDK. Kibotos's own PKI does not depend on a public CA.
• Instance metrics storage is needed only when the E2B SDK should read sandbox metrics. The deployer runs it as well. See Instance metrics storage.
• The deployer collects service and cluster metrics and logs. See Cluster and service metrics and Logs. Trace storage is also optional. See Traces.

PrerequisitesCopy Anchor Link

• Hosts run Debian 13, Ubuntu 24.04, or later, on amd64.
• On first installation, all components come from the same release set. Afterward, ctrl and nodes can run different versions on the same release line (release and -rc.N versions with the same MAJOR.MINOR). See Rolling upgrades within a MINOR.
• The source repository is not public yet, so downloading the release set from GitHub Releases requires an account with read access to the repository. Log in once with the GitHub CLI, then download with scripts/fetch-release.sh from the source repository. You can copy the script to the host on its own and run it there. It downloads all assets of the given version to ./kibotos-releases/<tag>/, checks the size and digest of each file, checks that SHA256SUMS covers every file, and restores the executable bit on binaries. If any step fails, it leaves no directory behind. It does not overwrite an existing target directory. Use --dir to change the root directory:

gh auth login
bash fetch-release.sh v1.2.3
cd kibotos-releases/v1.2.3

• Verify integrity in the release set directory. Checksums prove only that the files transferred intact; they do not replace trust in the source:

sha256sum --check SHA256SUMS
./kibo_1.2.3_linux_amd64 version --client --json

• In the release set, only the Operator CLI is a directly executable file. All other components are delivered as Debian packages. Check their versions after installing the packages, for example with kibotos-ctrl version --json.

Package dependencies and system accounts:

• The ctrl package depends on ca-certificates, systemd, and systemd-sysv.
• The node package also depends on libguestfs-tools, e2fsprogs 1.47.0 or later, and AppArmor 4 or later.
• Each package creates its own system accounts: kibotos-ctrl for ctrl, kibotos-vmm and kibotos-egress for node, kibotos-e2b for the E2B gateway, and kibotos-gw for the Exposure gateway. kibotos-vmm belongs to the kvm group. kibotos-egress is the account that egress processes run as. See Egress processes.
• /dev/kvm and /dev/net/tun must actually exist on node hosts. Package dependencies do not check devices.

Time synchronizationCopy Anchor Link

Enable time synchronization on every ctrl and node host. A node uses its own clock to verify data-plane credentials. Each time ctrl reconciles with a node, it measures the clock offset between them. A node whose offset exceeds 30 seconds no longer accepts new Workspaces; its existing Workspaces keep running.

• Both chrony and systemd-timesyncd work. Use the same kind of time source on all hosts: do not mix leap-smearing sources (most cloud providers' internal time services) with non-smearing sources (for example, pool.ntp.org). Around a leap second, the two kinds differ by nearly one second.

• The ctrl and node service units are ordered after time-sync.target. To make boot actually wait for clock synchronization, enable the matching wait service:

  Copy Text

  sudo systemctl enable chrony-wait.service            # chrony
  sudo systemctl enable systemd-time-wait-sync.service # systemd-timesyncd

• To confirm synchronization, timedatectl show -p NTPSynchronized should print NTPSynchronized=yes. With chrony, the System time line of chronyc tracking is the offset between this machine and the time source.

• After a node joins the cluster, the Latest session section of kibo node get <node> has two lines: Clock offset is the offset most recently measured by ctrl (node minus ctrl), and Clock sync is the synchronization status reported by the node host's kernel. The CLOCK column of kibo node list shows the offset and marks it (over) above 30 seconds, and ctrl also logs an entry. After you fix time synchronization, the node resumes accepting Workspaces at the next reconciliation (within 1 minute).