node configuration
This page lists every table and key of the node configuration file, generated from the annotated example node.example.toml, which the node tests check against the keys node reads. A table or key the example leaves commented out is optional.
[backend]
| Key | Type | Example | Description |
|---|---|---|---|
type | string | "firecracker" | |
kibovm_jail | boolean | true | Optional. Only for type = "kibovm": jail defaults to true. The VMM always runs as kibotos-vmm; setting false omits jail filesystem isolation. Firecracker rejects this key. |
nested_virtualization | boolean | false | Optional. Both backends hide the CPU's virtualization extensions (Intel VMX, AMD SVM) from the guest, so a Workspace cannot run KVM of its own. Set true to show them; the nested-virtualization paths of the host kernel's KVM are then reachable from every guest. Only VMs started afterwards are affected. |
[resources]
What this host can give its VMs: the physical CPU and memory less what the host itself needs. The node refuses to start when either exceeds the host's online CPUs or MemTotal. Overcommit is not written here: ctrl multiplies these by the ratios an operator sets with kibo node overcommit.
| Key | Type | Example | Description |
|---|---|---|---|
cpu_limit | float | 3.5 | |
memory_limit | string | "32.5G" |
[storage]
The node runs as root and starts every VMM as the fixed system user kibotos-vmm (supplementary group kvm); the user is not configurable. Every directory from / down to the storage root and the socket root must be traversable by that user; the node checks this at startup and creates its own directories with group kibotos-vmm.
| Key | Type | Example | Description |
|---|---|---|---|
root | string | "/var/lib/kibotos/node" | |
template_cache_limit | string | "100G" |
[[storage.pool]]
Every node declares at least one storage pool: a directory holding volumes/, retained-layers/, and a marker file the node never creates on its own (see kibotos-node storage-pool init). A pool's path may equal storage.root, which is how a single-node deployment keeps its Volumes and retained layers where they have always lived. Several nodes may declare the same id and path pointing at the same shared (for example NFS) storage; mark that pool shared = true so maintenance checks do not fail over objects another node holds there. Exactly one pool is default; with only one pool declared, it is the default implicitly.
A pool only this node reaches, such as its local disk, needs an id no other node uses: replace node-a below with this node's ID, the --node value its enrollment key was created with. ctrl takes the first storage registered under an id as that pool; another node declaring a different directory under the same id gets that pool refused and, without its default pool, can take no Workspace.
The pool's capacity is not set here. Once a node has declared the pool, set it in ctrl with kibo storage-pool set-capacity; until then the pool takes no new Volumes or retained layers.
| Key | Type | Example | Description |
|---|---|---|---|
id | string | "node-a" | |
path | string | "/var/lib/kibotos/node" | |
default | boolean | true | Optional. default marks the pool that receives an object whose caller named no pool. |
shared | boolean | false | Optional. shared records that more than one node reaches this pool. |
[runtime]
| Key | Type | Example | Description |
|---|---|---|---|
socket_root | string | "/run/kibotos/node" |
[data_plane]
| Key | Type | Example | Description |
|---|---|---|---|
listen | string | "0.0.0.0:18091" | |
advertise | string | "https://node-a.example.com:18091" | advertise is the endpoint ctrl hands to callers: https with the exact host (a lowercase DNS name or an IP literal) covered by the derived node host, ctrl default_extra_hosts, or enrollment data-plane extra hosts. ctrl's server CA issues and renews the certificate for that set; the leaf lives under <storage.root>/pki/data-plane/ and there is no certificate file to configure. |
[telemetry]
Optional.
Optional. The node's own metrics in the Prometheus text format at GET /metrics. Plain HTTP without authentication: bind loopback and scrape from this host. Omit the section to open no listener.
| Key | Type | Example | Description |
|---|---|---|---|
listen | string | "127.0.0.1:18191" |
[ctrl]
The node authenticates to ctrl with the managed identity under <storage.root>/pki, created by kibotos-node join; nothing else is configured.
| Key | Type | Example | Description |
|---|---|---|---|
endpoint | string | "https://ctrl.example.com:18101" | host is ctrl's exact [identity].host: a lowercase DNS name or an IP literal. |
proxy | string | "socks5://proxy.example.com:1080" | Optional. An http, https or socks5 forward proxy the connection to ctrl goes through, with optional user information. When it is set, HTTPS_PROXY, ALL_PROXY and NO_PROXY are not read; when it is absent, they apply. |
[host_network]
Omit the whole section when host networking is not assembled.
| Key | Type | Example | Description |
|---|---|---|---|
address_families | array of strings | ["ipv4", "ipv6"] | Choose ["ipv4"], ["ipv6"], or both; configure pools and uplinks only for enabled families. |
ipv4_pool | string | "100.64.0.0/10" | |
ipv6_pool | string | "2001:db8::/64" | Each VM receives one /128. Replace this documentation prefix with a pool routed to the node. |
[host_network.public_internet]
Optional.
Omit this sub-table when the node must never carry guest traffic to the public internet: every attachment then accepts the deny-all policy only. With it, the node advertises kibotos.service.public-internet=true once the provider has validated the profile against the host at startup. The host must forward every enabled family (net.ipv4.ip_forward=1, net.ipv6.conf.all.forwarding=1, persisted under /etc/sysctl.d/) and must have no forward chain the node does not own with a drop policy. The node checks these and never changes them; when they do not hold it still starts, logs "node offers no public internet egress" with the reason, and advertises no public egress until it is restarted with the host fixed.
| Key | Type | Example | Description |
|---|---|---|---|
ipv4_upstream_interface | string | "eth0" | ipv4_upstream_interface carries IPv4 source translation: it must be up, hold a global IPv4 address, and be the outgoing interface of the IPv4 default route. ipv6_upstream_interface carries routed IPv6 egress under the same conditions for IPv6. Supply only the enabled families; a shared uplink names the same interface for both when dual-stack is selected. |
ipv6_upstream_interface | string | "eth0" | |
resolvers | array of strings | ["1.1.1.1", "2606:4700:4700::1111"] | resolvers are the node resolvers. Every guest lists only its slot's link DNS address, the slot gateway in the family of the first resolver (fe80::1 on the guest interface for IPv6); the queries of a Workspace that references no Resolver go to the first resolver alone. The egress process resolves the hostname of a Resolver upstream on the direct path with all of them, in order. At least one address in an enabled family is required; A and AAAA queries both use the configured resolver transport. |
management_cidrs | array of strings | ["192.0.2.0/24"] | management_cidrs are deployment-owned ranges added to the node's hard deny set. They may be omitted, and must not cover either address pool. |
ctrl configuration
This page lists every table and key of the ctrl configuration file, generated from the annotated example ctrl.example.toml, which the ctrl tests check against the keys ctrl reads. A table or key the example leaves commented out is optional.
Metrics
This page lists every metric the Kibotos processes serve at GET /metrics, generated from the source code that creates them. For which process serves which metrics and how to collect them, see Metrics.