Kibotos

Networking overview

A Workspace's network policyThe rules that decide where a Workspace's outbound traffic may go: its mode, destination rules, and any Egress or Resolver it references. A new Workspace has no network access by default. decides which external addresses it can reach and which path its traffic takes. This page introduces the parts of the policy and the paths, and walks through sending all of a Workspace's traffic out through your own exit.

Overview

The network policy has four parts:

PartEffectIn kibo
modeDENY_ALL denies all outbound traffic; PUBLIC_INTERNET allows the internetdeny-all or public-internet, as the value of --network or the mode argument of network set
Destination rulesNarrow outbound internet traffic by destination address--allow, --deny
EgressAn outbound SOCKS5 proxy that the caller runs. When a Workspace's network policy references it, TCP connections to the given ranges leave from the caller's network.Send TCP to some destinations out through a SOCKS5 endpoint the caller runs--egress, --egress-destination
ResolverA DNS server that the caller provides. When a Workspace's network policy references it, the guest's DNS queries go to it.Forward the guest's DNS queries to an upstream the caller chooses--resolver, --resolver-path
  • The default mode is DENY_ALL.
  • The last three parts apply only to PUBLIC_INTERNET.
  • You give the policy when you create the Workspace, with the flags of the same names on kibo workspace create.
  • You can change the policy with kibo workspace network while the Workspace is AVAILABLE or STOPPED. See Effect of a change.

Egresses and Resolvers are separate objects. Create them first, then reference them by ID in the policy. One Egress or Resolver can be referenced by many Workspaces.

Traffic paths

The diagram below shows one node and the three networks it connects to: the internet; the deployment network, meaning the deployer's management network and other private and special-purpose addresses; and the caller network, meaning the caller's SOCKS5 endpoint and internal network. Click "Next" below the diagram, or select the diagram and press the arrow keys, to add the three paths one by one.

DirectVia EgressDNS query
PathWhen it is usedWhat it reachesDetails
DirectUnder PUBLIC_INTERNET, connections whose destination is outside the Egress rangesThe internet. Private and special-purpose addresses are refusedNetwork policy
Via EgressNew TCP connections whose destination is inside the Egress ranges referenced by the policyWherever the SOCKS5 endpoint can reach, including the caller's internal networkEgress
DNS queryQueries the guest sends to the link DNS addressThe deployment DNS servers, or the Resolver referenced by the policyDNS and Resolvers

Under DENY_ALL, none of the three paths works: outbound connections are refused, and DNS queries fail immediately.

Use cases

NeedApproach
Outbound connections leave from your own network, and destinations see your own exit addressReference an Egress with ranges 0.0.0.0/0 and ::/0
Reach services in your own internal networkReference an Egress with your internal network's ranges
Migrate from E2B, with code that already uses egressProxyNo code changes; the gateway creates the Egress. See Egress proxy
The addresses from the deployment DNS servers do not work, or CDN domains must resolve according to where your own exit isReference a Resolver with a path via Egress
Queries must not go through the deployer's DNS servers, or you want to use your own filtering DNS serverReference a Resolver
Allow access to only a few addressesDestination rules: deny with 0.0.0.0/0 and ::/0, allow with the allowed ranges

Steps

The steps below send all outbound TCP and DNS queries of a Workspace out through the caller's own SOCKS5 endpoint. When the server does not support Egress or Resolver, kibo exits with 14 and sends no request.

  1. Create an Egress, reading the password from a file. --udp lets the udp Resolver in step 3 send queries through it:

    egress_id=$(kibo egress create --address proxy.example.com:1080 --username alice --password-file ./proxy-password --udp -q)
  2. Reference the Egress in the policy, with ranges covering all IPv4 and IPv6 destinations:

    kibo workspace network set "$workspace_id" public-internet --egress "$egress_id" --egress-destination 0.0.0.0/0 --egress-destination ::/0
  3. Create a Resolver and reference it with a path through this Egress, so that DNS queries also go out through it:

    resolver_id=$(kibo resolver create --udp 10.0.0.53 -q)
    kibo workspace network add "$workspace_id" --resolver "$resolver_id" --resolver-path "egress:$egress_id"
  4. Show the Workspace to confirm the change is in effect:

    kibo workspace get "$workspace_id"

    When the revision on the Egress applied line matches the one from kibo egress get, new connections are going out through this Egress. If there is an Egress fault line, see Network troubleshooting.

With the API

Each part of the policy is a field of the network policy:

PartAPI field
modemode
Destination rulesallow, deny
Egressegresses
Resolverresolvers
  • Give the policy at creation in WorkspaceCreateSpec.network_policy.
  • Change it with UpdateWorkspaceNetworkPolicy while the Workspace is AVAILABLE or STOPPED.
  • Before using Egress or Resolver, check the capability names.

The steps above map to these calls:

  1. Create the Egress with EgressService.CreateEgress.
  2. In UpdateWorkspaceNetworkPolicy, write one entry in egresses with egress_id and destinations, and one entry in resolvers with resolver_id and path.egress_id.
  3. Use network_policy_status from GetWorkspace to check whether the change is in effect.

Next steps

  • Network policy: mode, destination rules, and how to change the policy and confirm it is in effect.
  • Egress: send some connections out through the caller's own SOCKS5 endpoint.
  • DNS and Resolvers: where the guest's DNS queries go, and how to switch to an upstream the caller chooses.
  • Network troubleshooting: confirm the policy is in effect, and tell causes apart when an Egress or Resolver fails.

On this page