Files

Upload, download, and delete files in the guest, and watch a directory for changes. The examples use the Go SDK, the E2B SDK, and kibo workspace file. Programs transfer file content over /v1/files and manage directories with FilesystemService; see With the API.

Uploading and downloadingCopy Anchor Link

The following examples upload a file and then download it again. The Go example uses the client from Connect to ctrl:

session, err := dataplane.Open(client, workspaceID, dataplane.WithScopes(
	ctrlv1.DataPlaneScope_DATA_PLANE_SCOPE_FILE_READ,
	ctrlv1.DataPlaneScope_DATA_PLANE_SCOPE_FILE_WRITE,
))
if err != nil {
	log.Fatal(err)
}
defer session.Close()

content := "hello\n"
if _, err := session.PutFile(ctx, "/data/report.txt", strings.NewReader(content), int64(len(content)), dataplane.PutOptions{}); err != nil {
	log.Fatal(err)
}

file, err := session.GetFile(ctx, "/data/report.txt", dataplane.FileOptions{})
if err != nil {
	log.Fatal(err)
}
defer file.Close()
data, err := io.ReadAll(file)

UploadingCopy Anchor Link

kibo workspace file put and the Go SDK's PutFile check the size and SHA-256 digest after the upload. The flags below are those of kibo.

When the local path is -, put reads from standard input. This command writes a short text file:

echo hello | kibo workspace file put "$workspace_id" - /data/hello.txt

DownloadingCopy Anchor Link

get first writes to a temporary file next to the target and replaces the target only after everything has arrived. On failure, the target is unchanged.

• For the permission rules of download targets, see Writing downloaded files.
• When the local path is -, get writes to standard output.

This command prints the first lines of a file in the guest:

kibo workspace file get "$workspace_id" /data/report.txt - | head -n 20

Size and time limitsCopy Anchor Link

• An upload is at most 1 GiB; a larger file fails with resource_exhausted. Downloads have no size limit: kibo reads a file over 1 GiB in parts, and if the file is rewritten during the download, it exits with an integrity failure and leaves the local target unchanged.
• Each transfer runs at up to 64 MiB/s.
• Uploads and downloads are not limited by the global --timeout and can take as long as they need. --timeout limits only the requests sent to ctrl before the transfer.
• A transfer that sends and receives no data for one continuous minute exits with a failure.

DeletingCopy Anchor Link

delete deletes a file or directory in the guest. A directory is deleted with everything in it. The command is also available as rm, and -y (--yes) skips the confirmation:

kibo workspace file delete "$workspace_id" /data/report-copy.txt --yes

Watching a directoryCopy Anchor Link

The watch subcommand prints changes to the entries of a directory, one per line, until you press Ctrl-C. This command watches the Volume:

kibo workspace file watch "$workspace_id" /data

Each line gives the event and then the entry's path. --entry also prints the metadata of each entry.

• Only one level is watched. Creating, deleting, and renaming a subdirectory itself are reported as usual, but changes inside the subdirectory are not.
• Consecutive identical changes may be reported only once.

watch exits with a failure in these cases, and later changes are no longer reported:

• The directory is deleted or moved away.
• The Workspace stops.
• Events cannot be delivered in time.

Guest user and pathsCopy Anchor Link

Every subcommand accepts --user to choose which guest user reads and writes. Remote paths can be absolute, or relative to the home directory of that user. For the rules, see User and working directory.

With the APICopy Anchor Link

File content travels over the data-plane HTTP endpoint /v1/files, not over RPC. FilesystemService handles directories and metadata, and watches directories with WatchDir.

• Programs need a data-plane credential with the FILE_READ or FILE_WRITE scope; see Requesting a session credential. kibo requests the credentials it needs by itself.
• Paths in /v1/files and FilesystemService requests can be relative to the home directory of the user. For the rules on the user field and paths, see User and working directory.
• For every parameter, header, status code, method, and message, see /v1/files and FilesystemService.

Transferring file contentCopy Anchor Link

/v1/files is at the same data-plane address as the RPC services. GET reads a file and PUT writes one:

GET  <endpoint>/v1/files?path=/data/report.txt
PUT  <endpoint>/v1/files?path=/data/report.txt&mode=0644&create_only=true

The query parameters user, mode, and create_only work like the kibo flags --user, --mode, and --create-only. Unknown or repeated parameters and invalid values return 400 (invalid_argument) before the request reaches the guest.

Reading a file:

• One response carries at most 1 GiB. Read a larger file in parts, one Range: bytes=… range per request. A request with several ranges returns 400.
• To make sure the parts come from the same version of the file, send the ETag of the first response in If-Range. If the file has changed, the request answers with the whole file instead of the range, or with 413 (resource_exhausted) when the file is over 1 GiB.

Writing a file:

• The body of one PUT is at most 1 GiB, and a file cannot be written in parts.
• When PUT returns 200, the file has been written to storage. The response body is JSON: {"path": …, "size_bytes": …, "sha256": …}. Check the size and digest.

Two write failures are not retryable. In both, any existing file at the target stays unchanged:

Write operations of FilesystemService return the same error codes in these two cases.

Every request runs at up to 64 MiB/s. It is aborted when the caller sends no request body or reads no response for 5 seconds. Time spent waiting for the guest does not count.

An error response carries the Connect error code and the ErrorDetail as JSON, so the program can handle it like an RPC error.

Directories and metadataCopy Anchor Link

Tracking a directoryCopy Anchor Link

WatchDir reports changes to the entries of a directory, so the caller does not need to poll. To keep a complete view of a directory:

1. Call WatchDir and wait for the first message, started.
2. Fetch the full listing with ListDir. If you fetch it before started, changes in between may be missing from both the listing and the events.
3. From then on, update from the events. An event only says which path may have changed. For details, call Stat, or set include_entry in the request.
4. When the stream ends for any reason, start over from step 1.

After started and until the stream ends, every change in the directory produces at least one event. If that cannot be guaranteed, the stream ends with resource_exhausted.

The event types and the one-level rule are the same as for kibo; see Watching a directory. In addition:

• path is the requested directory plus the entry name. If the requested directory is a symbolic link to a directory, it is followed.
• include_entry makes CREATE, WRITE, and CHMOD carry the entry's EntryInfo, which is omitted when the entry no longer exists.
• Each Workspace allows at most 64 watch streams. Beyond that, new requests return resource_exhausted.
• Each stream delivers at most 1000 events per second and sends a heartbeat every 10 seconds when idle.

A stream never ends successfully. It ends for these reasons:

NotesCopy Anchor Link

• If the Workspace's guest is older than v0.2.0, WatchDir returns failed_precondition (CAPABILITY_UNAVAILABLE). Restart the Workspace to enable it.
• If the Workspace's guest is older than v0.3.0, file requests accept only absolute paths. See Working directory and relative paths.

Next stepsCopy Anchor Link

• Commands and terminals: run commands and open terminals in the guest.
• Volumes, export, and restore: keep data independently of a Workspace.
• kibo CLI reference: global flags and local file output.
• API overview: credentials and conventions for your own program.