Kibotos
API 参考kibotos.operator.v1

NodeRegistrationService

本页列出 kibotos.operator.v1 中 NodeRegistrationService 的方法及每个方法用到的消息与枚举,由 operator.proto 及其注释生成,注释保持英文。多个服务共用的消息与枚举见 kibotos.operator.v1 类型。

ctrl API使用 Operator 证书调用。调用方式

NodeRegistrationService is the Operator-only view and desired-state surface for nodes admitted to this control plane. It is never mounted on the Native Workspace endpoint and does not expose transport or provider locators.

ListNodeRegistrations

POST/kibotos.operator.v1.NodeRegistrationService/ListNodeRegistrations一元调用
请求ListNodeRegistrationsRequest
响应ListNodeRegistrationsResponse

GetNodeRegistration

POST/kibotos.operator.v1.NodeRegistrationService/GetNodeRegistration一元调用
请求GetNodeRegistrationRequest
响应GetNodeRegistrationResponse

DrainNode

POST/kibotos.operator.v1.NodeRegistrationService/DrainNode一元调用
请求DrainNodeRequest
响应DrainNodeResponse

CancelNodeDrain

POST/kibotos.operator.v1.NodeRegistrationService/CancelNodeDrain一元调用
请求CancelNodeDrainRequest
响应CancelNodeDrainResponse

IsolateNode

IsolateNode records that the operator has made this node unable to write to any of its storage pools: its host is powered off, or it is cut off from its pools. ctrl cannot verify the claim, so this is an assertion by the operator and the operator carries its consequence: a node that is in fact still writing corrupts the block files it has open. It is refused while the node still has a live control session.

In one transaction ctrl then releases every use right the node held, in every pool, shared or not, and every Allocation charged to it; every Workspace whose execution was on the node fails as unavailable and retryable, and commands still addressed to the node fail. A Workspace whose objects are in a pool another node reaches can then start on that node. Its data is crash-consistent: writes still in the old host's page cache may be lost. A Workspace whose objects are in a pool only this node reaches can start again once this node, or another host reaching that pool, is back.

POST/kibotos.operator.v1.NodeRegistrationService/IsolateNode一元调用
请求IsolateNodeRequest
响应IsolateNodeResponse
  • released_volumes and released_retained_layers count the Volumes and retained layers whose use right this declaration cleared: every one the node held, whether its pool is shared with other nodes or reached only by this node.

  • failed_workspaces counts the Workspaces this declaration moved to a failed state because their execution was on the isolated node, whichever pools their objects are in. Each fails as unavailable and retryable.

RetireNode

RetireNode ends a node for good: a host that is gone or will not serve again. It takes a node that is DRAINED or whose current incarnation is isolated, and refuses any other as failed_precondition. A replay of the same command answers the same result.

In one transaction the node stops being a member of every pool and its pending enrollment grants are revoked. Its registration stays as a tombstone reading RETIRED; it can never register again, and enrolling, draining or isolating the same node_id is refused. A new host takes a new node_id. A pool the node alone reached is left with no member until another host declares it, and its objects wait for that host.

POST/kibotos.operator.v1.NodeRegistrationService/RetireNode一元调用
请求RetireNodeRequest
响应RetireNodeResponse

ListStoragePools

POST/kibotos.operator.v1.NodeRegistrationService/ListStoragePools一元调用
请求ListStoragePoolsRequest
响应ListStoragePoolsResponse

GetStoragePool

GetStoragePool reads one storage pool, with the same content ListStoragePools gives for it. A pool no node has ever declared is not_found.

POST/kibotos.operator.v1.NodeRegistrationService/GetStoragePool一元调用
请求GetStoragePoolRequest
响应GetStoragePoolResponse

SetStoragePoolCapacity

SetStoragePoolCapacity replaces the capacity placement admits new Volumes and retained layers against in one pool. It takes effect for the next placement on every node that reaches the pool, without a node restart; lowering it below what is already reserved stops new objects there and evicts nothing. It requires the pool version the caller read, and a replay of the same command returns the current pool without writing again. A pool no node has ever declared is not_found.

POST/kibotos.operator.v1.NodeRegistrationService/SetStoragePoolCapacity一元调用
请求SetStoragePoolCapacityRequest
响应SetStoragePoolCapacityResponse

DeclareStoragePoolLost

DeclareStoragePoolLost records that a pool's storage is gone and will not come back. ctrl cannot verify the claim, so the operator carries its consequence. It takes a pool every member of which has its current incarnation isolated, or that has no member, and refuses any other as failed_precondition naming the members still in the way. A replay of the same command answers the same result.

In one transaction the pool's Volumes become VOLUME_STATE_LOST, its reservation is released and the removals waiting on it are dropped. A Workspace bound to a lost Volume fails to start until it is bound to another. A Workspace whose writable layer was in the pool starts with a new layer and reports RESET with RETENTION_RESET_REASON_LAYER_UNAVAILABLE. The pool id never comes back into use: a node still declaring it reports it unavailable, even with the original disk.

POST/kibotos.operator.v1.NodeRegistrationService/DeclareStoragePoolLost一元调用
请求DeclareStoragePoolLostRequest
响应DeclareStoragePoolLostResponse

ListWorkspacePlacements

ListWorkspacePlacements pages the Workspaces whose current Instance holds an unreleased Allocation, narrowed to one node, one Workspace or one Instance: where each executes now, and with which Instance. A Workspace that is stopped, deleted or not placed yet has no row. A node or Workspace that does not exist is not_found; an Instance that is not current anywhere yields an empty page, because it cannot be told apart from one that never existed. It reads ctrl's records and never calls a node.

POST/kibotos.operator.v1.NodeRegistrationService/ListWorkspacePlacements一元调用
请求ListWorkspacePlacementsRequest

ListWorkspacePlacementsRequest names exactly one filter. The page token is bound to it, so a token is refused with any other filter.

响应ListWorkspacePlacementsResponse

WorkspacePlacement

WorkspacePlacement is where one Workspace executes now.

GetOvercommitDefaults

GetOvercommitDefaults returns the cluster-wide overcommit ratios. A node's schedulable CPU and memory are the host resources it reports times the ratio in effect for it: its own override, or else this default.

POST/kibotos.operator.v1.NodeRegistrationService/GetOvercommitDefaults一元调用
请求GetOvercommitDefaultsRequest

无字段。

响应GetOvercommitDefaultsResponse

SetOvercommitDefaults

SetOvercommitDefaults replaces both cluster-wide ratios. Nodes take the new schedulable resources without a restart; lowering them below what is already reserved stops new placement there and evicts nothing.

POST/kibotos.operator.v1.NodeRegistrationService/SetOvercommitDefaults一元调用
请求SetOvercommitDefaultsRequest
响应SetOvercommitDefaultsResponse

SetNodeOvercommit

SetNodeOvercommit replaces one node's overrides. A ratio left out clears that override, so the node follows the cluster-wide default again.

POST/kibotos.operator.v1.NodeRegistrationService/SetNodeOvercommit一元调用
请求SetNodeOvercommitRequest
响应SetNodeOvercommitResponse

MoveVolume

MoveVolume moves a Volume to another storage pool. Its ID, content, capacity and binding stay the same. A node that reaches both pools copies the blocks the Volume's filesystem uses, and ctrl then records the Volume in the target pool; the copy left in the source pool is removed after.

The move is refused, and the Volume stays where it is, unless the Volume is AVAILABLE, the Workspace it is bound to, if any, has no execution (STOPPED, or FAILED with its execution released, as a start finds it), some node reaches both pools, and after the move some node still reaches the Workspace's Volume and writable layer together: failed_precondition. A target pool without room for the Volume's capacity is resource_exhausted; the Volume reserves its capacity in both pools until the move settles. A Volume or Workspace with a command in progress, and a target pool still keeping an earlier copy of the Volume for removal, answer unavailable until that is over. A Volume already in the target pool is answered at once.

The call waits for the move to settle, at most one minute and never past the request deadline. A move still copying then answers unavailable with the StoragePoolObject as it stands in the error details; the same command_id waits again, and once the move settled answers its outcome. A move that fails leaves the Volume in its pool, with the failure in the error. While the Volume moves, the Workspace it is bound to does not start, and the Volume is not resized, bound, released, deleted or exported: each answers unavailable until the move settles.

POST/kibotos.operator.v1.NodeRegistrationService/MoveVolume一元调用
请求MoveVolumeRequest
响应MoveVolumeResponse

MoveWritableLayer

MoveWritableLayer moves a Workspace's writable layer to another storage pool, under the same rules as MoveVolume: the layer must have been placed by an earlier start, the Workspace must have no execution, and its layer and Volume must still share a node after the move. The layer's content moves unchanged, so the Workspace continues from it on its next start as it would have in the source pool.

POST/kibotos.operator.v1.NodeRegistrationService/MoveWritableLayer一元调用
请求MoveWritableLayerRequest
响应MoveWritableLayerResponse

ListStoragePoolObjects

ListStoragePoolObjects pages the Volumes and writable layers that live in one pool and are not deleted, those StoragePool counts in volume_count and retained_layer_count. An object moving out is listed with the pool it moves to; an object moving in is listed in its source pool until the move settles. A pool no node has ever declared is not_found.

POST/kibotos.operator.v1.NodeRegistrationService/ListStoragePoolObjects一元调用
请求ListStoragePoolObjectsRequest
响应ListStoragePoolObjectsResponse

GetStoragePoolObject

GetStoragePoolObject reads one Volume, or one Workspace's writable layer, where it lives, as ListStoragePoolObjects lists it. A Volume that does not exist, is deleted or has not been placed yet, and a Workspace that does not exist, is deleted or whose writable layer was never placed, are not_found.

POST/kibotos.operator.v1.NodeRegistrationService/GetStoragePoolObject一元调用
请求GetStoragePoolObjectRequest

GetStoragePoolObjectRequest names exactly one object.

  • object 中的一个
响应GetStoragePoolObjectResponse

CreateWorkspaceOnNode

CreateWorkspaceOnNode is CreateWorkspace with its placement fixed to one node. The node is the only candidate: it must meet every constraint an ordinary placement checks, drain included, and when it does not the creation fails as an ordinary placement that found no node fails. No other node is chosen instead. Only this placement is fixed: the Workspace keeps no record of the node, and its later starts are placed as usual.

The command is the Workspace command the embedded request names: its command_id, replay, GetWorkspaceCommand and errors follow the Workspace API, and a retry or a restarted ctrl still places it on this node only. The node is part of the command's input, so the same command_id with another node, or sent to CreateWorkspace, is already_exists. A node that was never registered is not_found, and no command is admitted.

POST/kibotos.operator.v1.NodeRegistrationService/CreateWorkspaceOnNode一元调用
请求CreateWorkspaceOnNodeRequest
响应CreateWorkspaceOnNodeResponse

StartWorkspaceOnNode

StartWorkspaceOnNode is StartWorkspace with its placement fixed to one node, under the rules of CreateWorkspaceOnNode. A Workspace whose Volume or writable layer lives in a pool the node does not reach fails to start as unavailable.

POST/kibotos.operator.v1.NodeRegistrationService/StartWorkspaceOnNode一元调用
请求StartWorkspaceOnNodeRequest
响应StartWorkspaceOnNodeResponse

消息与枚举

NodeRegistrationService 的多个方法用到以下消息与枚举。

NodeResources

NodeSessionSummary

NodeClockObservation

NodeClockObservation is how far the node's wall clock is from ctrl's, as ctrl measured it around one call on the control session.

  • offsetgoogle.protobuf.Duration= 1

    offset is the node clock minus the ctrl clock; positive means the node runs ahead.

  • uncertaintygoogle.protobuf.Duration= 2

    uncertainty bounds the error of offset: half the round trip of the measuring call.

  • measured_atgoogle.protobuf.Timestamp= 3

    measured_at is when ctrl received the reading, on ctrl's clock.

  • exceeds_tolerance is true when the offset, less its uncertainty, is beyond the allowed clock skew. Such a node receives no new placement until a later measurement is back within it.

NodeObservationSummary

observation_version is the version of this observation snapshot. It is not the registration record version; NodeRegistration carries that separately under its own name so the two cannot be confused at a call site.

NodeRegistration

registration_version is this record's optimistic concurrency version. The nested observation carries its own version under a different name.

NodeMemoryTight

  • sincegoogle.protobuf.Timestamp= 1

    since is when the node began to judge its host's memory tight, on the node's clock.

OvercommitDefaults

Overcommit ratios are in thousandths: 1000 is 1x and 4500 is 4.5x. CPU and memory ratios both lie in [1, 16000].

NodeOvercommit

NodeOvercommit is the ratio in effect for each resource on one node and where it comes from.

OvercommitRatio

NodeHostUsage

NodeHostUsage is the node's latest usage sample as ctrl received it. Each measurement is absent when the node could not read it. Ratios are in basis points: 10000 is 100 percent.

StoragePool

StoragePool is one storage location declared by the nodes that reach it. Volumes and retained layers live in a pool, and a Workspace runs only on a node that reaches every pool its objects live in.

  • pool_idstring= 1
  • attachments are the pool's members: the admitted nodes that declare it, whether or not each currently reports it available. A revoked or retired node is not one.

  • default_pool is true when at least one node that reaches this pool names it as its default.

  • volume_countuint64= 4
  • capacity_bytesoptionaluint64= 6

    capacity_bytes is the capacity placement admits new Volumes and retained layers against in this pool, as SetStoragePoolCapacity last set it. It is absent until the operator sets one, and then the pool takes no new objects.

  • reserved_bytesoptionaluint64= 7

    reserved_bytes is the declared capacity of every Volume and retained layer in the pool that is not deleted, together with every one moving into the pool, the same sum placement compares with capacity_bytes. It can exceed capacity_bytes when the capacity was lowered below what is already reserved.

  • versionuint64= 8

    version is the pool's own optimistic concurrency version. It advances only when the capacity changes; what nodes report about the pool does not move it.

  • cleanup_countoptionaluint64= 9

    cleanup_count counts the copies the pool still keeps of objects that no longer live there: the source a move left, the target of a move that failed, or an object deleted while the pool had no node to remove it. A node that reaches the pool removes each; until then it takes space in the pool but reserves nothing, and the object cannot move back into the pool.

  • lost_atgoogle.protobuf.Timestamp= 10

    lost_at is when DeclareStoragePoolLost declared the pool lost, absent while it was not. A lost pool reserves nothing and takes no new objects.

StoragePoolAttachment

StoragePoolAttachment is one (node, pool) pair. It is read from both ends: a pool lists the nodes that reach it, and a node lists the pools it declares, so both identities are present.

  • node_idstring= 1
  • availablebool= 2
  • unavailable_reason is what the node reported: a missing or mismatched marker, or an unreadable directory. It is written for the operator and may name paths on the node's host. Empty while available.

  • pool_idstring= 4
  • default_pool is this node's own default: an object whose request names no pool is created in it.

  • held_object_countoptionaluint64= 6

    held_object_count counts the Volumes and retained layers in this pool, not deleted, whose use right this node holds. Isolating the node releases them, and the count drops to zero.

StoragePoolObject

StoragePoolObject is a Volume or a Workspace's writable layer, and the pool it lives in.

  • volume_idstring= 2

    volume_id names the Volume; it is empty for a writable layer.

  • workspace_idstring= 3

    workspace_id is the Workspace the writable layer belongs to, or the one the Volume is bound to; it is empty for a Volume bound to none.

  • storage_poolstring= 4
  • capacity_bytes is the object's declared capacity, what it reserves in its pool.

  • moving_tostring= 6

    moving_to is the pool a move is copying the object to, where it reserves its capacity too until the move settles. It is empty while no move is in progress.

  • move_command_id is the command_id of that move.

NodeAdmission

  • NODE_ADMISSION_UNSPECIFIED= 0
  • NODE_ADMISSION_ADMITTED= 1
  • NODE_ADMISSION_REVOKED= 3
  • NODE_ADMISSION_RETIRED= 4

    NODE_ADMISSION_RETIRED is final: RetireNode ended the node for good.

NodeRegistrationStatus

  • NODE_REGISTRATION_STATUS_UNSPECIFIED= 0
  • NODE_REGISTRATION_STATUS_SYNCING= 1
  • NODE_REGISTRATION_STATUS_READY= 2
  • NODE_REGISTRATION_STATUS_DRAINING= 3
  • NODE_REGISTRATION_STATUS_DRAINED= 4
  • NODE_REGISTRATION_STATUS_UNAVAILABLE= 5
  • NODE_REGISTRATION_STATUS_REVOKED= 7
  • NODE_REGISTRATION_STATUS_RETIRED= 8

    NODE_REGISTRATION_STATUS_RETIRED wins over REVOKED: the registration is a tombstone of a node RetireNode ended.

NodeClockSynchronization

NodeClockSynchronization is what the node host's kernel reports about its clock. UNSPECIFIED means the node could not read it.

  • NODE_CLOCK_SYNCHRONIZATION_UNSPECIFIED= 0
  • NODE_CLOCK_SYNCHRONIZATION_SYNCHRONIZED= 1
  • NODE_CLOCK_SYNCHRONIZATION_UNSYNCHRONIZED= 2

OvercommitSource

  • OVERCOMMIT_SOURCE_UNSPECIFIED= 0
  • OVERCOMMIT_SOURCE_DEFAULT= 1

    DEFAULT is the cluster-wide default.

  • OVERCOMMIT_SOURCE_NODE= 2

    NODE is this node's own override.

StoragePoolObjectKind

  • STORAGE_POOL_OBJECT_KIND_UNSPECIFIED= 0
  • STORAGE_POOL_OBJECT_KIND_VOLUME= 1
  • STORAGE_POOL_OBJECT_KIND_WRITABLE_LAYER= 2

本页内容