Session Recordings
How the Gateway records sessions, and how to route recordings to storage.
The Gateway records interactive sessions, such as kubectl exec and SSH shells, in asciicast v2 format. Recordings stay in your environment and are never uploaded to Twingate. Replay them with the Twingate session player or asciinema play.
Recording format
By default, the Gateway writes each recording as a structured JSON log entry to stderr. Because recordings are ordinary log lines, any log pipeline (Vector, Fluentd, Fluent Bit) can route them to storage. To keep recordings separate from other logs, set log.sessionRecording.output to stdout or to a file, with optional rotation.
A recording log entry has these fields:
| Field | Description |
|---|---|
logger | Always session_recording. Use it to separate recordings from other logs. |
asciicast | The recording content in asciicast v2 format. |
asciicast_sequence_num | The segment number. It increases each time the Gateway flushes a segment. |
conn_id | The connection ID. All segments of a session share it. |
user | The Twingate user who ran the session. |
ts | The log timestamp. |
Long sessions produce multiple log entries, one per segment. To rebuild a session, collect every entry with the same conn_id in asciicast_sequence_num order.
Logger name changed in Gateway 2.0
Before Gateway 2.0, the logger name was gateway.audit. Update any log filter that matches on it.
Segment size and duration
The Gateway flushes a segment when it reaches a maximum duration or size. Set both under log.sessionRecording.segment in the Gateway configuration file or Helm values:
log: sessionRecording: segment: maxDuration: 5m maxSize: "64KiB"maxSize defaults to 64KiB (65,536 bytes) and can be at most 256KiB (262,144 bytes). It accepts bytes with no unit or B, decimal units (KB or K, multiples of 1,000), and binary units (KiB or Ki, multiples of 1,024).
Ship recordings to Google Cloud Storage with Vector
This example uses the Vector Helm chart to collect recordings from Gateway pods and write each segment as a .cast file to a Google Cloud Storage bucket. It reads container logs, so it assumes recordings go to stderr or stdout.
Prerequisites
- A Google Cloud Storage bucket.
- A Google Cloud service account with write access to the bucket. Vector authenticates as this service account through Workload Identity.
- The label
vector.dev/twingate-gateway=trueon the Gateway pods. Set it with the Gateway’spodLabelsHelm value.
Install Vector
-
Create a
values.yamlfile for Vector. Replace<SERVICE_ACCOUNT_EMAIL>and<YOUR_GCS_BUCKET_NAME>:role: AgentlogLevel: "info"env:- name: VECTOR_SELF_NODE_NAMEvalueFrom:fieldRef:fieldPath: spec.nodeNamerbac:create: trueserviceAccount:create: trueannotations:iam.gke.io/gcp-service-account: <SERVICE_ACCOUNT_EMAIL>name: vectorcustomConfig:data_dir: /vector-data-dirapi:enabled: trueaddress: 0.0.0.0:8686playground: falsesources:gateway_logs:type: kubernetes_logsextra_label_selector: vector.dev/twingate-gateway=trueinternal_metrics:type: internal_metricstransforms:gateway_json_logs:type: remapinputs:- gateway_logsdrop_on_abort: truemetric_tag_values: singlesource: |-parsed_json = parse_json!(.message)if parsed_json == null {abort}if parsed_json.logger != "session_recording" {abort}. = parsed_jsontimezone: localsession_logging:type: remapinputs:- gateway_json_logsdrop_on_abort: truemetric_tag_values: singlesource: |-if .asciicast == null {abort}parsed_ts = parse_timestamp!(.ts, "%Y-%m-%dT%H:%M:%S%.3fZ"). = {"filename_ts_part": format_timestamp!(parsed_ts, "%Y%m%d%H%M%S"),"user_id": replace!(.user.username, r'@|\.', "_"),"conn_id": .conn_id,"message": .asciicast,"asciicast_sequence_num": .asciicast_sequence_num}timezone: localsinks:gcs:type: gcp_cloud_storageinputs:- session_loggingbucket: <YOUR_GCS_BUCKET_NAME>key_prefix: sessionrecordings/{{"{{"}} .user_id {{"}}"}}_{{"{{"}} .filename_ts_part {{"}}"}}_{{"{{"}} .asciicast_sequence_num {{"}}"}}filename_extension: castfilename_append_uuid: falsecompression: noneencoding:codec: textbatch:max_events: 1timeout_secs: 1The pipeline reads logs from pods labeled
vector.dev/twingate-gateway=true, keeps entries whoseloggerissession_recording, and writes each segment to the bucket assessionrecordings/<user>_<timestamp>_<sequence>.cast. -
Add the Vector Helm repository:
helm repo add vector https://helm.vector.devhelm repo update -
Install Vector in its own namespace:
helm install vector vector/vector -n vector --create-namespace -f values.yaml
Limitations
- Kubernetes session recording requires the WebSocket streaming protocol. Sessions that fall back to SPDY are not recorded. See Kubernetes known issues.
- SSH: only interactive shell sessions are recorded. Remote commands (
ssh host <command>), file transfer, and port forwarding are not recorded.
Last updated