Kubernetes Troubleshooting

Diagnose connection, TLS, and kubeconfig sync problems with Privileged Access for Kubernetes.

Use this page when kubectl can’t reach a cluster through Privileged Access for Kubernetes.

First checks

  • Sign in to the Twingate Client. If autosync is off, sync your kubeconfig with twingate kube config sync.

  • Switch to the context the Twingate Client added, and print its configuration:

    kubectl config view --minify --output yaml
  • Run kubectl auth whoami to test the connection.

  • Check the Gateway logs for errors. If there are no log entries for your request, the request never reached the Gateway. See Connection reset with no Gateway logs.

  • In the Admin Console, check the Kubernetes Resource’s network events.

Connection reset with no Gateway logs

kubectl auth whoami fails with connection reset by peer:

couldn't get current server API group list: Get "https://my-k8s-cluster.int/api?timeout=32s": read tcp 100.96.0.2:51459->100.109.236.181:443: read: connection reset by peer

The Gateway pod has no log entries for the request, and the Resource’s network events in the Admin Console show DNS lookup failed. The Connector can’t reach the Gateway.

Causes:

  • The Gateway uses the default ClusterIP service, but a Connector in the same Remote Network runs outside the cluster. A ClusterIP service is reachable only from inside the cluster, and any Connector in the Remote Network can carry the connection. Expose the Gateway with a LoadBalancer service. If Connectors reach it another way, such as a NodePort, an ingress controller, a proxy, or a private endpoint, set twingateOperator.gateway.address to the address they use (Gateway 2.1 or later). See Connectors outside the cluster.
  • The Resource’s Gateway address is wrong, or the Connector can’t resolve it. If Connectors reach the Gateway indirectly, set twingateOperator.gateway.address (Gateway 2.1 or later).

Connection reset from a TLS mismatch

kubectl auth whoami fails with connection reset by peer, the Gateway pod has no log entries, and the Resource’s network events show a successful connection that lasted less than one second. The Twingate Client couldn’t complete the TLS handshake with the Gateway.

Diagnose: From a machine running the Twingate Client, test the Resource address and alias with curl:

curl -k -v https://kubernetes.default.svc.cluster.local
curl -k -v https://my-k8s-cluster.int

The address test succeeds if you installed the Gateway with the Helm chart or the Twingate Operator. If the Resource has an alias, the synced kubeconfig uses the alias. If the alias test fails, the Gateway’s certificate doesn’t include the alias.

Causes:

  • The Gateway’s TLS certificate doesn’t cover the Resource address or alias. Add the alias to tls.certificates.autoGenerated.dnsNames in the Gateway Helm values. An alias set in twingateOperator.kubernetesResource.alias is added automatically. See Gateway TLS Certificates.
  • The CA certificate in the Resource configuration doesn’t match the Gateway’s TLS certificate.

kubeconfig sync problems

~/.kube doesn’t exist

The Twingate Client can’t sync kubeconfig if ~/.kube doesn’t exist. Create the folder, then sync again.

kubeconfig isn’t written on macOS

The Twingate Client logs show:

[ERROR] [client] Sync kubernetes resources finished: Failed to create kubeconfig at /var/root/.kube/config: Permission denied (os error 13)

The Twingate Client is running as root, whose home folder on macOS is /var/root. This can happen when an MDM installs the Client. Run the Client as the signed-in user. To check which user runs it:

ps aux | grep '[T]wingate.app'

Known issues

Sessions over SPDY are not recorded

The Gateway records kubectl exec and similar sessions only over the WebSocket streaming protocol. Sessions that fall back to the deprecated SPDY protocol are not recorded.

  • kubectl and clusters on Kubernetes 1.31 or later use WebSockets, unless you turn off the KUBECTL_REMOTE_COMMAND_WEBSOCKETS or KUBECTL_PORT_FORWARD_WEBSOCKETS environment variables.
  • For a 1.29 or 1.30 cluster with a current kubectl, enable the TranslateStreamCloseWebsocketRequests (1.29+) and PortForwardWebsockets (1.30+) feature gates. Otherwise, streaming falls back to SPDY.
  • With kubectl or a cluster on 1.28 or earlier, streaming always falls back to SPDY.

The Kubernetes WebSocket transition post has more detail.

kubectl cp fails on large files

kubectl cp can fail on large files (Kubernetes issue #60140). Use kubectl cp --retries to work around it.

ArgoCD regenerates the Gateway certificate

ArgoCD renders Helm charts without Kubernetes API access, so Helm’s lookup function returns nothing (ArgoCD issue #5202). The Gateway chart uses lookup to reuse its existing TLS Secret. With the default auto-generated certificate, every ArgoCD sync creates a new CA and certificate. Use cert-manager or your own certificate instead.

The same limitation means the chart can’t derive the certificate authority name. Set twingateOperator.gateway.certificateAuthority.name yourself. See Upgrading the Gateway.

Last updated