The functionality described here requires running Connector v1.22.0 or higher. See the Connector changelog for more details, and Updating Connectors for guidance on how to update your running Connectors.

This describes how to enable and interpret real-time connection logs from running Connectors. If you wish to export historical data instead, see Exporting network traffic.

Enabling real-time connection logs

Real-time connection logs are controlled by setting an environment variable before the Connector starts. If the environment variable TWINGATE_LOG_ANALYTICS is set to v2, the Connector will output network connection logs (described below in detail).

In all cases, local connection logs will be output to stdout in single-line JSON format. This allows log collection by SIEM platforms such as AWS CloudWatch, Datadog, Splunk, Promtail/Loki, etc.

Docker

If you are using Docker, you can add --env TWINGATE_LOG_ANALYTICS="v2" to the Docker run command to enable local network connection logs.

K8s Helm Chart

The official Twingate Helm Chart includes a configurable parameter env which can be used to set additional environment variables, including TWINGATE_LOG_ANALYTICS="v2" to enable real-time connection logs.

More information about this and other configurable parameters can be found in our Helm Chart’s README.

systemd

If you are using our systemd Connector, you can add the line TWINGATE_LOG_ANALYTICS=v2 to the file /etc/twingate/connector.conf.

Local connection logs can be read using journalctl. For example, the following command will output the most recent 100 logging lines and follow any new output:

journalctl -u twingate-connector -n 100 -f

Single-line JSON output schema (v2)

Connection logs are output as single-line JSON objects, making them straightforward to ingest and parse by SIEM and other log aggregation products. Each network connection line starts with ANALYTICS followed by a single JSON object.

An example log line is shown below:

ANALYTICS {"connection": {"cbct_freshness": -97,"client_ip": "192.0.2.0","duration": 3034753,"id": "e755ba99988db558-24","protocol": "tcp","resource_ip": "1.2.3.4","resource_port": 443,"rx": 234867,"tx": 23363},"connector": {"id": "84014","name": "nondescript-caterpillar"},"device": {"id": "200903"},"event_type": "closed_connection","relays": [],"remote_network": {"id": "6938","name": "AWS Network"},"resource": {"address": "app.website.com","applied_rule": "*.website.com","id": "2255492"},"timestamp": 1698356150045,"user": {"email": "user@twingate.com","id": "113256"}}

For clarity, the JSON object in the log line above is formatted below for readability:

{
  "connection": {
    "cbct_freshness": -97,
    "client_ip": "192.0.2.0",
    "duration": 3034753,
    "id": "e755ba99988db558-24",
    "protocol": "tcp",
    "resource_ip": "1.2.3.4",
    "resource_port": 443,
    "rx": 234867,
    "tx": 23363
  },
  "connector": {
    "id": "84014",
    "name": "nondescript-caterpillar"
  },
  "device": {
    "id": "200903"
  },
  "event_type": "closed_connection",
  "relays": [],
  "remote_network": {
    "id": "6938",
    "name": "AWS Network"
  },
  "resource": {
    "address": "app.website.com",
    "applied_rule": "*.website.com",
    "id": "2255492"
  },
  "timestamp": 1698356150045,
  "user": {
    "email": "user@twingate.com",
    "id": "113256"
  }
}

Notes on the connection schema

• connection.id is common to events for the same network connection. In success scenarios, this will correspond to a log with an event_type of established_connection and another with an event_type of closed_connection. Error states will not have a corresponding closed_connection event type.
• connection.client_ip corresponds to the IP address of the client as visible from the internet. This usually means that this is an Internet-facing NAT address.
• connection.resource_ip will always be the private IP address of the resource being accessed. For resources defined by a DNS address in Twingate, connection.resource_ip will be the local address as resolved by the Connector. See How DNS Works with Twingate for more information on how we handle DNS and local address translation.
• connection.rx and connection.tx represent bytes that have been received and transmitted during the lifetime of the connection.
• device.id is an internal ID that Twingate uses to identify the user’s device. This may not correspond to an ID presented by the device itself depending on the operating system. We will be standardizing this device ID in a future logging schema version.
• resource.address is the address of the resource as defined in the Twingate Admin console. If this is a DNS address, it will appear as defined here, with the resolved IP address logged in the field connection.resource_ip.

SIEM integration

Most SIEM solutions will support filtering and parsing via grok patterns or similar. The following example is a Vector configuration for filtering and parsing real-time connection logs.

[sources.twingate_connector]
type = "journald"
current_boot_only = true
include_units = ["twingate-connector"]

[transforms.tg_analytics_filter]
type = "filter"
inputs = ["twingate_connector"]
condition = """starts_with!(.message, "ANALYTICS")"""

[transforms.tg_analytics_transform]
type = "remap"
inputs = ["tg_analytics_filter"]
source = """.message = parse_json!(parse_grok!(.message, "ANALYTICS%{SPACE}%{GREEDYDATA:json_event}").json_event)"""
drop_on_abort = true