Zero Trust for AI/ML at Scale: How H2O.ai Transformed Global Infrastructure Security
Twingate powers secure, infrastructure-as-code access for H2O.ai’s global teams, so they can focus on accelerating AI development without compromising security or control.
"Moving to Twingate has made a big difference in our day-to-day operations, especially for our global team. One of the first things we noticed was the improved network performance — particularly in APAC, where our engineers used to struggle with latency and flaky VPN connections. With Twingate, access is faster and more reliable across the board."
Ophir Zahavi
Senior Manager, Cloud Engineering
H2O.ai, the world’s leading agentic AI company, converges Generative and Predictive AI to build open-source and enterprise-grade platforms trusted by over 20,000 organizations—including more than half of the Fortune 500 - H2O.ai powers AI transformation for companies like Commonwealth Bank of Australia, AT&T, and Chipotle.
Operating across multiple cloud providers, the company runs intensive ML workloads on high-performance GPU clusters and manages sensitive proprietary data at scale.
Their global engineering teams spanning North America, Europe, and APAC move quickly and require frictionless, secure access to critical infrastructure. As the company scaled rapidly across continents, the team faced mounting challenges in securing its global teams and infrastructure without compromising developer agility.
Their previous patchwork of VPN solutions introduced performance bottlenecks and visibility gaps. To keep up with growth, H2O.ai needed an access model that aligned with their infrastructure-as-code mindset and scaled without introducing operational overhead. By making the switch to Twingate, the H2O team simplified access controls, reduced overhead, and implemented stronger security policies that actually enhance, rather than slow, developer workflows.
Complex Infrastructure, High Stakes
H2O.ai operates in a demanding environment:
Multi-cloud architecture spanning 3+ cloud providers
A globally distributed team, including substantial presence in APAC
High-value GPU clusters used for intensive ML training
Proprietary datasets and models requiring tight access control
An open-source-driven culture that values speed and collaboration
Rapid hiring, creating frequent onboarding and access needs
This infrastructure needed to support fast-paced development while protecting intellectual property and customer data. Security had to be built-in, not bolted on.
Ophir Zahavi, Senior Manager of Cloud Engineering, leads the DevOps and Platform Engineering teams responsible for ensuring this infrastructure is secure, scalable, and reliable, while also empowering developers to focus on building products that advance H2O.ai’s mission to democratize AI.
The Challenge: Fragmented Access and Security Gaps
As H2O.ai grew, its legacy access architecture, which relied on three separate VPN solutions, became increasingly difficult to manage. This patchwork caused performance degradation, particularly for APAC-based teams, and created inconsistencies in access workflows. Connection failures, client-specific quirks, and access delays were common. These issues translated into lost time, increased frustration, and a steady stream of support tickets for Zahavi’s team.
Consistent security visibility was limited. “We were basically guessing whether a device was secure, patched, and managed by our MDM," Zahavi said.
Onboarding new team members was another challenge, and often involved manual, multi-step configuration work: slow, error-prone, and resource-intensive.
As H2O deepened its commitment to GitOps practices, the need for a programmable, policy-driven access model became critical.
Selection Criteria: Built for Scale, Aligned with Practice
Zahavi and his team defined a clear set of criteria for replacing the legacy access stack:
Infrastructure-as-Code Compatibility
Declarative management via Terraform was essential. Access controls needed to be treated like any other infrastructure change: automated, peer-reviewed, and versioned.Global Performance & Reliability
Consistent, low-latency access across all regions was a requirement. With teams across multiple continents, performance parity was essential, especially in APAC.Seamless Developer Experience
Access workflows needed to integrate into daily engineering routines without added friction. That meant eliminating the need to juggle multiple VPN clients and credentials.Security Posture Enhancements
Device posture enforcement, least-privilege access, audit visibility, and geo-restrictions were all key requirements. The new solution needed to support these controls natively and at scale.Integration with Existing Identity & Workflows
Compatibility with Okta for centralized authentication and support for internal tooling used for access approvals and policy management were must-haves.
Building the Business Case
Security has always been a core value at H2O.ai, not just for compliance, but because protecting customer data and intellectual property is mission-critical.
"Security is everyone's responsibility, not just the security team's, and that mindset has to be built into the culture from day one,” said Zahavi. “Personally, I focus on leading by example: we treat every infrastructure change as code, go through proper reviews, and constantly ask, 'Is this the secure way to do it?'"
That strong security-first culture made it easier to build internal consensus around replacing the legacy VPN stack.
The business case centered on three quantifiable pain points:
Lost developer productivity, especially among remote teams
Operational overhead, particularly around onboarding and support
Security risk, stemming from unmanaged devices and fragmented visibility
Once Twingate showed immediate improvements in reliability, simplicity, and enforcement, internal alignment followed quickly.
"Getting buy-in wasn't hard, honestly. The pain was shared across teams. Once they saw how seamless the new approach was - no more juggling VPN clients, no more unreliable connections - adoption came naturally," Zahavi recalled.
Implementation Approach
Infrastructure-as-Code Deployment
Twingate was rolled out using Terraform, integrating directly into H2O.ai’s infrastructure-as-code pipeline. This allowed access policies and connectors to be deployed, peer-reviewed, and versioned alongside other infrastructure changes—enabling consistency, transparency, and automation.
"Our goal was to make the Zero Trust rollout feel like a natural extension of our existing workflows,” Zahavi said.
Phased Rollout, Zero Disruption
The team began by securing high-impact production environments, then expanded to development and staging. This phased approach minimized risk, gave teams time to adapt, and allowed the DevOps group to collect feedback and refine policies incrementally.
Unified Access Management
H2O.ai saw immediate value in Twingate’s native integration with Okta for SSO. Twingate became their centralized management pane, providing visibility across users, devices, and access attempts, supported by detailed audit logging.
From there, the team built their own custom internal tools for just-in-time access, guaranteeing access is given only when it is needed.
Strengthened Security Posture
H2O.ai significantly improved their security posture without impacting development speed thanks to Twingate’s:
Native Device Posture Enforcement: Access is now conditional on meeting device requirements, including OS version, disk encryption, and screen lock. This significantly reduces the risk of unmanaged or outdated devices reaching sensitive environments.
Geo-Restriction Controls: Access to critical production systems and internal tools is now limited based on geographic location, adding another layer of control on top of identity and device verification.
Comprehensive Audit Logging: Twingate’s detailed logs, combined with H2O.ai’s custom access request tooling, provide a clear audit trail - key for ongoing compliance efforts.
Internet Security: With native DNS filtering and content protection, the team can block risky domains and apply consistent security policies even when devices are off-network.
“Twingate helped us take our security posture to the next level,” Zahavi said. “Teams only see and access exactly what they need, nothing more. It's a huge improvement over the all-or-nothing access we had with traditional VPNs."
Results & Outcomes
Developer Productivity
Twingate eliminated access-related friction, particularly for remote and APAC-based teams. Developers now get reliable, always-on access through SSO, without managing multiple clients or credentials.
“Developers don't have to think about how to get access or whether something is compliant. It just works, securely and reliably," Zahavi said.
Operational Efficiency
Support overhead dropped significantly, as did provisioning time for new hires. The DevOps team reclaimed 16+ hours per month, the equivalent two working days, in administrative effort.
Resilience at Scale
Twingate’s architecture has proven resilient. With 99.99% uptime, global teams like H2O.ai’s benefit from consistent, high-performance connectivity, no matter where they’re connecting from.
H2O.ai’s shift to a Zero Trust framework with Twingate showcases how security can be tightly integrated with developer workflows, enhancing rather than hindering innovation. For AI/ML organizations scaling globally and handling high-value infrastructure and data, this approach provides a blueprint for combining robust access control with operational agility.