Inside the Neocloud (Part 1): When the Container Is the Only Wall

Some GPU cloud and neocloud providers use containers to share computing resources between customers. It’s an efficient approach, but it raises an important security question: What happens when a tenant container can reach the infrastructure underneath it?

During our research, we found an environment where a simple HTTP request from a tenant container returned the administrative credentials of the underlying host.

No sophisticated exploit. No container escape vulnerability. Just a gap in how different infrastructure components were configured.

Where the Tenant Boundary Sits

GPU cloud providers offer compute resources in different ways, each with its own security boundaries:

• Bare metal: Customers control an entire physical server, including its operating system.

• Virtual machines: Customers run separate operating systems isolated by a hypervisor.

• Containers: Customers share the host’s operating system kernel, relying on Linux isolation mechanisms to separate workloads.

Some neocloud providers combine these approaches, running tenant containers inside host VMs on physical GPU servers.

Below each container sit the host operating system, storage, networking, and management services. These components may have access to infrastructure that customers should never control.

Container isolation is only effective when the surrounding infrastructure enforces the same security boundary.

The Hidden Risk in Metadata Services

GPU cloud providers use provisioning and management systems to deploy servers, configure operating systems, and manage infrastructure.

Tools such as Canonical MAAS, OpenStack Ironic, Nova, and custom orchestration platforms help automate these operations.

Some environments also rely on metadata services to provide machines with configuration information, including SSH keys, startup scripts, user accounts, and sometimes administrative credentials.

A commonly used metadata endpoint is:

http://169.254.169.254

This service is intended to provide configuration information to an instance. However, when tenant containers run inside that instance, certain network configurations can unintentionally allow them to access metadata intended for the host.

That’s exactly what we encountered.

What We Found: One Request, Host Credentials

While examining a GPU cloud environment, we discovered that a tenant container could access the metadata service of its underlying host VM.

From inside the container, we sent a simple request:

curl -s http://169.254.169.254/latest/user-data

The response included the host’s configuration:

#cloud-config
users:
- name: admin
plain_text_passwd: <PASSWORD>
lock_passwd: false
sudo: ALL=(ALL) NOPASSWD:ALL
ssh_pwauth: true

The account name and password have been changed.

The tenant received the host’s administrative password, along with configuration enabling SSH password authentication and passwordless sudo.

Why Did This Work?

In this environment, the container used Docker bridge networking.

When the container sent a request to the metadata endpoint, network address translation (NAT) made the traffic appear to originate from the host VM.

The metadata service responded with the host’s configuration because it treated the request as coming from the instance itself.

The metadata service responded as designed, but the platform failed to isolate host-level configuration from tenant workloads.

From Container Access to Host Compromise

Exposing host credentials creates a risk far beyond the container itself.

If the host’s SSH service is reachable and accepts those credentials, an attacker could authenticate to the host and gain administrative access.

From there, the potential impact includes:

• Other workloads: Access to services and containers running on the same host.

• Sensitive data: Exposure of local storage, configuration files, and credentials.

• Management networks: Access to internal infrastructure services normally hidden from tenants.

• Control plane systems: Potential access to management APIs and agents connected to the provider’s infrastructure.

Host access doesn’t automatically mean control over an entire GPU cluster. The impact depends on network segmentation, available credentials, and the permissions of the compromised machine.

However, a tenant gaining administrative access to the underlying host represents a critical failure of the isolation boundary.

What Should Providers Check?

Preventing this type of exposure requires more than securing the container runtime.

1. Restrict metadata access. Ensure tenant workloads cannot retrieve host-level metadata. Where supported, token-based access and appropriate network hop limits can help reduce exposure.

2. Protect administrative credentials. Avoid storing plaintext passwords or unnecessary privileged credentials in metadata and instance configuration.

3. Segment management infrastructure. A compromised host should not automatically provide access to provisioning systems, management controllers, or other infrastructure.

4. Validate tenant isolation. Test what customer workloads can actually reach, including metadata endpoints, host services, internal APIs, and management networks.

The Bigger Picture

This finding highlights a security challenge we’ve encountered in neocloud environments: individual infrastructure components can work as designed while their integration creates unexpected security gaps.

In this case, a single HTTP request exposed credentials that could potentially allow a tenant to take control of the underlying host.

The issue wasn’t a vulnerability in the container runtime or the metadata service itself. It was a failure to enforce the security boundary between them.

This research relates to #3: Unsafe Multi-Tenant Isolation & Resource Reuse in the OWASP Top 10 Neocloud & AI Data Center Security Risks.

Tenant isolation isn’t just about securing the container. It’s about securing everything the container can reach.

Inside the Neocloud is a three-part research series by Lava exploring security risks uncovered during real-world assessments of GPU clouds and AI infrastructure.

Explore the other articles in the series:

Part 2: When the Management Network Isn’t Isolated

Part 3: When InfiniBand Isolation Fails