Manage Amazon SageMaker HyperPod Spaces directly from SageMaker Studio

We recently introduced the ability to create and manage Amazon SageMaker Spaces on Amazon SageMaker HyperPod EKS clusters directly from the Amazon SageMaker Studio UI. Data scientists and machine learning (ML) engineers can now launch JupyterLab and Code Editor environments on HyperPod clusters without leaving their browser or using command-line tools, reducing the time from cluster access to productive development to a few clicks.

Amazon SageMaker HyperPod provides purpose-built infrastructure for foundation model (FM) training and inference at scale. With Amazon Elastic Kubernetes Service (Amazon EKS) orchestration, teams can run distributed training jobs across hundreds of accelerators with built-in resiliency and automatic fault recovery. In addition to training, HyperPod extends this EKS orchestrated infrastructure to serve low-latency, scalable inference for multi-billion-parameter foundation models.

Earlier this year, we launched Amazon SageMaker Spaces for HyperPod, an add-on ML developers can use to create interactive development environments directly on HyperPod EKS clusters. Organizations could then maximize their GPU investments by running interactive workloads alongside training jobs and model deployment on the same infrastructure, with support for fractional GPU allocations.

Previously, creating and managing Spaces relied primarily on the HyperPod CLI or kubectl commands. While this approach provides powerful, granular control for infrastructure administrators, data scientists who prefer a visual interface can now use this new SageMaker Studio capability to bypass command-line tools and focus entirely on model development.

With this new capability, data scientists can now create, configure, start, stop, and open Spaces directly from SageMaker Studio. The new IDE and Notebooks tab on the HyperPod cluster detail page provides a complete user interface for Space management, eliminating the need for CLI tools for day-to-day Space operations.

Key capabilities available through Studio include:

Figure 1: The IDE and Notebooks tab on the HyperPod cluster detail page shows all Spaces with their status, compute allocation, and quick actions to stop, open, or open in a remote IDE

Setup involves two roles: administrators prepare the cluster, and data scientists create and open Spaces. The following sections cover each.

Administrators install the SageMaker Spaces add-on on their HyperPod EKS cluster using either the Quick install (one-click with optimized defaults) or Custom install option (required to set up web UI access) from their SageMaker HyperPod EKS cluster’s IDE and Notebooks tab. After it’s installed, the administrator can configure namespaces, create Space templates, and manage access through EKS access entries.

The following is a one-time setup that administrators must do:

Run the following command once per Studio domain:

After updating the domain, verify that the following command returns “USER_IDENTITY”:

Existing running apps are unaffected. Users pick up the new setting on their next sign-in.

After the add-on is installed and access is configured, data scientists navigate to their HyperPod cluster in SageMaker Studio under Compute → HyperPod and select the IDE and Notebooks tab to see the Spaces management interface (see Figure 1). Learn more about creating and managing Spaces: Create and manage Spaces on HyperPod.

After the Space status shows Running (typically a few minutes on a cold cluster, approximately 30–40 seconds with over-provisioning), choose Open to launch JupyterLab or Code Editor in your browser (Figures 2 and 3), or Open in VS Code to connect from your local editor through SSH-over-SSM (Figure 4).

Figure 2: A JupyterLab Space accessed through the web browser, showing the Launcher with available notebook kernels, consoles, and terminal access

After you open a JupyterLab Space, you get a fully configured development environment with access to:

Your work persists on the attached Amazon Elastic Block Store (Amazon EBS) volume, so you can stop and restart Spaces without losing progress.

Figure 3: A Code Editor Space running on HyperPod, showing the VS Code-style web interface with file explorer, editor, and integrated terminal

For developers who prefer a VS Code-style experience in the browser, Code Editor Spaces provide a lightweight, web-based IDE with:

Code Editor Spaces are well-suited for writing and debugging training scripts, managing experiment configurations, and working with code repositories, all without leaving the browser.

Figure 4: A local VS Code instance connected remotely to a HyperPod Space, showing the remote connection indicator and full IDE capabilities running on cluster compute

Choose Open in VS Code from the Spaces table to connect your local Visual Studio Code to the Space running on HyperPod. This uses SSH-over-SSM tunneling internally, providing a secure connection without requiring you to manage SSH keys or expose port 22.

You get the full power of your local VS Code environment, including extensions, themes, and keybindings, while executing code on HyperPod cluster compute.

You can also connect using the AWS Toolkit for Visual Studio Code, which lists your Spaces under SageMaker AI > HyperPod, and you can start, stop, and connect to Spaces directly from the toolkit panel.

Every capability that follows is optional and composable. Enable any combination that fits your team’s needs.

By default, SageMaker Spaces on HyperPod EKS clusters using Karpenter autoscaling incur a cold-start delay of 5–7 minutes the first time a Space is created on a scale-to-zero cluster, dominated by:

For latency-sensitive interactive workloads (JupyterLab, Code Editor), you can maintain a pool of pre-warmed, image-cached nodes using the standard Kubernetes over-provisioning pattern. This drops Space startup time from minutes to around 30–40 seconds.

Learn more about Pro tip deployment: Overprovisioning for HyperPod Spaces.

Verified startup latencies on ml.m5.12xlarge (24 vCPU allocatable, 2 vCPU placeholder, 8 GiB placeholder memory) with the sagemaker-distribution:latest-cpu SMD image, which is about 3.5 GB:

Note: These numbers are CPU-only. GPU Spaces need their own placeholder Deployment requesting nvidia.com/gpu with the GPU image pre-pulled. Otherwise, GPU nodes stay cold, and at roughly 10 GB the GPU image costs far more to pull than the 3.5 GB CPU image.

Note: Each warm node holds one EC2 instance in the Running state. For on-demand nodes, this is an extra cost of keeping the warm nodes up and running.

For more information about installing the HyperPod Spaces add-on and getting started, see the AWS documentation.

Configuring the SageMaker Spaces add-on doesn’t incur additional charges. You pay for the underlying HyperPod cluster compute consumed by your Spaces, and a per-hour charge for the AWS Systems Manager Advanced On-Premises Instance used for SSH-over-SSM remote connectivity. See AWS Systems Manager pricing for details.

If you use the over-provisioning described earlier, note that there’s an added cost for warm nodes. Depending on the instance type and size, these nodes stay in the running state waiting to provision Spaces.

Managing HyperPod Spaces with SageMaker Studio bridges the gap between data scientists and high-performance compute infrastructure. Teams can now go from cluster access to a running JupyterLab or Code Editor environment in minutes, without needing to learn CLI tools or Kubernetes concepts. Combined with features like HyperPod Task Governance, fractional GPU support, and idle shutdown, organizations can provide self-service access to shared clusters while maintaining cost control and resource fairness.

To get started, navigate to your HyperPod EKS cluster in the SageMaker AI console and select the IDE and Notebooks tab. For more information, see the SageMaker HyperPod Spaces documentation.