How Smartsheet built a remote MCP server on AWS

Smartsheet is an enterprise work management platform that hundreds of thousands of organizations rely on. As enterprise teams adopt AI agents, those agents need structured access to the data inside systems like Smartsheet, but most systems aren’t built for that. To bridge this gap, Smartsheet built a remote Model Context Protocol (MCP) server on AWS that gives AI clients direct access to its data and capabilities. AI assistants like Amazon Quick and Claude Desktop help users interact with Smartsheet’s capabilities through natural language, analyzing project data, updating tasks, creating sheets, managing workspaces, and more.

Enterprises are also building custom AI agents for workloads that run without human prompting. These AI agents can work autonomously in their roles, coordinating through Smartsheet using MCP. A few examples are capturing requirements, picking up tasks, attaching test results, drafting documentation. These happen in the same sheets their human counterparts use, compressing workflows that took weeks into days or hours.

The MCP server connects to Smartsheet’s existing APIs and central intelligence layer. It also adds an AI-optimized interface on top, designed to minimize token cost, help prevent hallucination, and help large language models (LLMs) work reliably with enterprise data. Since launch, Smartsheet saved over 3 billion tokens, based on internal telemetry, through these optimizations.

In this post, we cover a high-level view of the Smartsheet remote MCP architecture, with a focus on the AWS infrastructure behind it. This includes security, governance, scaling and deployment, and the AI-specific optimizations Smartsheet built on AWS.

One MCP layer serves both internal and external agents. Smartsheet’s own Smart Assist (the in-product AI experience) and externally connected AI clients like Amazon Quick run on the same infrastructure, with the same tools, optimizations, and intelligence stack. This parity is a deliberate architectural choice: Smartsheet builds once and every agentic client benefits immediately.

The architecturally critical AWS services in the data path are:

The detailed architecture flow is as follows:

Figure 1: Smartsheet MCP Server architecture on AWS.

Supporting services such as edge protection, container registry, observability, secrets appear in the relevant sections in the following sections.

On the scaling side, AI traffic differs from conventional request patterns. Agents autonomously orchestrate sequences of tool calls, firing several requests in a second as they work through a task, then going quiet while the model reasons. This bursty pattern demands a scaling strategy that responds to both sudden spikes and sustained throughput.

To handle and validate this pattern, Smartsheet built the MCP server to run on AWS Fargate for Amazon ECS. ECS Auto Scaling uses target-tracking policies, combining traffic volume with compute utilization. Compute-aware scaling matters here because each request involves server-side processing like LLM-optimized serialization, not only proxying. Extensive load testing under production-like traffic patterns validated that the infrastructure absorbs agent bursts without degradation.

On the deployment side, shipping updates without disrupting active agent sessions is equally critical. Container images are stored in Amazon Elastic Container Registry (Amazon ECR) and rolled out by the continuous integration and continuous delivery (CI/CD) pipeline through a layered safety net. ECS deployment circuit breakers detect failing containers during rollout and automatically revert to the last stable version. Manual intervention is not needed, and customer impact is avoided.

Deployments roll out to the smallest region first, following the AWS Well-Architected principle of reducing impact radius. After each region, automated end-to-end tests validate tool behavior against the live environment. Canary tests run every 15 minutes, executing a multi-step MCP workflow through the full authentication and gateway path. Results feed into the monitoring stack so degradation surfaces before customer reports. The ECS Fargate with ALB pattern is documented in the AWS Guidance for Deploying MCP Servers.

For enterprise customers, governance is the gating factor for AI adoption. Smartsheet built it into the tool framework itself: access control, error handling, and audit trails ship with every tool by default.

Access is tiered per organization: administrators can turn on AI access globally, restrict to non-destructive operations only, or open up full write and destructive capabilities, giving each organization control over their adoption curve. Tools carry MCP protocol annotations like readOnlyHint and destructiveHint so AI clients apply appropriate confirmation flows automatically.

The server emits OpenTelemetry signals (logs, traces, and metrics) across the full request lifecycle. Every tool invocation captures the maximum context possible within privacy constraints: user, organization, tool name, outcome, and more. This provides the foundation for usage insights and compliance auditing.

Agent traffic is harder to observe than traditional API traffic. A single user request can produce a chain of tool calls, and failures often trace back several steps. Smartsheet is extending its observability with agent-first identity and tracing, correlating context across tool chains. Logs stream through Amazon Kinesis into Amazon OpenSearch Service following the AWS Observability Best Practices pattern, with infrastructure metrics surfaced through Amazon CloudWatch. Datadog provides per-tool application performance monitoring (APM) visibility, and PagerDuty handles incident routing.

Every invocation also emits a structured analytics event through Amazon Simple Queue Service (Amazon SQS) into the Intelligence Layer. This closes the feedback loop: production usage data informs which tools to prioritize and how optimization strategies perform across real workloads.

The MCP server runs behind the same security infrastructure as Smartsheet’s production APIs. AWS WAF and AWS Shield are at the edge, with private subnets in a virtual private cloud (VPC), mutual TLS (mTLS) for service-to-service calls, and an OAuth2 proxy that rejects unauthenticated requests before they reach compute. The MCP server follows the defense-in-depth model in the AWS Guidance for Deploying MCP Servers. The API gateway layer handles authentication and scope validation. Domain services handle fine-grained permissions. If a user can’t access a sheet through the UI, they can’t access it through MCP either.

AI traffic adds a distinctive rate-limiting challenge. A single user question can trigger several tool calls in seconds. Many enterprise users sit behind shared corporate proxies making IP-based rate limiting unreliable.

To address this, Smartsheet implemented layered rate limiting through AWS WAF. Three layers work together: blanket protection at the outer edge, per-user metering using custom aggregation keys on an identity header, and path-specific controls for expensive operations. Per-user metering means sessions are metered individually rather than pooled by IP. The layered rate limiting follows the three most important AWS WAF rate-based rules pattern.

Testing non-deterministic AI workflows

Smartsheet maintains the standard testing layers: unit tests, integration tests, tool-level validation. The MCP server, however, introduces a testing challenge that traditional API services don’t face. A conventional API response gets rendered deterministically by the UI. An MCP tool response passes through an LLM first. The model interprets it, reasons over it, and generates what the user actually sees. That layer of non-determinism changes what “correct” means for testing.

Smartsheet invests heavily in end-to-end workflow tests that include the LLM in the loop. These tests simulate realistic business scenarios: creating workspaces, writing data, querying results, and verifying that the model’s interpretation makes sense to the end user. These tests run in the CI/CD pipeline (GitLab CI with runners hosted on AWS) and continuously as canary tests against each production AWS Region.

As enterprises scale AI agent deployments, token consumption becomes a real cost driver. Every tool response costs money at the LLM and competes for context window capacity. Most MCP tool calls today run without sub-agent orchestration. The agent calls tools directly, one at a time, reasoning between each step. Without intelligent tool design, this gets slow, expensive, and error-prone fast. Each tool call must be self-contained and efficient on its own, which is why Smartsheet optimizes at three levels:

Each tool response targets a token budget. The server dynamically calculates how many rows fit based on column count and data density. For example, a sheet with five columns can return more rows than one with 15 columns, but the total stays within budget. Whether a sheet has 50 rows or 50,000, the response size stays bounded. The model sees enough to orient, then narrows with filters based on what the user is actually asking about.

Metadata fields tell the model exactly what happened: is_sampled indicates whether data was truncated, rows_in_sheet gives the full count, rows_actual shows how many were returned, and filters_applied describes active filters. The model uses this to decide whether it has the full picture or needs to narrow its query with filters. Progressive disclosure is a server-side decision. The MCP server handles budgeting and sampling, while the metadata it returns gives the AI client the signals to orchestrate follow-up queries on its own.

Figure 2: Progressive disclosure in action: the AI client receives sampled data with metadata, then makes targeted follow-up requests.

Grounding the LLM is critical. Without constraints, models hallucinate parameter names, invent operators, and waste tokens on failed calls. Each tool publishes a strict JSON Schema through MCP’s tool discovery, generated from Pydantic models. Parameters are constrained to valid enums, column names are validated against the actual sheet before execution, and mismatches return structured errors with valid options instead of failing silently. Schema validation catches hallucinations at the boundary and means agents navigate the tool catalog reliably without trial and error.

JSON’s structural overhead (braces, quotes, repeated keys) typically consumes 15–25 percent of a response’s token count. For a server returning spreadsheet data with thousands of rows, the overhead adds up fast.

Smartsheet built a proprietary serialization format that alleviates this overhead. Key names appear once instead of repeating per row, and structural syntax is replaced by delimiters that tokenize more efficiently. On a representative 33-item filtered query, the optimized response is approximately 3,900 tokens versus approximately over 6,000 tokens for the equivalent JSON, roughly 35 percent fewer tokens carrying the same information. At 1,000 rows the gap widens further, because JSON repeats key names on every object while the optimized format declares them once.

AI agents integrate with Smartsheet through MCP today. In the first four weeks after general availability (GA), Smartsheet saw over 87 percent week-over-week user growth.

MCP is the distribution layer. What comes next is intelligence at the connection point itself. One example is resources that shape themselves to the person, team, and organizations using them. Another is agents that run autonomously on workflows, and a routing layer that lets specialists hand off reasoning to each other rather than starting cold on every step. Same MCP connection, different intelligence per customer, with no deployment required.

AWS is evolving its infrastructure to meet these emerging agentic requirements. Amazon Bedrock AgentCore provides runtime execution, discovery, personalization, and governance by default. Smartsheet continues to adopt and shape these capabilities with AWS.

The MCP protocol itself continues to evolve. Elicitations allow human-in-the-loop confirmation before destructive actions. MCP Apps bring interactive UI directly into AI conversations. Tasks support asynchronous, long-running operations. Smartsheet is evaluating these as they mature.

AI moves fast. Building on AWS gives us the infrastructure to keep pace whether that means new protocols, new models, or entirely new agent architectures.

To connect to Smartsheet’s MCP server, visit the AWS Marketplace listing, or see the Smartsheet MCP documentation.