How DiDi built intelligent contact center QA with Amazon Bedrock

DiDi partnered with AWS to build an intelligent contact center quality assurance (QA) system on Amazon Bedrock for its International Business Group’s Customer Experience (CX) department. The system covers Spanish and Portuguese across three business lines (ride-hailing, food delivery, and financial services) and migrates QA capabilities from an opaque third-party solution to a transparent, self-owned AI architecture. It comprises three core pipelines: intent verification, compliance evaluation, and Voice of Customer (VOC) analysis. In DiDi’s production validation, intent verification accuracy improved from 38 percent to 86 percent, compliance scoring accuracy exceeded 90 percent, and VOC analysis compressed hours of manual summarization into minutes.

In this post, we walk through how DiDi International Business Group and AWS built a self-owned, transparent contact center QA system on Amazon Bedrock, replacing an opaque third-party solution. We describe the three core pipelines that power the system: intent verification, compliance evaluation, and Voice of Customer (VOC) analysis. We also explain the design rationale behind each. Along the way, we show how a single principle, precise context management (controlling exactly what information the model sees in each call), drove the key results, and VOC trend analysis compressing hours of manual work into minutes.

About DiDi International Business Group

DiDi International Business Group (IBG) is the overseas arm of DiDi Global, operating across 14 countries and regions with three business lines: ride-hailing, food delivery, and financial services, serving tens of millions of users. The CX department processes a large volume of Spanish and Portuguese tickets each month across live chat and phone channels, where service quality directly affects user retention and brand trust.

As the business scaled and QA standards evolved rapidly, the existing third-party QA solution lacked transparency and flexibility. DiDi’s IBG CX team faced four core challenges:

To address these challenges, DiDi’s IBG CX team partnered with AWS to build a self-owned intelligent QA system on Amazon Bedrock. The team chose Amazon Bedrock for three reasons. First, its model-agnostic access to a broad selection of foundation models through a single API means the team can choose the best-fit model for each pipeline without re-architecting. Second, its built-in governance and security controls keep sensitive customer service data within DiDi’s network boundary and deliver the transparency and oversight the previous third-party solution lacked. These controls include private connectivity through Amazon Virtual Private Cloud (Amazon VPC) endpoints powered by AWS PrivateLink, encryption in transit and at rest, and fine-grained access control through AWS Identity and Access Management (IAM). Third, Amazon Bedrock Guardrails provides configurable safeguards such as content filtering and sensitive information redaction, giving the team a foundation for building the QA system in line with responsible AI practices.

The system implements three specialized pipelines on Amazon Bedrock, each targeting a different QA dimension and producing a complete reasoning chain alongside every judgment. The intent pipeline verifies whether contact reasons assigned by representatives are correct. The evaluation pipeline audits service compliance and extracts business insights. The VOC pipeline aggregates large volumes of similar tickets to surface systemic trends.

The following diagram shows the end-to-end system architecture, from dual-channel data ingestion through preprocessing into the three pipelines powered by Amazon Bedrock.

Figure 1: System architecture. Dual-channel data flows through preprocessing into three parallel pipelines powered by Amazon Bedrock, producing structured outputs with full reasoning

The preprocessing layer ingests live chat and phone data, normalizes it into a common schema, and fans out to three parallel pipelines. The overall workflow is as follows:

Because the system scores and classifies real customer conversations at scale, responsible AI controls are built in. We use Amazon Bedrock Guardrails to mask sensitive information such as personally identifiable information (PII) before it reaches the model and to apply contextual grounding checks that flag ungrounded responses, reducing hallucinated judgments. Beyond Guardrails, the system does not treat the model’s output as final: for rule-deterministic criteria, a programmatic post-validation layer re-checks the model’s judgments against the raw conversation. For example, reported spelling errors are verified only against the agent’s own messages, and the pass or fail threshold is applied on the verified count rather than the model’s own tally. Computable facts, such as agent response-wait times, are derived deterministically in code and injected into the prompt rather than left to the model to infer. Together with a full reasoning chain attached to every score for human review, these controls keep the system’s compliance decisions auditable and reliable in production.

Intent pipeline: Information isolation for accurate classification

During ticket handling, the representative assigns contact reason labels. DiDi’s contact reason taxonomy (CR Tree) branches from broad categories into progressively finer subcategories across many levels. The deeper the hierarchy, the more subtle the distinctions, so mislabeling is inevitable at scale. The system must automatically verify whether these labels are accurate and recommend alternatives when wrong. It also audits the CR Tree in reverse, identifying coverage gaps, suggesting new labels, and closing the loop between QA and taxonomy maintenance.

The most direct approach was to feed the complete CR Tree plus conversation to the LLM in a single call. This approach produced accuracy far below expectations. We identified the root cause: when the LLM sees the complete list of options, it automatically compares them one by one. Even when the original label is entirely reasonable, finding a slightly more precise alternative triggers a judgment that the current label is wrong. Multiple rounds of prompt tuning could not change this behavior. The problem lay in context management, not prompt wording.

The team redesigned the pipeline in two ways:

Through this two-level design, intent verification accuracy improved from 38 percent to 86 percent in DiDi’s production validation.

Evaluation pipeline: Dynamic assembly for multilingual, multi-business-line coverage

The evaluation pipeline scores multiple compliance items and extracts business insights for each ticket in one pass. Given the combinatorial complexity of languages and business lines and frequently changing QA standards, maintaining a separate prompt for each combination is unsustainable.

The solution is a unified prompt template with dynamic variable injection. Language context, business-line context, and each criterion’s definition and judgment rules live as external configurations. At call time, the system assembles them into a complete prompt based on the ticket’s metadata. Adding a new evaluation item, language, or business line requires only a configuration update. One template covers all combinations in a single LLM call.

The following diagram illustrates how the evaluation pipeline assembles external configuration into a unified prompt and produces structured output.

Figure 2: Evaluation pipeline. External configuration dynamically assembled into a unified prompt, processed by Amazon Bedrock to produce structured compliance scores and business insights

The following example is simplified to illustrate the two mechanisms working together: dynamic prompt assembly from external configuration, and the Tool Use capability of Amazon Bedrock forcing the model to return schema-validated JSON.

The Tool Use capability in Amazon Bedrock constrains output to structured JSON, with each score containing both the judgment and its reasoning chain. For rule-deterministic metrics, the system applies programmatic post-validation, using deterministic logic to calibrate the model’s semantic judgments.

In production validation, compliance scoring achieved over 90% average accuracy. The pipeline also outputs business insights (such as issue resolution rate and customer satisfaction). Because every judgment carries a reasoning chain, QA becomes a two-way improvement mechanism: representatives can understand why a score was assigned and adjust their service accordingly.

VOC pipeline: Multi-stage processing for proactive trend discovery

The VOC pipeline is triggered on demand to analyze large batches of contacts within a specific time window, surfacing high-frequency issues that warrant systemic action. The challenge is extracting structured insights from thousands of conversations efficiently. Feeding them all to the LLM at once produces overly coarse categories and buries critical details.

The VOC pipeline uses a three-stage approach: parallel extraction to issue clustering to report generation, with each stage precisely controlling the information scope visible to the LLM.

Example: When cancellation fee complaints surged across Latin American markets within a short period, the operations team triggered a VOC analysis. Within minutes, the system identified the primary root causes and high-frequency trigger scenarios from multilingual conversations, producing a structured report with actionable recommendations. This work previously took hours of manual reading and summarization.

Reflecting on the collaboration, DiDi’s team summarized both the measurable outcomes and the deeper lesson the project delivered:

“In international operations, multilingual and multi-business customer service QA is a scaling challenge — the existing solution was not only opaque, but also difficult to iterate quickly as business standards changed. After rebuilding this system with Amazon Bedrock, we achieved full transparency of QA judgments: intent verification accuracy improved from 38% to 86%, compliance scoring accuracy exceeded 90%, and trend analysis across large volumes of tickets was compressed from hours to minutes. This project taught us something fundamental: building reliable LLM applications depends not on the tool itself, but on the team’s deep mastery of context management and data definitions. That capability cannot be outsourced — and it is the most valuable outcome of this collaboration.”

— Raphael Hua, Data Analytics Team, DiDi International Business Group

DiDi’s intelligent QA system, built on Amazon Bedrock, achieved the transition from a third-party opaque system to transparent, controllable quality assurance through three pipelines:

The core takeaway: the performance bottleneck of LLM applications often lies not in prompt wording, but in context management. Precisely controlling what information the model sees in each call matters more than how you phrase the instruction. This principle runs through all three pipelines: information isolation in intent verification, dynamic assembly in evaluation, and multi-stage processing in VOC analysis.

DiDi plans to extend the system to additional business lines and languages, explore cross-pipeline data integration, and build deeper analytical capabilities on top of the existing foundation.

To learn more about the services used in this post, see the Amazon Bedrock User Guide. For related guidance on building LLM applications and contact center AI on AWS, see AI | AWS Contact Center. If you’re interested in building similar intelligent QA systems, explore Amazon Bedrock or contact the AWS Generative AI Innovation Center.