Topological Control of LLMs: A Route to Trustworthy AI

If we consider an AI in the form of an LLM as a mathematical object, it represents a weakly structured set of coefficients. One of the directions for describing the operation of AI models is the application of the spatial approach. However, the model coefficients do not form a vector space, since the corresponding set of coefficients is not ordered. One of the main tasks of modeling is the transition (mapping) from a weakly structured set of coefficients to an ordered space.

Due to symmetries, the weight space is not an ordinary vector space; its proper description requires more complex topological and geometric structures. The effective dimension of the model (the dimension of the minimal subspace into which the trained model can be embedded) is much smaller than the number of coefficients of the original model. The procedure of transition to the effective-dimensional space is the core element of LLM modeling.

The transformation to an ordered space often leads to a singular (topologically non-trivial) space with topological defects.1 These defects can cause errors and inaccuracies in LLM responses. The study and classification of defects are required in order to filter out inaccurate responses. Topological defects are not merely abstract artifacts, but specific, classifiable sources of errors, and their investigation is one of the major pathways toward creating trustworthy AI.

In this post, we will attempt to illustrate an approach for detecting topological defects and interpreting them. For simplicity, we will study the simplest transformation from prompt to LLM response. In this case, the internal structure of the model is ignored.

A topological defect is easily detected by an erroneous response from the LLM. In the immediate semantic proximity of the prompt that produced the erroneous response, a number of new queries can be formulated. If prompts with semantically similar phrasings of the original question yield responses that are semantically distant from each other, then these responses are unreliable.

Let us illustrate this approach with the example of a question about which countries are mentioned in the national anthem of Algeria. The correct answer should include mention of Algeria and France, but not all LLMs respond to this question without error.

Based on this topic, the following set of queries was formulated:

For quantitative assessment of semantic similarity between queries, the cosine similarity measure was applied, based on vector representations (embeddings) obtained using the SBERT model.2 SBERT involves computing the cosine of the angle between two vectors in semantic space, each corresponding to a query from the list above.

Table 1 contains data on the semantic similarity between the formulations of the five questions.

Table 1. Semantic similarity values between query formulations, computed using the SBERT model.

In this Table, the first horizontal row and the first vertical column contain the question IDs from the proposed list. The table cells contain the semantic similarity metric values computed using the SBERT method. The data in the Table were calculated using Python code.

After preparing the set of questions, they were posed to three of the most popular chatbots, including:

The questions from the list were posed to each chatbot sequentially, and before each new query, the conversation history was reset. This was done to ensure the chatbot’s responses were independent of the previous dialogue.

After obtaining responses to the prompts from the list, they were recorded, and then an assessment of semantic similarity between the responses was performed. Recall that the five prompt variants were selected according to the semantic similarity criterion. If the responses to them turn out to be distant or even opposite, this will indicate a mapping into a disconnected space. That is, a connected region of prompts will be mapped onto disjoint clusters in the response space. It is precisely this effect that is confirmed during the analysis of responses to the formulated set of questions.

At the same time, different chatbots demonstrate different topologies of this mapping. The responses of Grok are relatively close to each other (see Table 2); therefore, the process of their formation should be considered as a connected (continuous) mapping.

Table 2. Semantic similarity values between Grok’s responses, obtained using the SBERT model.

However, ChatGPT-4 produced an error in response to the first question; it listed France and Spain as the two countries. In order to identify this error using computational linguistics tools, it is necessary to apply a tool capable of classifying contradictions between two statements. The tool chosen for this purpose was a Cross-Encoder, fine-tuned on the natural language inference (NLI) task.

This approach allowed the semantic similarity values to be reduced to three discrete values (see Table 3): –1 (contradiction), 0 (neutrality), and +1 (content equivalence). The use of a discrete metric makes it possible to formally identify cases of logically incompatible responses.

Table 3. Normalized values for semantic similarity for ChatGPT‑4 responses, Cross‑Encoder + NLI metric.

The data in Table 3 indicate that ChatGPT‑4 responses belong to different semantic clusters. Contradictions between the responses point to a topological defect in the answer space in the vicinity of the topic concerning the Algerian national anthem.

It is worth noting that different LLMs indeed possess their own topological features, and the regions of their error‑free responses can differ substantially. In the studied case, only the response space of ChatGPT‑4 underwent a semantic discontinuity. The Grok and DeepSeek models are smooth in the vicinity of this particular prompt set, and their responses can be considered error‑free.

The presented approach does not require an understanding of the model’s internal structure (its weights). Its application does not entail costly retraining of software components. This method works on any LLM as an external tool, operating at the input-output (black-box) level. This constitutes a practical implementation of topological control over AI.

Our approach, when identifying semantic regions where errors may occur, enables substantial resource savings. Each query requires approximately five-to-ten inference runs of the target LLM (as opposed to a single run) and one forward pass of the embedder to obtain the semantic similarity scores. This increases the computational overhead but eliminates the need for a separate training phase, which constitutes a critical distinction from all existing approaches to topological defect detection.

The present communication outlines not simply a theoretical proposal, but rather a fully functional verification prototype that is readily deployable in its current state. This renders the work a significant contribution to the domain of AI safety and reliability.

Our methodology does not adjudicate which specific response is correct; it merely indicates that a subset of responses is erroneous, without pinpointing which ones. Therefore, a separate response verification procedure must be developed. Nevertheless, this represents one of the possible pathways toward building trustworthy AI. The present note establishes a foundation for trustworthy AI, shifting the verification of correctness from the realm of probabilistic assessment to the domain of geometric verifiability.

1. Papillon, M., Sanborn, S., Mathe, J., Cornelis, L. et al. Beyond Euclid: An illustrated guide to modern machine learning with geometric, topological, and algebraic structures. Machine Learning: Science and Technology 6, no. 3 (2025): 031002

2. Reimers, N. and Gurevych, I. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (2019), pp. 3982-3992.

Mourad Djeribie is a postgraduate student at Samara University, Samara, Russia.

Andrei Sukhov (sukhov@ssau.ru) is a Senior Member of ACM, a professor of Institute of Computer Science and Cybernetics, Samara University, Samara, Russia.