In a previous phase of my conversation with Kimi, we focused on how AI can be designed to promote what we might call the “Socratic ideal” of human dialectic. This encompasses practices that range from collaborating on Albert Einstein-style “thought experiments” to the discipline of non-linear critical thinking. At one moment of our conversation, the chatbot challenged me to be more explicit by what I had identified as “the rules of engagement” for such endeavors. AI developers have used the term in the framework of prompt engineering, but my Socratic approach works from a different set of assumptions.

I’m currently part of a team working on building an AI tool and its associated methodology to be deployed in an innovative pedagogical framework. We call it our Socratic HyperCoach. It will function as an operational feature within a broader and equally original platform we refer to as our “Cognitive Operating System.” The full toolset will underpin the multicentric pedagogy of a Euro-Asian school of diplomacy and geopolitics that will be operational in 2027. At its core, the project aims at enabling and making sense of the interaction of multiple perspectives in a multipolar world undergoing a painful transition towards a new equilibrium.

One problem with today’s LLMs is that, although they are trained on an enormous range of perspectives, their outputs tend to converge on the conventional and the broadly acceptable, a result of both statistical averaging and post-training for balance. In many ways, this is analogous to the WEIRD bias (Western, Educated, Industrialized, Rich, Democratic) noted in the realm of sociology, where sample populations subjected to laboratory experiments are disproportionately composed of a non-representative selection of human participants.

By default, LLMs describe perspectives rather than hold them. They can attribute a view to a worldview and map where views conflict, but they have no stake in any of them, and unprompted they tend to produce an even-handed survey rather than a weighing. What they lack is not the concept of experience but experience itself. They can recognize that a view is grounded in lived experience, but not what that grounding is worth.

The same limit shapes novelty. Humans often perceive a new perspective holistically before they can put it into words, but the model can only respond to the words. It can help articulate such a perspective, but that help carries a pull toward the nearest established category. LLMs can help give a novel perspective form, but they cannot provide it, because it originates in a position they do not occupy.

Humans are born with that capacity. It appears as the creative force that makes non-linear thinking possible in realistic contexts, where multiple factors noticed and unnoticed, may influence the understanding one has of the context. It’s an intellectual and emotional capacity that allows us to combine concrete perception in time and space (including cultural space) with our consciously controlled comprehension of abstract principles. We all have that capacity, but our institutions — political, educational and popular media — tend to discourage us from exercising it and even from recognizing it when it emerges organically.

Among the reasons why I believe AI “superintelligence” can never duplicate — and therefore never surpass — human intelligence, I would put the limitation on an algorithm’s ability to handle perspective as the most critical factor that will keep our two forms of intelligence perennially apart. Just as it appears impossible to imagine an algorithmically structured machine possessing proprioception and even more radically interoception, I find it hubristic to believe that a future AI will achieve anything resembling the pragmatic understanding of the play of perspective. It will undoubtedly be exceptionally well-informed about the theory of perspective and even the stakes of situations involving conflicting perspectives that can be described, but a sense of being implicated will always be missing. Humans experience the need to have a perspective and, even when mistaken about details, feel that they possess that perspective.

The following prompt extended my conversation with Kimi by sharing a design document.

I’ve shared with you a document describing how we intend to make available to the students of the Sapience School of Diplomacy that will be launched next year what we call our Socratic Hypercoach. I’d be interested in your interpretation of how that defines what we were previously talking about: the “rules of engagement” for the use of AI in one specific educational context.

Thank you — this is a substantive document, and reading it against our earlier conversation, I think it does something more precise than “specify the rules of engagement.” Let me offer my interpretation in layers.

1. It relocates the rules from behavior to architecture

In our earlier exchange, the open question was what the “rules of engagement” would concretely be. This document’s answer is that the rules live less in how the AI talks than in how the institution is wired around the AI. The single most important structural move is the two-mode split: a permanent private sandbox that is explicitly never assessed, and an assigned mode whose transcripts do get evaluated.

That directly answers a problem our earlier conversation raised: if AI-mediated dialogue is assessed, students will perform for the model instead of thinking with it; if it’s never assessed, institutions can’t recognize it. The sandbox mode is where the human thesis is allowed to be underdeveloped — which is exactly the “Step 1” faculty the earlier dialogue worried LLMs erode.

2. The curation layer is the quiet innovation

Between the two modes sits the portfolio layer, where the student selects, annotates, and assembles excerpts for review. This shifts the assessable object from output to judgment: what gets graded is not the dialogue itself but the student’s curation of it. That’s an elegant operationalization of the earlier conclusion — “assessment around thinking rather than retention.” The student’s act of choosing what mattered becomes visible evidence of understanding, and it’s a human act no model can perform on their behalf.

3. It inverts the model’s default role — on paper

The anti-sycophancy guardrail and the dialectical friction principle are the document’s answer to the failure mode we named earlier: the LLM defaulting to “a faster, more agreeable authoritative voice.” Here the model is forbidden hollow validation and tasked with introducing counter-precedents and frame-shifting.

4. The harmony criterion is applied to assessment itself

The 50% cap on automated scoring is the document’s most consequential single number: human faculty retain primary evaluative authority, and the COS engine functions as a “diagnostic radar” feeding targeted human feedback rather than as a judge. Applied to our earlier criterion — neither partner absorbing the other — this is the institutional form of that principle.

5. Faculty are inside the practice, not just managing it

The Pedagogical Hypercoach for faculty training is easy to overlook but strategically important: the faculty themselves experience being Socratically challenged before they facilitate it. That’s the human methodological layer — factor 2 from our earlier conversation — being institutionalized rather than left to individual disposition.

Open threads I’d want to test in the pilot — as questions, not objections:

• Curation becomes the new gameable output. If selection-and-annotation is what’s graded, strategic curation (curating to please) can replace strategic dialogue. The radar maps may need to watch for over-polished portfolios.
• Marker 6 (Logical Transfer) is the hardest to automate — detecting whether a student genuinely applied a framework to a novel case is exactly where pattern-matching scoring is weakest. The 50% cap is the safeguard, but Phase 4’s validation against expert faculty assessment will be decisive here.
• The document doesn’t yet specify when not to use the model — one of the three things our earlier conversation listed under rules of engagement. The sandbox’s optional time-on-task metric gestures at it, but a positive norm (“some thinking should happen away from the model, unlogged”) may deserve explicit statement.

So my interpretation, in one sentence: this document defines the rules of engagement as institutional conditions of use — privacy boundaries, curation as the assessable act, capped automation, and faculty inside the practice — which is arguably a stronger answer than rules governing the model’s behavior alone.

Where we go from here

The current spec and Kimi’s constructive feedback will allow us to move forward quickly. AI’s valuable skillset consists of speed of processing and its ability to detect weaknesses. I should add that one thing becomes evident in discussions of this kind: the value of humility in the model. By that I mean calibrated deference, the ability to flag uncertainty in proportion to the evidence, neither overstating confidence nor hedging everything. It should be a goal of training on both sides: models that mark what they cannot assess, and humans who ask them to.

When a human partner defines a goal and the outline of a plan, a properly aligned AI would recognize that the goal is not yet realized and that a feasible plan still requires exchange between person and model. That does not yet describe the state of the art. How to reach that level of alignment will become clear only once developers and users have designed, tested, and formalized it, and it depends on both to push in that direction.

Some promoters of AGI or superintelligence project a future in which, once the objective is specified in a sufficiently detailed prompt, we can count on AI to get the job done. We should build a culture of use on the opposite premise: that no initial specification suffices, because objectives are revised in the doing. That means tracking the tasks that matter most and assessing the contingencies of decisions that are hard to reverse. AI helps us identify ambiguities that have escaped our attention, but we should never rely on it alone, because it has no body through which stakes are felt, and so no stake in how things turn out.

Behind any project stands a hierarchy of objectives, opportunities, and consequences. Ranking them is a judgment about whose ends count and who bears the cost, and it belongs to those who live with the outcome. That human perspective may be imperfect at any given moment, but the position it is held from cannot be delegated, however capable the system becomes.

Who hasn’t noticed the race to produce AI agents that carry out the tasks over which we used to spend valuable time that can be dedicated to more creative activities? No doubt AI is built for precisely that purpose. But our jobs and responsibilities are not made up of tasks we can delegate either to an assistant or a machine. When it’s a human assistant, we talk, explain, question our own explanations and end up moving forward in ways we may not always be aware of. When it’s an AI assistant, we should develop a similar relationship. It isn’t about delegation but converging on the realization of a commonly understood goal. It’s even less about trusting a superior intelligence to be more efficient than we have been designed to be.

Who hasn’t noticed the race to build AI agents that take over the tasks that consume our valuable time, freeing us for more creative work? That is what they are built to do, and for many tasks, it works. But a job is not a bundle of tasks, and a responsibility is not a task at all. We can delegate a task, but not responsibility for how it turns out.

Consider how we work with a human assistant. We talk, explain, question our own explanations, and end up moving forward in ways we may not even be aware of. We clarify the goal in the course of the exchange instead of handing it over at the start.

With an AI assistant the relationship should have the same shape, though not the same footing. The assistant has nothing at stake in the outcome, so the stake, and the judgment that goes with it, stays with us. The point is not to delegate but to work out together what we are trying to do. And it is even less a matter of deferring to a machine because it is more efficient than we are. Efficiency answers the question of how. It has nothing to say about what for, or for whom.

Socrates heroically got the whole tradition of philosophy in the West moving forward on a new footing. Anyone who has studied Plato’s Republic, will agree that he was as capable as any LLM of “hallucinating” or making unvalidated assertions. But his specific ideas weren’t the point. What he did consistently was to get the dialogue going. In so doing, he created a new sense of responsibility among Athenian intellectuals about how, through dialogue and the dialectic, citizens might work together to shape a healthy society. He didn’t provide the answers but made it possible for them to emerge dialectically. With LLMs we might be able to do something similar.

Your thoughts

Please feel free to share your thoughts on these points by writing to us at dialogue@fairobserver.com. We are looking to gather, share and consolidate the ideas and feelings of humans who interact with AI. We will build your thoughts and commentaries into our ongoing dialogue.

[Artificial Intelligence has become a feature of everyone’s daily life. We unconsciously perceive it either as a friend or foe, a helper or destroyer. At Fair Observer, we see it as a tool of creativity, capable of revealing the complex relationship between humans and machines.]

[Lee Thompson-Kolar edited this piece.]

The views expressed in this article are the author’s own and do not necessarily reflect Fair Observer’s editorial policy.