The Brittle Compass: Navigating LLM Prompt Sensitivity in Slovak Migration Media Discourse

Why LLMs Are a "Brittle Compass": Prompt Sensitivity and the Future of AI Media Analysis

By · Published November 4, 2026 · Updated November 4, 2026

Ask a large language model the same question twice and you may get two different answers. Change one word in your instructions and the entire tone of the response can shift. That is not a bug you can patch away, it is a fundamental property of how these systems work. New analysis framed around a concept called "the brittle compass" takes this problem seriously, using an unusually demanding test case: how large language models interpret news coverage of migration in Slovak media.

The image is a sharp one. A compass is a tool you trust to point you in the right direction. But a brittle compass, one that wobbles, drifts, or snaps under pressure, is worse than no compass at all, because it gives you the feeling of direction without the reliability. That is exactly the risk facing every organization that now uses AI to read, sort, and summarize the world's information.

What "Prompt Sensitivity" Actually Means

When you send a request to a language model, the exact wording of that request is called a prompt. Prompt sensitivity is the measure of how much the output changes when the prompt changes, even in small ways. Swap "summarize" for "describe." Add the phrase "be neutral." Ask about "migrants" instead of "migration." Reorder two sentences. A robust system would return roughly the same meaning every time. A sensitive system can swing wildly.

This matters more than it first appears. Most people imagine AI analysis as a stable measuring instrument, like a thermometer. In reality, it behaves more like a conversation partner with a strong urge to please. It reads subtle cues in your wording, your framing, your emphasis, even your implied expectations, and shapes its answer accordingly. When the topic is politically charged, those cues can quietly steer the result.

Why Language and Topic Both Matter

The choice of Slovak media as a testing ground is not accidental. It layers two difficult conditions on top of each other.

First, language resource imbalance. Most large language models are trained overwhelmingly on English text. Slovak, a West Slavic language spoken by a relatively small population, has far less training data behind it. Models can still produce fluent Slovak, but their grip on nuance, idiom, and local political context is thinner. That thinness is exactly where sensitivity creeps in.

Second, topic sensitivity. Migration is one of the most politically loaded subjects in European media. Word choice carries weight: whether a person is called a "migrant," a "refugee," or an "arrival" signals an entire political stance. A model asked to classify or summarize such coverage is being asked to handle a topic where neutral language is itself contested. Small prompt changes can flip a story's apparent framing from sympathetic to hostile, and the model may not flag that it has done so.

The Deeper Problem: Fluency Creates False Confidence

The most dangerous part of prompt sensitivity is that it rarely looks like failure. A model that gives you a confident, well-written, grammatically perfect paragraph does not announce that a slightly different prompt would have produced a contradictory one. The output feels authoritative regardless of whether it is stable.

This is the core insight behind the "brittle compass" framing. In traditional software, a bug produces an error message. In language model analysis, instability produces a polished answer. Teams that build workflows on top of these outputs can end up with dashboards, reports, and strategy documents built on measurements that would move if someone rephrased the question.

For media monitoring specifically, the stakes are concrete. If an organization tracks public attitudes toward migration across outlets, it needs to know whether a shift in its data reflects a real change in coverage or a subtle change in its own prompt. Without that distinction, the analysis is not measurement at all, it is noise dressed as insight.

What This Means for the Future of AI

1. Evaluation Will Shift from Accuracy to Stability

For years, the dominant question about AI systems has been "how often is it right?" The prompt sensitivity problem points toward a different and harder question: "how often does it stay the same?" A model that is right 85% of the time but gives inconsistent answers across rephrasings is far less useful for decision-making than a model that is right 80% of the time and never wavers.

Expect stability testing, deliberately rephrasing the same task dozens of ways and measuring the spread of results, to become a standard part of how AI systems are evaluated and procured. Buyers will start asking vendors not just for benchmark scores but for variance under paraphrase.

2. Low-Resource Languages Become a Front Line

The Slovak case is a preview. As AI spreads into media analysis, public administration, and compliance work in smaller language markets, the reliability gap between well-resourced and under-resourced languages will become a governance issue. A tool that works acceptably in English may produce unstable results in Slovak, Czech, Slovenian, or Estonian. Organizations operating across European markets need to test per language, not assume a single quality bar.

3. Human Judgment Moves Up the Stack

If the same question can yield different framings, the human role shifts from checking individual outputs to designing and stress-testing the questions themselves. Prompt design becomes a discipline closer to survey methodology, where wording effects are a known, studied hazard, than to simple instruction-writing.

Practical Implications for Businesses and Society

The consequences reach well beyond media research. Any organization using language models to interpret unstructured text faces the same brittleness.

For society, the risk is subtler but larger. If media-monitoring tools systematically misread framing in smaller-language markets, the picture of public opinion that policymakers and journalists rely on can drift, not because anyone lied, but because the instrument wobbled.

Actionable Insights: How to Build With a Brittle Compass

Brittleness is not a reason to abandon these tools. It is a reason to use them with discipline.

The Bottom Line

The "brittle compass" is a useful warning for an industry that has spent years celebrating capability. Language models are extraordinarily capable. They can read thousands of articles, summarize them fluently, and surface patterns no human team could find at that scale. But capability without stability is a shaky foundation for decisions.

The Slovak migration media case shows where the fault lines run: under-resourced languages, politically charged topics, and prompts that quietly shape answers. As AI becomes the default lens through which organizations read the world, the winners will not be those with the most impressive model. They will be the ones who tested their compass, and knew exactly how much to trust it.

TLDR: Large language models are highly sensitive to how prompts are worded, a problem captured by the "brittle compass" concept and stress-tested on Slovak media coverage of migration. Because Slovak is a lower-resource language and migration is a politically loaded topic, small wording changes can produce significantly different analyses, while still sounding confident. The lesson for the future of AI is that stability matters as much as accuracy. Businesses should paraphrase-test prompts, version and lock them, measure output agreement, test each language separately, keep humans in the loop on sensitive topics, and demand stability evidence from vendors.