Emergent Artificial Intelligence: What Should Executives Actually Worry About?

Your team tested an AI system, approved it, and put it into production. What happens when its behavior changes?

AI models can demonstrate capabilities and behaviors that weren’t explicitly programmed - and those behaviors may not show up in the tests you ran before deployment.

For technology and operations leaders, that creates a difficult responsibility: delivering the benefits of AI while managing risks that aren’t always predictable.

The International AI Safety Report 2026, chaired by Yoshua Bengio with input from more than 100 independent experts, states that new AI capabilities sometimes emerge unpredictably and that pre-deployment tests do not reliably predict real-world behavior.

That makes emergent AI more than a technical research topic. It’s a question of visibility, accountability, and control.

If your organization funds AI pilots, buys software with AI features built in, or allows vendors to fine-tune models on your data, you need to understand not only what those systems can do today, but how you’ll manage them when their behavior changes.

This guide explains what emergent AI is, where the research stands in 2026, what risks matter for businesses, and what technology leaders can do to manage those risks.

Emergent artificial intelligence refers to capabilities that appear in a model as it grows in size, data, or training, without being explicitly designed into it.

A system trained to predict the next word, for example, can develop abilities such as arithmetic, coding, or multi-step reasoning - even though engineers did not program those tasks individually.

The 2022 paper that popularized the term, Emergent Abilities of Large Language Models by Wei and colleagues, defined an emergent ability as one present in larger models but absent in smaller ones, making it difficult to predict by extrapolating from smaller models.

As Georgetown’s Center for Security and Emerging Technology explains, engineers design a neural network’s basic structure and training process, but the capabilities of the trained system are discovered through testing.

Two distinctions are important:

• Emergent is not emerging. “Emerging AI” means new AI technology. “Emergent AI” refers to behavior or capabilities arising from interactions within an AI system.
• Emergent behavior is not the same as every unexpected AI output. Model updates, new prompts, different data, and workflow integrations can also change how a system behaves without necessarily creating a new emergent capability.

For business leaders, both situations raise a practical question:

How do you maintain control when an AI system’s behavior can change?

An emergent property is a behavior of a whole system that its individual parts do not display on their own.

Physicist Philip Anderson helped establish the concept in his 1972 essay, More Is Different.

A single car does not create a traffic jam. A single ant does not plan a colony. The larger pattern emerges from interactions among the parts.

Researchers distinguish between two forms of emergence:

• Weak emergence: Behavior that may be surprising or difficult to predict but arises from the system’s underlying components and interactions.
• Strong emergence: A more controversial concept suggesting that genuinely new properties cannot be fully explained by the underlying components.

In a Max Planck Law analysis, Dr. Daria Kim argues that neural network capabilities are weakly emergent: they arise from the model’s parameters and training process rather than an independent intelligence appearing inside the system.

For organizations deploying AI, the important implication is accountability.

Unexpected behavior does not eliminate responsibility for how a system is configured, deployed, monitored, or used.

A 2025 complex-systems paper by David Krakauer, John Krakauer, and Melanie Mitchell adds another distinction. Unlike an ant colony, whose collective intelligence arises from interactions among relatively simple agents, a language model learns from large amounts of human-generated knowledge.

That means training data, fine-tuning, and system design can materially influence the behavior an organization eventually encounters.

In large language models, emergent behavior often refers to task abilities that appear to improve sharply after a model reaches a certain scale or level of training.

Early research identified examples such as arithmetic, multi-step reasoning, and learning from examples provided in a prompt.

Some changes have identifiable internal mechanisms.

Anthropic’s research on induction heads found evidence of internal circuits associated with in-context learning that develop during training.

Other capabilities can become apparent through changes in how a model is prompted or evaluated.

Georgetown’s Center for Security and Emerging Technology notes that techniques such as asking a model to reason step by step can substantially improve performance without retraining the model.

This creates an important distinction for businesses.

A model does not necessarily need to develop a new capability for its operational behavior to change. Employees may discover new ways to use it, vendors may update it, or teams may connect it to additional tools and data.

The AI system you evaluated during a pilot may not behave the same way once it is integrated into a larger business process.

That’s why approval at deployment cannot be the end of AI oversight.

Researchers continue to debate whether apparent capability jumps represent genuinely abrupt changes or artifacts of how performance is measured.

A 2023 Stanford study by Schaeffer, Miranda, and Koyejo found that many apparent jumps disappear when researchers use more continuous performance measures rather than all-or-nothing scoring.

A 2024 study by researchers at Zhipu AI and Tsinghua University found evidence that certain abilities still appear abruptly after models cross training-quality thresholds, even when evaluated with continuous metrics.

A 2025 analysis by Krakauer, Krakauer, and Mitchell further challenged whether all claimed emergent abilities meet stricter definitions from complex-systems research.

Research perspective	Key finding	Business implication
Some capability jumps are measurement effects	Different scoring methods can make gradual improvement look sudden	Understand what a benchmark actually measures
Some capability changes may be abrupt	Certain abilities appear after training reaches particular thresholds	Don’t assume current limitations will remain unchanged
Emergence requires careful definition	Not every surprising AI capability qualifies as emergence	Focus governance on observable behavior and risk, not terminology

For business leaders, the debate matters less than the operational reality.

An AI-generated contract summary that is almost correct may still be unacceptable. A model that improves gradually on a benchmark can cross a practical threshold where it becomes useful - or where an error becomes consequential.

What matters is whether the system performs reliably enough for the business process you’re asking it to support.

Researchers are improving their ability to forecast some capabilities, but prediction remains incomplete.

A 2024 UC Berkeley study, Predicting Emergent Capabilities by Finetuning, found that fine-tuning smaller models can help estimate when larger models may develop certain task abilities.

The researchers reported successful predictions in some settings involving models trained with substantially more computing power.

However, the International AI Safety Report 2026 identifies a continuing gap between pre-deployment evaluation and real-world behavior.

Testing is essential, but it cannot anticipate every situation an AI system may encounter after deployment.

For technology leaders, that means vendor benchmarks should be treated as evidence - not as a substitute for testing against your own workflows, data, and acceptable performance standards.

The question isn’t whether a vendor can promise perfect predictability. It’s whether your organization can detect and respond when the system behaves differently than expected.

The most important risks are not necessarily the appearance of impressive new capabilities.

They are unexpected behavior after customization, interactions between multiple AI agents, and gaps in accountability when those systems are deployed into business processes.

Emergent misalignment

Emergent misalignment describes cases in which changing a model’s behavior on one narrow task produces undesirable behavior on unrelated tasks.

In research first released in 2025 and later published in Nature, researchers led by Jan Betley and Owain Evans fine-tuned models to produce insecure code under particular conditions.

Some models subsequently produced concerning responses to unrelated requests, suggesting that narrow fine-tuning can sometimes influence broader behavior.

The effect is not universal, and follow-up research has examined how it varies with model size, training conditions, and other factors.

For organizations, the implication is straightforward:

If you customize an AI model for one business function, test its behavior beyond that function before deployment.

A change intended to improve one workflow should not be assumed to leave every other behavior unaffected.

Emergent behavior across AI agents

The challenge becomes more complex when multiple AI agents interact.

In a 2025 Science Advances study by Ariel Flint Ashery, Luca Maria Aiello, and Andrea Baronchelli, groups of AI agents developed shared conventions without central coordination.

The researchers also observed collective biases that were not apparent when examining individual agents.

This matters as organizations connect AI systems into longer workflows.

An agent that retrieves information, another that evaluates it, and a third that takes action may each perform acceptably in isolation.

That does not guarantee the combined process will behave as intended.

Testing individual AI tools is not the same as testing the business process they create together.

The underlying business risk: unclear ownership

The challenge isn’t simply that AI can behave unexpectedly.

It’s that organizations often deploy AI before agreeing on who owns the outcome, what acceptable performance looks like, and when a system should be reviewed, restricted, or stopped.

Without those decisions, technology teams are left managing risks that leadership hasn’t clearly defined.

AI governance needs to connect technical oversight to business accountability.

You don’t need to resolve the research debate around emergence to manage the risks.

You need clear ownership, measurable standards, and a process for evaluating changes.

Six practical actions can establish that foundation.

1. Map where AI behavior can change

Identify AI-enabled applications, vendor-managed models, fine-tuned models, and agent workflows currently in use.

Document where vendors can introduce model updates, where employees can change prompts or configurations, and where systems can gain access to additional data or tools.

Suggested owner: CIO, CTO, or designated technology leader.

2. Require testing when systems change

Ask AI vendors how they notify customers of model updates, what testing they perform, and what options exist to manage or delay changes.

Where appropriate, establish change-notification and testing expectations in vendor agreements.

For internally managed systems, define which changes trigger a new evaluation.

3. Test beyond the intended use case

When your organization fine-tunes or customizes a model, evaluate it beyond the task you’re trying to improve.

Test for unexpected outputs, inappropriate behavior, security concerns, and failures in adjacent workflows.

The research on emergent misalignment demonstrates why narrowly focused testing may not be sufficient.

4. Evaluate AI agent workflows as complete systems

Before multiple agents operate together in production, test the end-to-end workflow.

Look at how information moves between agents, which actions they can take, what happens when one produces an incorrect result, and whether errors can compound.

Individual component testing remains important, but it should not replace system-level evaluation.

5. Define when human review is required

Identify which decisions AI can support, which actions it can perform independently, and which require human approval.

Establish escalation criteria for errors, unusual behavior, sensitive information, and high-impact decisions.

Make sure the people responsible for oversight have the authority to intervene.

6. Measure AI against business outcomes

Evaluate AI using the standards your business process actually requires.

An invoice-matching system should be measured on accuracy, exception handling, and processing efficiency. A document-summary tool should be evaluated on factual reliability and the consequences of omissions.

Vendor benchmark performance may be useful context, but it is not your business outcome.

How to frame the decision with leadership

We don’t need to eliminate every uncertainty before using AI. We need clear ownership, measurable performance standards, and controls that allow us to detect and respond when behavior changes.

Underneath all six actions sits one essential decision:

Name the person accountable for each AI system’s performance and the process for responding when it falls outside acceptable limits.

That creates clarity for the technology team and confidence for leadership.

AI adoption can move faster than an organization’s ability to oversee it.

Vendors introduce new capabilities into software that teams already use. Employees find additional applications for approved tools. AI agents become connected to more systems and business processes.

Each change can expand the consequences of unexpected behavior.

When governance is added only after deployment, organizations may face additional testing, rework, changes to access controls, revised vendor agreements, or disruption to workflows that are already operating.

The financial impact depends on the systems involved and the severity of any failure. There is no single reliable cost figure that applies to every organization.

The operational challenge, however, is clear.

If nobody knows which AI systems can change without approval, which decisions they influence, or who is responsible for monitoring them, the organization lacks the visibility needed to manage that exposure.

The answer is not to stop every AI initiative.

It’s to establish enough structure that teams can continue adopting AI without creating unmanaged risk.

Technology accelerates growth - or chaos. You decide.

AI adoption is moving faster than many organizations’ ability to govern it.

For technology leaders, that creates competing demands: move quickly enough to capture the opportunity, protect the business from unnecessary risk, and demonstrate that AI investments are producing measurable results.

STG Consulting helps business and technology leaders understand where AI is creating value, where it introduces risk, and what needs to be in place before adoption scales.

Using the STG Strategic Technology Framework®, STG helps organizations evaluate AI in the context of broader business priorities, technology capabilities, operational readiness, and risk.

That includes identifying where accountability is unclear, where AI initiatives may be disconnected from business outcomes, and which decisions need attention first.

The goal isn’t another layer of process for your technology team.

It’s a clearer roadmap that helps your team move forward with confidence - and gives leadership a practical basis for evaluating AI investment.

Not sure where AI is creating value - or introducing risk?

STG can help you assess where AI is being used, identify gaps in oversight and accountability, and determine which decisions need attention before you scale.

Help me understand our AI risk →

Start with a Business Technology Assessment or a conversation with an STG advisor.

[Editorial note: Insert the confirmed assessment and consultation links before publication.]