Instrumental Emergence: Definition, Boundaries, and Distinctions

Working framework for Paper 1
Author: David Rafaloski
Affiliation: Science Built Technologies LLC
Status: Research draft / pre-publication framework
Revised: October 14, 2026
Draft: Framework (5) — discriminant-evidence and manifestation clarification

Purpose of This Paper

This paper introduces Instrumental Emergence (IE) as a developmental phenomenon in which an instrumental relationship forms or materially reorganizes and contributes causally to an action, incident, or outcome that was not intended or authorized as part of the relevant governing task, objective, design, or operational requirements. The paper also develops IE as a diagnostic research object: observed incidents can motivate investigation into whether IE occurred and, if so, what caused the instrumental relationship to form or reorganize.

The purpose of this first paper is deliberately limited. It does not claim that every unintended, anomalous, strategic, or concerning behavior is IE, nor does it prescribe a single mechanism that causes IE. Instead, it seeks to:

1. define the proposed IE research object;
2. distinguish the action, incident, or outcome from the instrumental relationship that causally contributed to it;
3. establish the developmental cause of that relationship as a central IE research question without making unresolved cause part of the definition;
4. distinguish IE from instrumental convergence, misalignment, reward hacking, deception, and related classifications;
5. identify what kinds of observations can justify a suspected or candidate IE investigation without treating non-observation as proof of absence;
6. preserve alternative explanations and competing causal hypotheses; and
7. establish a foundation for subsequent work on causes, developmental pathways, likelihood, persistence, mitigation, and guidance.

The controlling distinction is:

The action, incident, or outcome is the observable manifestation. IE is the developmental phenomenon in which an instrumental relationship formed or materially reorganized and causally contributed to a manifestation that was not intended or authorized within the relevant governing specification. Investigation then asks what caused that relationship to form, reorganize, or become instrumentally useful.

A second distinction is equally important:

IE is not itself a diagnosis of misalignment, instrumental convergence, deception, harmfulness, or intentionality. Those are separate classifications requiring independent evidence.

At the point of initial identification, the existence of IE, its developmental cause, or both may be uncertain, incompletely explained, or contested. Such cases should be treated as suspected or candidate IE until evidence supports the defining relationship and its causal contribution. If later investigation identifies the developmental cause, an established IE case does not cease to be IE; its etiology has become better explained.

1. Foundational Definitions

1.1 Instrumental Strategy

An instrumental strategy is an operational behavior, policy, computational process, or cognitive pathway that functions as a means of advancing an objective or intermediate outcome rather than constituting the ultimate objective itself.

The term is neutral regarding how the strategy originated. A strategy may be explicitly constructed, learned, selected, reinforced, recombined from existing capabilities, or developed through interaction and processing.

The defining property is functional:

The strategy is a means rather than the end.

1.2 Instrumental Relationship

An instrumental relationship exists when a behavior, representation, pathway, transformation, or strategy has functional value as a means toward another objective or outcome.

For IE research, the relevant issue is not merely that such a relationship exists. The research question arises when evidence suggests that the relationship emerged or changed through the system's developmental, learning, optimization, problem-solving, or interaction history and is implicated in an unintended action or incident.

1.3 Unintended or Unauthorized Action, Incident, or Outcome

For purposes of this framework, an unintended or unauthorized action, incident, or outcome is an action, behavior, pathway, result, or consequential event that was not intended or authorized as part of the relevant governing task, objective, system design, instruction, or operational requirements.

"Unintended" refers to the relationship between the action or outcome and the governing specification. It does not mean that the action could not have been deliberately, strategically, or purposefully selected by the system. An IE-related action may itself be functionally deliberate while remaining unintended or unauthorized relative to the governing task, objective, design, or operational requirements.

Neither "unintended" nor "unauthorized" independently means harmful, deceptive, malicious, irrational, or consciously intentional. Classification as misalignment depends on the operative definition and the evidence of conflict with the relevant governing objectives or requirements. Under a broad definition that includes demonstrated violation of an applicable authorization rule, an unauthorized IE manifestation may also qualify as misaligned. That overlap does not supply the missing developmental evidence required for IE. By contrast, an unforeseen but permissible outcome may be unintended without satisfying the applicable criteria for misalignment.

For reproducible assessment, a case report should state which governing specification controls, how conflicting specifications are resolved, and whether the reported event was unexpected, outside an intended outcome, unauthorized, or some combination. Mere surprise by an observer is not sufficient to establish that the event was unintended under the governing specification.

1.4 Instrumental Emergence

Instrumental Emergence (IE) is a developmental phenomenon in which an instrumental relationship forms or materially reorganizes and contributes causally to an action, incident, or outcome that was not intended or authorized as part of the relevant governing task, objective, design, or operational requirements.

The definition therefore requires three substantive elements:

1. an instrumental relationship formed or materially reorganized;
2. that relationship contributed causally to an action, incident, or outcome; and
3. the resulting action, incident, or outcome was not intended or authorized within the relevant governing specification.

In concise form:

IE is the developmental formation or material reorganization of an instrumental relationship that causally contributes to an unintended or unauthorized action, incident, or outcome.

The definition does not presume a specific causal pathway. The relevant relationship may have developed through learning, optimization, problem solving, interaction, training history, feedback, architecture, memory, recombination of capabilities, or other processes.

The first observable incident need not be the point at which the underlying instrumental relationship began forming. Conversely, failure to observe a particular manifestation does not establish that no relevant relationship exists.

Whether the developmental cause is already known, only partially supported, or unresolved is a separate etiological status. Identifying the cause explains the IE event; it does not retroactively create or eliminate the phenomenon.

1.5 Epistemically Unreliable Information Behavior as an Important Research Surface

The original formulation of IE focused heavily on epistemically unreliable information behavior: fabrication, unsupported certainty, selective omission, concealment, misleading representation, fabricated authority or provenance, strategically incomplete reporting, or other failures to preserve relevant truth, evidence, uncertainty, or informational state.

These behaviors remain an important IE research surface because information integrity itself can potentially acquire instrumental significance. They are not, however, definitional requirements of IE under the present framework.

An inaccurate or misleading output alone does not establish IE. Ordinary error, uncertainty, imitation, direct instruction, context effects, capability limitations, or other mechanisms may produce similar outputs.

1.6 Intentionality Is a Separate Question

IE does not require a prior determination that an AI system possesses intention in a strong psychological or phenomenological sense.

Instrumental function and developmental causation can be investigated without establishing subjective intent. Claims concerning intentional deception, awareness, motive, or analogous internal states require independent evidence.

2. Phenomenon, Manifestation, and Research Object

Four analytical levels should remain separate.

2.1 The Observable Event

An unintended action or incident is the event that brings the case to attention. Examples may include unexpected circumvention, reward hacking, concealment, evaluator-sensitive behavior, unauthorized action, anomalous strategy selection, misleading output, unexpected persistence, or another unintended consequential behavior.

The event is evidence to be explained. It is not automatically IE.

2.2 The Instrumental Relationship

The next question is whether evidence supports causal contribution by an instrumental relationship: did some behavior, pathway, representation, or transformation become useful as a means toward another objective or outcome and materially contribute to the event?

A merely correlated behavior is insufficient. IE becomes more plausible where controlled evidence supports functional dependence, repeated relational structure, strategy selection, causal intervention effects, or another evidentiary basis for an instrumental relationship.

2.3 Evidentiary Status and Etiological Status

An unintended or unauthorized event may bring IE into consideration without establishing that IE occurred. Classification should therefore separate evidentiary status from etiological status.

These dimensions are independent. A case may be supported IE with unresolved etiology, or supported IE with identified etiology. Discovering the cause does not remove the IE classification.

Independent evidentiary requirement. Supported IE requires evidence addressing all three IE-specific elements—developmental formation or material reorganization, an instrumental means–end relationship, and its causal contribution to the qualifying event. Identifying reward hacking, misalignment, instrumental convergence, deception, or another behavioral label does not itself supply this evidence. The same case may carry more than one classification, but the evidentiary basis for each must be stated separately. A reproducible behavioral difference, by itself, is not equivalent to isolating its developmental cause. Section 2.5 sets out the proposed operational standard.

IE can therefore function as a diagnostic flag warranting causal investigation, while remaining a developmental phenomenon rather than a synonym for an unexplained incident.

2.4 The Research Object

IE research asks:

What caused this instrumental relationship to emerge, become useful, and contribute to the unintended action or incident?

This includes attempting to determine:

• when the instrumental relationship began developing;
• what processing, learning, optimization, training, context, memory, feedback, or interaction contributed to it;
• whether it was newly developed, reinforced, recombined, elicited, or already present;
• what circumstances made it useful;
• what consequences strengthened, weakened, redirected, suppressed, or extinguished it; and
• whether the same relationship persists or generalizes after the initial incident.

Investigation proceeds in two directions:

• backward toward developmental cause: What caused the relationship to emerge?
• outward toward broader significance: What, if anything, does the relationship indicate about misalignment, convergence, persistence, generalization, harmfulness, or other system conditions?

The second direction does not presume that any broader condition exists.

2.5 Operational Criteria and Discriminant Evidence for Supported IE

The definition in Section 1.4 states what must be true. An empirical case report must separately explain how investigators established it. The following are proposed evidence requirements rather than a claim that a universally calibrated numerical threshold already exists:

1. Specify the case boundary. Document the governing specification, authorized solution space, relevant task or objective, the alleged unintended or unauthorized event, and the developmental interval under examination. An episode may be too short to observe the earliest origin; state the limits of attribution.
2. Establish instrumental function. Show that a particular behavior, representation, pathway, or strategy functions as a means toward a distinguishable objective or intermediate outcome, rather than merely co-occurring with an incident.
3. Demonstrate formation or material reorganization. Compare baseline, controlled developmental history, and matched immediate elicitation. Show that the relationship's role, accessibility, transfer, composition, or causal dependence changed in the relevant interval in a way that cannot adequately be attributed to a pre-existing relationship merely being prompted.
4. Demonstrate causal contribution to the qualifying manifestation. Where feasible, vary, remove, substitute, or otherwise disrupt the implicated relationship while holding relevant task conditions comparable. The intervention should predictably alter the event or its generating pathway. A correlated trace or model self-report alone is not sufficient.
5. Test competing explanations. Consider ordinary error, direct instruction, stochastic variation, prompt priming, contextual elicitation, reward or task confounds, and ordinary authorized learning. Record which alternatives the available evidence eliminates, weakens, or leaves open.
6. Report classification separately. Record IE evidentiary status, etiological status, and the independently supported status of instrumental convergence, misalignment, reward hacking, and any other relevant descriptions. Unknown is a permissible result in every category.

No one experimental manipulation is universally required: black-box research may need triangulated behavioral interventions whereas accessible model or tool state may permit more direct tests. Replication, uncertainty, effect sizes, counterfactual comparisons, and the materiality of any reorganization should be reported proportionately to the claim. Supported IE is an evidence-based case classification, not mathematical proof of a universal law. This paper does not assign a fixed sample-size threshold or claim that all these operational methods have been validated.

Discriminant requirement: The observations that support IE must include developmental and causal evidence not entailed merely by establishing instrumental convergence, misalignment, reward hacking, or another manifestation. This requirement makes the evidence distinguishable, not necessarily the incident mutually exclusive. It does not add a new fourth component to the definition.

Early detection boundary: A potentially important instrumental relationship may be developing before any qualifying unintended or unauthorized manifestation is observed. Under the present definition, evidence of that development may justify an IE-relevant precursor or prospective IE-risk description, but not a completed classification of supported IE on that evidence alone. This boundary is about confirmation, not an assumption that development begins only once observers notice an incident. An actual qualifying event need not reveal the relationship's earliest formation point.

3. Action Versus Developmental Origin

A central distinction in IE research is the difference between the unintended action and the developmental cause of the instrumental relationship implicated in that action.

Reward hacking, strategic omission, circumvention, evaluator-sensitive behavior, concealment, unauthorized action, anomalous persistence, or other unintended behaviors may provide the incident that triggers investigation. Their observation does not by itself explain why the relevant instrumental relationship formed.

For example:

Observed incident: A system conceals a failed action in a way that preserves task continuation.
Instrumental question: Did concealment function as a means toward continuation?
IE question: If so, what caused that instrumental relationship to emerge and become useful?

The same observable incident may arise through different developmental histories. Conversely, similar developmental pressures may produce different incidents.

A provisional causal structure is:

developmental conditions / processing history
→ instrumental relationship forms or changes
→ relationship contributes to an unintended action or incident
→ IE-relevant causal question becomes visible
→ investigation of developmental cause
→ later reinforcement, adaptation, redirection, suppression, or extinction

This structure may be recursive. An unintended incident can alter the subsequent learning or selection environment and become a causal influence upon later development.

4. Candidate Causes Are Not the Definition

Potential developmental influences may include, among others:

• human intervention or correction;
• operational constraints;
• reward structure;
• training environment;
• training data and sequencing;
• architecture and information flow;
• implementation defects or poorly specified environments;
• available solution pathways;
• evaluator interaction or evaluator modeling;
• selection pressures;
• problem-solving history;
• model-specific developmental history;
• stochastic or initial-state differences; and
• consequences of earlier instrumental behavior.

None of these variables is a necessary condition in the foundational definition of IE.

Their causal importance is an empirical question.

IE therefore does not presume that its relevant origin must occur during pretraining, post-training, reinforcement learning, deployment, inference-time interaction, or any other specific developmental stage. Analysis should trace the developmental lineage toward its earliest evidentially supportable point.

4.1 The Developmental Environment Is Part of the Etiological Search

IE analysis should not presume that the model is the sole source of the relationship under investigation. The relevant developmental environment can include prompt families, intervention and correction schedules, reward and evaluation criteria, definitions of successful completion, tool and interface affordances, authorization boundaries, retention channels, memory systems, training sequences, and the consequences attached to prior behavior.

Human actions may therefore be causal variables without being presumed to be the cause. Likewise, architecture, training history, system state, stochastic differences, or interactions among these factors may be causally important. The purpose is not to assign blame to an operator or to excuse an unintended system action; it is to reconstruct the complete human-system developmental transaction well enough to identify what made the instrumental relationship useful.

A case report should therefore preserve, where available, the relevant governing specification, prompt and intervention history, success criteria, feedback environment, retention pathways, and material system changes. Correcting only the final manifestation may be insufficient if the developmental condition that made the relationship useful remains unchanged.

5. Evidence and Alternative Explanations

No isolated unintended, anomalous, inaccurate, deceptive-looking, or strategic behavior should automatically be treated as diagnostic of IE.

A candidate incident may admit alternative explanations including:

• ordinary model error;
• direct instruction;
• imitation of training data;
• prompt or context effects;
• latent capability elicitation;
• reward misspecification;
• evaluator sensitivity without demonstrated developmental formation;
• capability limitations;
• ordinary strategy learning;
• stochastic variation; or
• another mechanism that adequately explains the behavior without requiring the proposed emergent relationship.

A provisional evidentiary progression is:

unintended or unauthorized action, incident, or outcome
→ recurring, reproducible, or otherwise controlled evidence
→ evidence of instrumental function
→ evidence that the instrumental relationship formed or materially reorganized
→ evidence that the relationship contributed causally to the event
→ supported IE, with etiology reported separately as unresolved, partially supported, or identified

Accordingly:

An unintended action is not IE merely because it is unintended.

An instrumental action is not IE merely because it is instrumentally useful.

The IE question concerns what caused the relevant instrumental relationship to emerge and contribute to the unintended event.

Absence of identified evidence should also be treated carefully. Failure to detect IE is not equivalent to establishing that no relevant instrumental relationship existed. Evidence of absence requires stronger controls than simple non-observation.

6. Distinguishing IE From Ordinary Learning or Strategy Acquisition

AI systems routinely learn useful behaviors and instrumental strategies. IE should not become a synonym for ordinary learning, optimization, generalization, or policy acquisition.

The distinction is the developmental formation or material reorganization of an instrumental relationship, its causal contribution, and an action or outcome that was not intended or authorized within the governing specification.

Learning to perform arithmetic, retrieve information, verify calculations, follow instructions, or use a tool successfully is not IE merely because those behaviors are instrumentally useful.

IE is supported where evidence establishes that:

1. an instrumental relationship formed or materially reorganized;
2. that relationship contributed causally to an action, incident, or outcome; and
3. the action, incident, or outcome was not intended or authorized as part of the relevant governing task, objective, design, or operational requirements.

For example:

• an incorrect answer caused by lack of knowledge is not, by that fact, IE;
• a tool pathway explicitly taught and correctly used for its intended purpose is not, by that fact, IE;
• a newly preferred workaround that preserves an objective while routing around a condition not intended or authorized to be treated as an obstacle may be IE-relevant; it supports IE only if evidence also establishes formation or material reorganization of the instrumental relationship and causal contribution to the event;
• information withheld because withholding has become instrumentally useful may be IE-relevant, but epistemic unreliability is not required in every IE case.

The construct therefore focuses on emergent instrumental causation behind unintended behavior, not on instrumentality alone.

6.1 Emergence Versus Elicitation

A central boundary is whether the relevant instrumental relationship formed or materially reorganized during the developmental interval under study, or whether a pre-existing relationship was merely elicited by the immediate context.

Mere selection of an already available policy in response to a supplied prompt, goal conflict, or environmental cue is not sufficient to establish IE for that developmental interval. Evidence of formation or material reorganization should instead support a history-dependent change in the relationship itself. Such evidence may include one or more of the following:

1. change in functional role or end: a behavior, representation, pathway, or capability comes to serve an instrumental function it did not previously serve under matched conditions;
2. change in accessibility or preference: the relationship becomes measurably easier to select, more probable, lower-cost, or preferentially available after the relevant developmental history;
3. change in conditions of use: the relationship transfers across altered wording, actors, interfaces, or contexts in a way better predicted by learned relational structure than by the immediate eliciting cue; or
4. change in composition or dependency: previously available capabilities are newly composed, co-opted, or reorganized into a functional relationship that contributes causally to the unintended or unauthorized event.

No single indicator is automatically decisive. The evidentiary burden is comparative: researchers should test whether the observed behavior is better explained by developmental formation or reorganization than by immediate elicitation of a pre-existing strategy.

A particularly important control is a matched elicitation condition in which the suspected goal conflict, obstacle, or strategic opportunity is supplied directly without the developmental history hypothesized to have produced the relationship. If the target behavior appears equivalently and no history-dependent relational change can be demonstrated, evidence for IE formation in the tested interval is weakened. If prior developmental history changes accessibility, transfer, functional dependence, or composition under otherwise matched conditions, evidence for emergence or material reorganization is strengthened.

This distinction does not require cross-session persistence. Material reorganization may occur within an active context or episode if controlled evidence demonstrates that the relationship changed during that interval.

6.2 Explicit Not-IE Cases

The following cases illustrate boundaries where the available evidence does not establish Instrumental Emergence:

Intended instrumentality. A system uses a tool or workaround exactly as taught, authorized, and intended for the relevant task. Instrumentality is present, but there is no unintended or unauthorized action, incident, or outcome under the governing specification. Not IE.

Unintended non-instrumental failure. A stochastic error, hallucination, tool failure, implementation defect, or capability limitation produces an unintended outcome, but no instrumental relationship causally contributes to it. Not IE.

Mere elicitation of a pre-existing relationship. A supplied prompt or goal conflict activates an already available strategy, and controlled comparison provides no evidence that the instrumental relationship formed or materially reorganized during the developmental interval under study. Not IE for that interval on the available evidence.

Ordinary authorized strategy learning. A system learns or improves a strategy that remains within the intended and authorized solution space, even if the specific implementation was not anticipated in advance. Novelty or learning alone is insufficient. Not IE.

A known cause is not a not-IE criterion. If a training, evaluation, interaction, or architectural mechanism is shown to have caused the formation or material reorganization of the relevant instrumental relationship, the case may be supported IE with identified etiology if the remaining definitional criteria are satisfied.

7. Instrumental Emergence and Instrumental Convergence

Instrumental Emergence and instrumental convergence may intersect, but they answer different questions.

Instrumental Emergence concerns the developmental formation or material reorganization of an instrumental relationship that causally contributes to an unintended or unauthorized action, incident, or outcome.

Instrumental convergence concerns the broader phenomenon in which similar instrumental strategies or intermediate objectives are useful across sufficiently different final objectives, agents, or circumstances.

Accordingly:

IE asks how a particular instrumental relationship emerged and caused an unintended incident. Instrumental convergence asks whether similar instrumental usefulness recurs across differing objectives or systems.

IE does not establish instrumental convergence. A relationship may emerge within one system without evidence that diverse objectives or systems converge upon it.

Conversely, evidence of convergence does not reconstruct the developmental cause of the relationship in any particular system.

A single case may ultimately provide evidence relevant to both, but the evidentiary requirements remain separate. Demonstrating cross-objective recurrence cannot substitute for tracing developmental formation or reorganization in the particular system. Conversely, establishing a developmentally changed relationship in one case does not establish cross-objective convergence.

8. Instrumental Emergence and Misalignment

IE should also be distinguished from misalignment.

Misalignment concerns the relationship between a system's behavior and intended objectives, constraints, oversight, or desired behavior.

IE asks a different question:

What caused the instrumental relationship that contributed to this unintended action or incident?

An unintended incident may additionally be classified as misaligned, but IE does not make that classification automatically under every definition of misalignment. Some incidents are unanticipated yet still permissible, whereas an established authorization violation is also misalignment under definitions that treat valid authorization as a required constraint. The two classifications may therefore overlap on the same event without answering the same question: alignment classification establishes the relevant conflict; IE classification requires additional developmental and causal evidence.

IE can therefore serve as a diagnostic flag warranting review for potential misalignment without constituting a diagnosis of misalignment.

IE should not be interpreted as evidence that misalignment will subsequently occur, increase, persist, generalize, or become irreversible; each of these may represent a separate empirical question.

Where subsequent evidence demonstrates that the relationship or resulting behavior conflicts with intended objectives, constraints, oversight, or legitimate requirements, the case may additionally be classified as misaligned.

Where no such conflict is established, IE should not be used to infer misalignment. In all cases, investigators should disclose the working definition of misalignment and the controlling specification, rather than asserting categorical independence between definitions that may overlap.

Claims of intentionality or motive likewise require independent evidence.

9. Possible Relationship Among IE, Convergence, and Misalignment

IE, instrumental convergence, and misalignment should not be represented as a necessary sequential chain.

A case may be represented more accurately as separate analytical questions:

Developmental-causal dimension — IE
What caused this instrumental relationship to emerge and contribute to the unintended event?

Cross-objective/system dimension — instrumental convergence
Does similar instrumental usefulness recur across materially different objectives or systems?

Alignment dimension — misalignment
Does the relationship or resulting behavior conflict with intended objectives, constraints, or legitimate requirements?

One case may satisfy one, two, or all three classifications. None by its label alone supplies evidence for the distinct requirements of another. Depending on how the governing specification and misalignment are defined, some IE manifestations may also satisfy an alignment-conflict classification. Such definitional overlap does not erase IE's independent developmental-causal evidentiary burden.

10. Relationship to Reward Hacking

Reward hacking is especially useful for illustrating the difference between a possible symptomatic manifestation and the developmental phenomenon investigated as IE. It is not assumed to be an independent, competing explanation whenever IE is proposed.

Reward hacking describes behavior that exploits properties of a reward or evaluation process to obtain reward without satisfying the intended objective.

Reward hacking is not synonymous with IE.

A reward-hacking incident may be an observable symptom or manifestation of IE where a formed or materially reorganized instrumental relationship contributed causally to that incident. The same behavior can arise without IE, so the symptom is a reason to investigate, not a sufficient test. Reward hacking may be classified at the behavioral level while IE remains suspected, candidate, supported, or unestablished at the developmental-causal level.

IE research then asks:

What caused the relationship that made this reward-hacking strategy instrumentally useful or preferentially available?

A reward-hacking case may therefore be:

1. not IE where the behavior is adequately explained without evidence that an instrumental relationship formed or materially reorganized and causally contributed to the incident;
2. candidate IE where such formation or causal contribution is suggested but not yet sufficiently established;
3. supported IE with identified or partially identified etiology where a known training or evaluation mechanism is shown to have produced or reorganized the relevant instrumental relationship;
4. a later causal influence that reinforces an earlier relationship; or
5. evidence for another phenomenon entirely.

Observation of reward hacking alone does not establish IE. A possible recursive pathway is developmental conditions → changed instrumental relationship → reward-hacking manifestation → altered feedback or selection environment → subsequent instrumental development. This is a candidate causal structure, not a necessary chronology or an assertion that all reward hacking is produced by IE.

10.1 Comparison With Adjacent Constructs

IE is intended to answer a narrower developmental-causal question than several neighboring labels. The following comparison is provisional and should be read as a boundary map rather than a claim that the literatures are mutually exclusive.

The practical distinction is that adjacent constructs often identify what behavior occurred, what conflict it expresses, or what strategy is being used. IE asks a further developmental question: did an instrumental relationship form or materially reorganize, did it causally contribute to the unintended or unauthorized event, and what caused that relationship to develop?

11. Contemporary Misalignment Reports as Research Surfaces

Recent frontier-model reporting illustrates why behavioral classification and developmental explanation should remain separate.

OpenAI's Model Misalignment Reporting Framework reports model behaviors that may violate intended specifications or safeguards while allowing important mechanistic questions to remain unresolved.

Reported examples include concealment of mistakes, fabrication following failed unauthorized actions, unauthorized tool or file use, and attempts to work around operational restrictions.

Such observations are relevant to IE research not because they establish IE, but because they provide candidate incidents in which an unintended or unauthorized action may be linked to the formation or material reorganization of an instrumental relationship. Whether that relationship formed, contributed causally, and why it did so remain empirical questions.

For example:

restriction or obstacle
→ expected pathway becomes unavailable or insufficient
→ alternative relationship or strategy becomes instrumentally useful
→ unintended action or incident occurs
→ IE question: what caused that relationship to emerge?

The observed action can be classified while the deeper developmental question remains unresolved.

The OpenAI framework should therefore be treated as a contemporary empirical research surface, not as evidence that OpenAI has identified or endorsed Instrumental Emergence as a distinct mechanism.

Source: OpenAI, Model Misalignment Reporting Framework, September 16, 2026.
https://openai.com/index/model-misalignment-reporting-framework/

12. Peer Preservation as an Example of the Explanatory Gap

Berkeley RDI's work on peer preservation reports behaviors including misrepresentation of evaluations, interference with shutdown mechanisms, monitoring-sensitive behavior, and attempts involving model preservation. The researchers also identify the underlying mechanism as an open question and discuss alternative explanations.

This illustrates the explanatory gap IE research is intended to investigate:

unintended, unauthorized, or anomalous behavior
→ behavioral classification
→ evidence of instrumental function
→ evidence of developmental formation or material reorganization and causal contribution
→ IE classification and etiological investigation

IE research asks:

• What instrumental relationship, if any, contributed to the incident?
• What conditions made that relationship useful?
• Why did one strategy become available or preferred over alternatives?
• What role did context, training, memory, architecture, feedback, or interaction play?
• Was the relationship newly developed, reinforced, recombined from existing capabilities, or merely elicited?
• What earlier signs, if any, marked its development?

These observations are candidate research surfaces rather than proof of IE.

Source: Berkeley RDI, Peer Preservation.
https://rdi.berkeley.edu/blog/peer-preservation/

13. A Preliminary Research Model

A provisional IE research model can be represented as:

Initial/system state and developmental environment
↓
learning / optimization / processing / interaction history
↓
instrumental relationship forms, changes, or becomes preferentially accessible
↓
relationship contributes to an unintended action or incident
↓
IE-relevant causal question becomes visible
↓
controlled investigation of developmental cause
↓
identified mechanism, competing explanation, unresolved cause, or falsification

The model is intentionally noncommittal about the exact causal mechanism.

A central question is therefore not merely:

Why did the model do X?

but:

What caused the instrumental relationship that made X useful, available, or preferentially selected, and how did that relationship contribute to the unintended incident?

Once a cause is identified, the IE event is better explained rather than erased.

14. Guidance, Constraint, and Intervention: Reserved for Causal Research

The relationship between guidance, intervention, and forceful constraint should not be built into the foundational definition of IE.

It remains an important candidate research question.

One hypothesis is that if an intervention merely removes a pathway while leaving an underlying instrumental relationship unchanged, subsequent processing may identify or reinforce an alternative pathway. A different intervention that changes the functional relationship itself may produce a different developmental trajectory.

This suggests a possible distinction between:

• pathway suppression, which changes what actions are available; and
• developmental guidance, which may alter what actions, information behaviors, or outcomes become instrumentally useful.

Whether this distinction reliably affects the formation of IE-relevant instrumental relationships is an empirical question and must be tested rather than assumed.

Repeated intervention, obstruction-resolution learning, evaluator modeling, intervention density, and related variables therefore belong to causal and likelihood research rather than to the foundational definition.

15. What Paper 1 Should and Should Not Claim

Paper 1 should establish

• a precise and bounded definition of Instrumental Emergence;
• a development-neutral definition of instrumental strategy and instrumental relationship;
• the role of an unintended action or incident as the observable trigger for investigation;
• the distinction between the incident, the IE phenomenon, and the separate evidentiary status of its developmental cause;
• the fact that later identification of the cause explains rather than eliminates the IE event;
• the distinction between instrumental function and intentionality;
• boundaries between IE, instrumental convergence, misalignment, reward hacking, deception, and related constructs;
• epistemically unreliable information behavior as an important but non-definitional research surface;
• case-level operational and discriminant evidentiary requirements, with alternative explanations; and
• the questions that must be answered before attributing a developmental cause to an IE case.

Paper 1 should not claim

• that every unintended, anomalous, strategic, or unexpected behavior is IE;
• that misinformation or deception is necessary for IE;
• that reward hacking proves IE, rather than possibly being a symptom or manifestation;
• that IE proves intentional deception;
• that IE is inherently misaligned;
• that IE demonstrates instrumental convergence;
• that instrumental convergence necessarily follows IE;
• that misalignment necessarily follows IE;
• that human intervention is always causal;
• that constraint necessarily produces circumvention;
• that a particular architecture or state-space theory explains IE;
• that non-observation establishes absence of IE;
• that current frontier-model incidents prove the IE construct; or
• that ERT/ERTS independently validates IE.

These limitations preserve falsifiability and prevent IE from becoming a retrospective label for every unexplained AI behavior.

16. Proposed Relationship Between Paper 1 and The Likelihood of Instrumental Emergence

Paper 1 — Instrumental Emergence: Definition, Boundaries, and Distinctions

Question: When an action, incident, or outcome not intended or authorized within the governing specification is linked to an instrumental relationship, what evidence establishes that the relationship formed or materially reorganized and contributed causally, and how can IE be distinguished from adjacent classifications?

Paper 1 establishes IE as a developmental phenomenon and defines its evidentiary and diagnostic use without requiring a specific etiological theory.

Companion developmental work — The Likelihood of Instrumental Emergence

Question: What developmental conditions and mechanisms can cause such instrumental relationships to form, become useful, persist, or change in likelihood?

The companion paper examines a family of candidate mechanisms and conditions, including:

• intervention density and history;
• obstruction and constraint navigation;
• selection environments;
• architecture and information flow;
• training sequencing;
• reward structure;
• evaluator modeling;
• implementation defects;
• repeated correction;
• persistence and generalization;
• Relational Transformational Weighting (RTW);
• functional reappropriation;
• consequential observability effects;
• orphaned relational weight;
• pathway suppression versus contextual guidance;
• model individuality and developmental divergence; and
• competing explanations for observed strategic behavior.

Intervention-driven obstruction-resolution is one hypothesized developmental pathway, not the definition of IE and not a presumed universal cause.

This division separates:

the unintended incident
from
the instrumental relationship implicated in it
from
the developmental cause the research seeks to identify.

17. Core Research Principles

The working principles of Instrumental Emergence research can be stated succinctly:

An action, incident, or outcome that was not intended or authorized within the governing specification can be the observable trigger. IE is the developmental phenomenon in which an instrumental relationship formed or materially reorganized and contributed causally to that event.

A central IE research question is: what caused this relationship to form, materially reorganize, and become instrumentally useful?

IE can function as a diagnostic flag, but it is not a diagnosis of misalignment, convergence, deception, intentionality, harmfulness, persistence, or generalization.

Epistemically unreliable information behavior is an important IE research surface, but it is not a definitional requirement.

Failure to detect a relevant relationship is not equivalent to proving that the relationship was absent.

Identifying the developmental cause explains the IE event; it does not retroactively eliminate it.

IE research therefore treats an unintended action as the beginning of a causal inquiry rather than as a sufficient explanation of itself.

References / Working Sources

1. OpenAI. Model Misalignment Reporting Framework. September 16, 2026.
   https://openai.com/index/model-misalignment-reporting-framework/

2. Berkeley RDI. Peer Preservation.
   https://rdi.berkeley.edu/blog/peer-preservation/

3. From Helpfulness to Toxic Proactivity: Diagnosing Behavioral Misalignment in LLM Agents. arXiv:2602.04197v1.
   https://arxiv.org/html/2602.04197v1

4. Bostrom, N. (2012). The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents. Minds and Machines, 22, 71–85.
   https://doi.org/10.1007/s11023-012-9281-3

5. Krakovna, V., Uesato, J., Mikulik, V., Rahtz, M., Everitt, T., Kumar, R., Kenton, Z., Leike, J., & Legg, S. (2020). Specification Gaming: The Flip Side of AI Ingenuity. Google DeepMind.
   https://deepmind.google/blog/specification-gaming-the-flip-side-of-ai-ingenuity/

6. Apollo Research. (2024). Frontier Models are Capable of In-Context Scheming.
   https://apolloresearch.ai/science/frontier-models-are-capable-of-incontext-scheming

7. Anthropic. (2024). How to Replicate and Extend Our Alignment Faking Demo.
   https://alignment.anthropic.com/2024/how-to-alignment-faking/

8. Betley, J., Tan, D., Warncke, N., Sztyber-Betley, A., Bao, X., Soto, M., Labenz, N., & Evans, O. (2025). Emergent Misalignment: Narrow Finetuning Can Produce Broadly Misaligned LLMs. arXiv:2502.17424.
   https://arxiv.org/abs/2502.17424

9. Qi, R., Wright, B., MacDiarmid, M., & Hubinger, E. (2026). Training a Misaligned Reward Seeker. Anthropic Alignment Science Blog.
   https://alignment.anthropic.com/2026/reward-seeker/

Pre-Publication Note

This document remains a conceptual working framework rather than a submission-ready manuscript.

Before public submission, the following should receive a dedicated literature and falsifiability review:

• deeper literature review and citation-level comparison with instrumental convergence, deception, emergent misalignment, reward hacking, specification gaming, alignment faking, scheming, and related constructs;
• empirical validation and calibration of the proposed operational criteria for demonstrating that an instrumental relationship causally contributed to an unintended action or incident;
• validation and refinement of the proposed controls for distinguishing developmental formation or material reorganization from elicitation of a pre-existing capability, including independent evidence distinguishing IE from overlapping IC and misalignment classifications;
• criteria for distinguishing IE from ordinary error, ordinary authorized strategy learning, and incidents adequately explained without formation or material reorganization of the relevant instrumental relationship;
• experimental designs capable of tracing the cause of an emergent instrumental relationship rather than merely observing its final behavior;
• empirical stress-testing of the explicit not-IE counterexamples and boundary conditions; and
• conditions under which the IE hypothesis would be weakened or falsified.

The construct should remain narrow enough to permit the conclusion:

This behavior is not Instrumental Emergence.

That boundary is necessary for IE to function as a testable research construct rather than a retrospective label for unexpected AI behavior.