Ashyboy2/arm-inner-link-qwen2.5-14b

ARM Inner-Link Adapter (Qwen 2.5 14B-Instruct)

A small (~52M-parameter, ~200MB) residual MLP adapter trained to project a Qwen 2.5 14B-Instruct hidden state at the last residual layer toward the input embedding of the next token — a re-implementation of the inner-link training objective from RecursiveMAS (arXiv:2604.25917).

This checkpoint is released as part of a negative result: the trained adapter converges cleanly on its objective (mean cosine 0.74; control 0) but fails to transmit structured reasoning state across a session boundary (0/18 on the Hypothesis-Continuity benchmark — see the repo). The release is for reproducibility and to support any future state-reconstruction-objective experiments that want to use the same architecture as a starting point.

---

What it is

• Architecture: 2-layer residual MLP with LayerNorms, identical in shape to the RecursiveMAS inner-link adapter. See src/arm/inner_link.py in the ARM repository.
• Input: last-layer (post-norm) hidden state of Qwen 2.5 14B-Instruct, shape [d_model = 5120], float32.
• Output: a d_model-shaped vector intended to be close (cosine similarity) to the input-embedding of the next token.
• Parameters: 52,459,520 (≈52M).
• Training objective: L_in = 1 − cos(R_in(h), Emb(y_next)). Graph-free (no recursive backprop), as specified in the RecursiveMAS paper.
• Training data: simplescaling/s1K (1k high-quality reasoning traces; ~2M tokens; 8,078 chunks of 256 tokens).
• Training: 3,000 steps; final mean_cos 0.74, ctrl_cos ~0.005, loss 0.27.

What it is NOT

• Not a useful memory mechanism: the adapter scored 0/18 on the Hypothesis-Continuity hand-off benchmark, on par with random-vector and wrong-session controls. The full v1 result is in the repository under results/ARM_v1_update.md.
• Not the official RecursiveMAS adapter: this is an independent re-implementation. RecursiveMAS's training code was not publicly released; the objective specification was followed from the paper.

How to use (for reproducing the ARM negative result)

from arm.inner_link import load_inner_link
adapter = load_inner_link("path/to/inner_link_14b")
# adapter.forward(hidden_state) -> projected vector

For the full Phase 2B experimental pipeline, see scripts/08_phase2b_full.py --inner-link checkpoints/inner_link_14b in the ARM repository.

Intended use

• Reproducing the ARM v1 negative result.
• Starting point for a state-reconstruction-objective experiment (suggested in the paper §9 as the one experiment that could overturn the conclusion).
• NOT for production memory hand-off in any agent system.

Limitations

• Tested only with Qwen 2.5 14B-Instruct as the base model. Behavior with other base models (Llama, Mistral) is uncharacterized.
• The adapter optimizes a continuation objective (next-token alignment); it is not a state-reconstruction adapter. This is the explanation for the hand-off failure (see paper §7), not a bug.

License

Apache-2.0 (inherited from Qwen 2.5 14B-Instruct, of which this adapter is a derivative).

Citation

See the ARM repository's CITATION.cff. Also cite the RecursiveMAS paper (arXiv:2604.25917) for the inner-link architecture and training objective.