Shared representations and the evaluation of vulnerability detectors

Summary

Independently trained neural networks agree on part of what they compute from an input and differ on the rest. The part they agree on is the structure that the training data imposes on every network that fits it; the part they differ on is whatever the data leaves undetermined, and it changes from one training run to the next. A vulnerability label is a property of what a program does when it runs. The text of the program does not decide it, and no language model has been trained on it. The label is therefore not among the directions that training forces, and a model that holds it at all holds it in the undetermined part. The same observation accounts for five difficulties that evaluations of vulnerability detectors have reported for a decade, and it tells a benchmark designer what can be measured by observation and what can only be measured by intervention.

figure 1. Three models, each drawn in its own coordinates, with the same two function pairs. Circles are the vulnerable versions and squares the versions after the fix; the fix moves a function along the free direction, by a different amount and in a different sense in each model, and never along the forced direction. Each model's decision threshold is a cut on the forced direction at its own position, so a pair may sit on either side of it depending on the model, and both versions of a pair stay on one side in every model.

What independently trained networks share

The claim that networks converge on a common representation has been made in a strong form and a measured form. The strong form holds that representations across models and modalities approach one statistical model of the world as models grow (Huh et al. 2024), and it has a constructive demonstration: text embeddings from one encoder can be translated into the space of another with no paired data and then inverted to text (Jha et al. 2025). The measured form is more careful about what agrees and when. Kapoor et al. (2025) compared dozens of vision models at every layer and at many points in training, under three families of alignment map. They found that an orthogonal map aligns two networks almost as well as an unrestricted linear one, that nearly all of the final alignment is in place after the first epoch, before the networks have solved their task, and that under a change of input distribution the early layers stay aligned while the deep layers diverge in proportion to the shift. The alignment, on this evidence, is set by the input statistics and the architecture rather than by the solution.

Identifiability theory supplies the terms. Roeder et al. (2021) showed that a broad class of discriminative models, including contrastive and classification objectives, is identifiable in function space up to a linear map: two minimizers of the same objective on the same data are related by an invertible linear transformation. Nelson et al. (2026) then separated two properties that are usually conflated. Statistical identifiability is the consistency of representations across training runs; structural identifiability is their alignment with an unobserved ground truth. A model can have the first without the second, and most of the convergence literature measures the first. Calibrated measurements have since narrowed what survives. After permutation nulls, the structure that is stable across independently trained models is local neighbourhood structure rather than global geometry (Gröger et al. 2026); cross-modal alignment weakens as the number of samples grows (Koepke et al. 2026); and language models that agree on representations do not agree on how they reason (Usama and Chang 2026).

What these results leave is graded. Two networks share the directions of their representation that the data determines on both, which are also the directions of largest variance, and they do not share the directions the data leaves free. Which features a network learns when several would suffice is decided by the training dynamics, which prefer the easier feature, and easy features are represented more consistently across runs than hard ones (Hermann and Lampinen 2020). The shared part is the part of the input that every model trained on that data has to represent in order to fit it; the architecture and the objective set the rest.

What a vulnerability label is

A vulnerability label states what a program does on some input: a sanitizer reports an out-of-bounds write, or a proof-of-concept input causes an observable effect. The label is a property of execution. It is not a property of the program text in the sense that matters here, because two programs whose texts differ by a few tokens can have opposite labels, and because the same text can be vulnerable or safe depending on code outside the function (Risse et al. 2025). A language model is trained on text, and its training data contains no execution labels. Nothing in the training therefore forces any direction of the representation to encode the label. If the label is present in the representation, it is present in a free direction, learned incidentally and differently by different models, or it is present only through the surface regularities that correlate with it in the training corpus.

The paired test is where this becomes visible. A paired test presents a vulnerable function and the same function after its fixing commit, and asks the model to flag the first and clear the second. The two inputs differ along the direction the training did not force. Risse and Böhme (2024) found that six detectors separate a function from its patched version no better than chance, and that semantics-preserving edits change the verdicts. Ding et al. (2025) repaired the labels of the standard corpora, introduced pair-wise scoring, and found that GPT-4 with chain of thought does not pass the paired test above the rate of guessing. Ullah et al. (2024) found that renaming functions or variables alone leads models as strong as GPT-4 and PaLM2 to a wrong answer in about a quarter of cases.

We measured the same thing on a larger population (Cichoń and Dmitruk 2026). Sixty-eight models from 1.5 to 36 billion parameters and seven families, among them security fine-tunes and variants with the refusal behaviour removed, were given the same thousand pairs under one prompt. For 64 of the 68 models the two functions of a pair receive the same verdict more often than independent answers would give; the excess over independence is 0.055 at the median, and it is present at every flag rate from 0.16 to 0.96. Security tuning does not change it and neither does abliteration. The verdict is determined by the text the two sides share. On the same functions we recorded the internal activations and measured how much label a linear probe can read from them: 0.78 within-pair accuracy on length-matched pairs, against 0.76 for a bag of character n-grams of the raw text. We also measured how much the representations of the 68 models agree with one another: well above their agreement with the n-gram space, and well above an untrained network of the same architecture. The convergence is above the surface statistics and above the untrained network, and the label is not part of it.

Five difficulties with one cause

Label leakage. The corpora on which detectors are trained and tested come from fix commits, and their labels are correlated with input statistics that have nothing to do with execution. The patched version is usually the longer one and contains the guard; the vulnerable version is the shorter one and lacks it. Croft et al. (2023) report label inaccuracy between 20 and 71 percent and duplication between 17 and 99 percent in the common sets, and Risse et al. (2025) report that for most sampled functions the label cannot be decided from the function alone. A model that computes the input statistics will score well on such a corpus with no access to the bug, and will score well on new data drawn the same way. This is often discussed as contamination, which suggests that a model has seen the specific examples. The sharing view gives a different diagnosis: the label is reachable from the shared part of the representation, which every model computes whether or not it has seen the example. A temporal split does not remove this, and the finding that strict temporal splits show little contamination effect is consistent with it. Two things remove it. One can match the two versions on their surface statistics, as the length-matched subset above does, or one can edit the cause and observe whether the verdict moves.

Unstable rankings. If the shared part dominates every model's representation and the label is not in it, then the differences between models that a benchmark can detect are differences in the free part, which has low variance and is the part least reproducible from one training run to the next. A benchmark that scores reports against a list of known bugs therefore measures mostly the shared part, which is the same in every model, plus a small and noisy remainder. In our evaluation the choice of evaluation protocol changed a model's score more than the choice of model changed a benchmark's score, and 37 of 68 models had a pair-level score whose interval included zero (Cichoń and Dmitruk 2026). Ties are the expected outcome, and a leaderboard that reports one number per tool reports mostly the resampling noise of the free part. A benchmark should instead report, per tool, which edits to the code move its verdict, and should report how stable the ordering of tools is under resampling of the items, as the medical segmentation challenges learned to do.

Judges. It is common to let a language model grade the report of another. A judge computes the same shared structure as the detector it grades, because it was trained on the same kind of data. Agreement between judge and detector is therefore agreement on the shared part and says nothing about the truth, and the published findings that judges are moved by rhetoric, by self-affirming comments and by framing text in a pull request are what this predicts. A benchmark should not read the explanation at all. It should change the code and observe the verdict, which requires no model to read the report.

Training on the benchmark. Hermann and Lampinen (2020) showed that when two features both predict the label, a network preferentially represents the easier one, and that easy features are represented more consistently across training runs than hard ones. A model fine-tuned on pairs of real fixes and decoy fixes will therefore tend to learn the easiest difference between the two kinds of edit, which may be the shape or placement of the diff rather than the cause of the bug, and it will tend to learn it across seeds. A benchmark that is also used as a training signal therefore has to hold the edit operators used in training out of the test set, in the same way that projects are held out, or it measures its own gameability.

Scale. The sharing view gives no reason to expect that a larger model moves the label into the shared part. Scale sharpens the structure the data forces; it does not add a direction the data does not force. In our population the agreement of a model with the rest did not increase with parameter count between 1.5 and 36 billion parameters, which is a small range, but it is the range in which open models are evaluated. A larger model is therefore not the remedy. The remedy is a change in what the training forces, and execution feedback as a training signal is the direct form of it: a sanitizer or a proof-of-concept exploit gives a reward that depends on what the code does rather than on what it says. A benchmark score can rise for the reasons given above, so it does not show that such training worked. What shows it is whether the label direction has become shared between training runs, which is a measurement on representations and not on verdicts.

What this says about benchmark design

The practical consequence is a division between what observation can measure and what requires intervention. An observational benchmark, which scores a report against a list of known bugs in a fixed revision, measures recall against the list and cannot score the findings outside it; and because it scores the shared part, it will show small and unstable differences among models that compute the same structure. An interventional benchmark edits the cause and scores the movement of the verdict. For a finding that states, implicitly or explicitly, that input from one place reaches a dangerous operation at another without a check in between, the natural edits are the real fix, a surface-only rewrite of that fix, a decoy that looks like a fix and does not close the bug, the removal of the source, and the replacement of the sink. Each has an expected effect on the verdict if the finding tracks the cause, and none if it tracks the surface. The decoy is the informative row: a tool whose verdict disappears on the decoy as often as on the real fix is reading the shape of the edit, which is what the dynamics argument above predicts for every model trained on text alone.

Such a benchmark has to be read against each tool's own reference. The same-verdict excess shows that a model's answers on two versions of one function are not independent, so the chance level for any verdict-movement score is a responder with the tool's own flag rate and its own flip rate under no edit; a coin is the wrong reference. Both can be measured before any variant is scored, and both have to be subtracted. The remaining number, the probability that the verdict stays on a decoy minus the probability that it stays on the real fix, is the quantity the sharing view says will be near zero for text-trained tools, and the profile of that number across kinds of decoy is the quantity it says will separate tool classes without separating individual tools.

figure 2. The probability that a tool's verdict stays after each edit, for a tool that flips on 10 percent of functions with no edit. The verdict of a tool that tracks the cause moves on the real fix and on its rewrite and stays on the decoy; the verdict of a tool that reads the shape of the edit moves on all three; the verdict of a tool that reads the text both versions share stays on every edit. The dashed line is the tool's own no-edit reference and the dotted line a coin.

What execution-labelled code offers in return

In vision and in text the ground truth of a representation is another representation. A caption describes an image and nothing outside the pair checks either, and a model's embedding of a sentence can only be compared with another model's embedding of the same sentence. These fields therefore cannot ask whether the structure that converges corresponds to anything outside the input, and the strong form of the convergence hypothesis is, in those fields, not testable. Code with execution-verified labels is different. The label comes from running the program, and it is independent of every property of the text that a model could have learned. Whether the converged structure of a population of code models contains that label is a well-posed question, and the answer so far is that it does not. Vulnerability data is therefore a test case for the representation view as well as an application of it, and the measurement that would settle the question, a comparison of what two models share direction by direction against the execution label, is one that no vision or text corpus can provide.

References

Cichoń, M. and Dmitruk, B. (2026). A Function-Level Vulnerability Score Measures Flag Rate More Than the Model: Protocol Effects on Paired Benchmarks. NeurIPS 2026 Workshop on Trust-AI-Eval (TAE). https://arxiv.org/abs/2609.32890

Croft, R., Babar, M. A. and Kholoosi, M. M. (2023). Data Quality for Software Vulnerability Datasets. IEEE/ACM International Conference on Software Engineering (ICSE). https://arxiv.org/abs/2301.05456

Ding, Y., Fu, Y., Ibrahim, O., Sitawarin, C., Chen, X., Alomair, B., Wagner, D., Ray, B. and Chen, Y. (2025). Vulnerability Detection with Code Language Models: How Far Are We? IEEE/ACM International Conference on Software Engineering (ICSE). https://arxiv.org/abs/2403.18624

Gröger, F., Wen, S. and Brbić, M. (2026). Revisiting the Platonic Representation Hypothesis: An Aristotelian View. International Conference on Machine Learning (ICML). https://arxiv.org/abs/2602.14486

Hermann, K. L. and Lampinen, A. K. (2020). What Shapes Feature Representations? Exploring Datasets, Architectures, and Training. Advances in Neural Information Processing Systems (NeurIPS). https://arxiv.org/abs/2006.12433

Huh, M., Cheung, B., Wang, T. and Isola, P. (2024). The Platonic Representation Hypothesis. International Conference on Machine Learning (ICML). https://arxiv.org/abs/2405.07987

Jha, R., Zhang, C., Shmatikov, V. and Morris, J. X. (2025). Harnessing the Universal Geometry of Embeddings. Advances in Neural Information Processing Systems (NeurIPS). https://arxiv.org/abs/2505.12540

Kapoor, C., Srivastava, S. and Khosla, M. (2025). Bridging Critical Gaps in Convergent Learning: How Representational Alignment Evolves Across Layers, Training, and Distribution Shifts. Advances in Neural Information Processing Systems (NeurIPS). https://arxiv.org/abs/2502.18710

Koepke, A. S., Zverev, D., Ginosar, S. and Efros, A. A. (2026). Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scale. arXiv preprint arXiv:2604.18572. https://arxiv.org/abs/2604.18572

Nelson, W., Fumero, M., Karaletsos, T. and Locatello, F. (2026). Statistical and Structural Identifiability in Representation Learning. International Conference on Learning Representations (ICLR). https://arxiv.org/abs/2603.11970

Risse, N. and Böhme, M. (2024). Uncovering the Limits of Machine Learning for Automatic Vulnerability Detection. USENIX Security Symposium. https://arxiv.org/abs/2306.17193

Risse, N., Liu, J. and Böhme, M. (2025). Top Score on the Wrong Exam: On Benchmarking in Machine Learning for Vulnerability Detection. ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA). https://arxiv.org/abs/2408.12986

Roeder, G., Metz, L. and Kingma, D. P. (2021). On Linear Identifiability of Learned Representations. International Conference on Machine Learning (ICML). https://arxiv.org/abs/2007.00810

Ullah, S., Han, M., Pujar, S., Pearce, H., Coskun, A. and Stringhini, G. (2024). LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks. IEEE Symposium on Security and Privacy (S&P). https://arxiv.org/abs/2312.12575

Usama, M. and Chang, D. E. (2026). Convergence Without Understanding: When Language Models Agree on Representations but Disagree on Reasoning. arXiv preprint arXiv:2605.23315. https://arxiv.org/abs/2605.23315