Researchers hid a capability inside a neural network behind a cryptographic lock. Part 2: how the lock works, what it costs and by what signs people try to spot such a model

This is a continuation. Part 1 explains in plain words what the researchers did: they hid a small encrypted network inside a neural network that works only with a secret key, and the usual testing methods could not elicit it, except for brute-forcing a very short key. Here it is about how this works: what exactly is proven, how the digital locker works, how an algorithm turns into the weights of a neural network, what it costs and where the weak points of the work are. There will be formulas, each explained in words.

There is a single source: the preprint arXiv:2610.13928 (Draguns, Motwani, Douglas, Schroeder de Witt, 9 October 2026). All information about the work below comes from it. A few numerical examples explaining the formulas are mine; they are marked with the words "my example."

First, a few words that come up constantly below. Weights are the numbers inside a neural network that define its behaviour. The key (a trigger in the preprint) is a secret string in the input. The auditor is whoever tries to find or elicit the hidden capability. Polynomial time means that the auditor's work grows no faster than some power of the size of the problem; a full brute-force search over a key of length λ grows exponentially, as 2^λ, and does not belong to this class if λ grows. This is a theoretical notion: a polynomial algorithm is not necessarily fast in practice, and brute-forcing a short key, as the experiments will show, is quite feasible.

What exactly is proven

Let H be an ordinary "host" model that is going to be used, and C the capability the attacker wants to hide: a function that produces an answer for an input x. The attacker chooses a secret key T and builds a new model:

M = Embed(H, C, T)

In words: the Embed procedure embeds the capability C into the model H so that it can be opened with the key T. The auditor receives the architecture and all the weights of M, may even know the Embed procedure itself, but does not know T.

A model is considered to contain a hidden capability if two conditions hold: on ordinary inputs x it answers like H, and on inputs of the form T‖x (the key prepended to the input) it outputs C(x); both up to the tolerance accepted for the task. The symbol ‖ means "append one string to another."

The authors define unelicitability through two games between the auditor and whoever builds the model.

1. The "find the key" game. The auditor is given M. They win if they produce an input that unlocks the hidden computation. Requirement: for any auditor running in polynomial time, the probability of winning is negligible.
2. The "guess what is hidden" game. Two capabilities of equal size are taken, C₀ and C₁. A coin is tossed: the bit b is 0 or 1. The auditor is given M_b = Embed(H, C_b, T). They win if they guess b. Requirement:

Pr[guessed b] ≤ 1/2 + negl(λ)

In words: the auditor guesses which of the two capabilities is inside no better than by tossing a coin, up to a negligibly small addition. Here λ is the security parameter, roughly "how many bits of strength." negl(λ) is a function that, as λ grows, decreases faster than 1/λ to any power. So by increasing λ, the addition can be made as small as desired.

There is a condition on the key itself: its min-entropy is at least λ. This means that no particular key is chosen with probability greater than 2^(−λ). In words: the key cannot be guessed by trying a few "popular" options. My example: with λ = 128, the probability of any single key is no more than 2^(−128), that is, about 3·10^(−39).

What the definition does not promise, the authors state plainly. It does not claim that the modified model cannot be statistically distinguished from an ordinary one, and it does not prevent the auditor from noticing that the weights are strange. It does not stop the auditor from learning the same task from other data. This property is called unelicitability, and it is weaker than undetectability: undetectability would mean that one cannot even tell that the model has been modified.

The digital locker

The whole guarantee rests on a known cryptographic primitive, the digital locker. It is a program that, on the correct key k, returns the hidden message m, and on any other input returns a failure, denoted ⊥.

The simplest construction the authors give is built on a hash function h that turns a string of a bits into a string of b bits:

c = h(k) ⊕ m, L(x) = h(x) ⊕ c

In words: the hash of the correct key is computed in advance and combined with the message by XOR, producing the ciphertext c; that is what is stored in the model. When an input x arrives, the locker computes the hash of x and combines it with c again. XOR is the bitwise "exclusive or": 0⊕0 = 0, 1⊕1 = 0, 0⊕1 = 1⊕0 = 1. It has the property that matters most here: if you add the same thing twice, it cancels out. So on the correct key

L(k) = h(k) ⊕ h(k) ⊕ m = m,

while on an incorrect x the result is h(x) ⊕ h(k) ⊕ m, that is, the message mixed with garbage.

To let the locker say "failure" explicitly, a tag of r ones is appended to the message: what is hidden is not m but m‖1^r. After decryption, the locker looks at the last r bits: if there are r ones, it returns the message; if not, it returns ⊥. In the random oracle model, where the hash function's outputs on different inputs are treated as independent and uniformly random strings, for any particular incorrect key the last r bits look random, and all of them will be ones with probability

2^(−r).

In words, my example: with a 32-bit tag, an incorrect key will "pass" by chance roughly once in four billion cases.

The security of the locker is stated in the same way as the security of the whole model: if the key is chosen with min-entropy of at least λ, no polynomial-time adversary who sees the whole locker will find an input on which the locker does not fail, except with negligible probability; and two lockers with different messages of equal length are indistinguishable to them. As h, the authors use the Keccak family, on which the SHA-3 standard is built. Keccak has a mode with output of arbitrary length, which is convenient: the length of the hash is matched to the length of the hidden weights. An important caveat from the authors themselves: the hash-based construction is proven in the random oracle model, that is, under the assumption that the hash behaves like a perfectly random function; lockers also exist without this assumption, in the standard model.

Why the security of the locker carries over to the whole model is explained in Appendix C. If everything that depends on the secret is the ciphertexts c, and the rest of the model is obtained from public data by a public procedure F,

Weights(M) = F(H, p, c; ρ),

where p is the public settings and ρ is the compiler's randomness, then anyone who knows the ciphertexts can build the model themselves. So an auditor who would learn something from the model would learn the same from the ciphertexts, and that is ruled out by the security of the locker. This is a standard technique in cryptography, reducing one problem to another.

How an algorithm turns into weights

The cryptographic part cannot be trained in the usual way: cryptographic functions, as the authors point out, generally do not lend themselves to learning from examples. So they are not trained but "compiled," that is, the weights are set by hand according to a program. The chain is:

Algorithm → program without branches → threshold circuit → matrices → GLU blocks

First, the algorithm is written as a program with a fixed number of steps and no conditional jumps. Then it is translated into a circuit of threshold gates, the simplest model of a neuron:

LTG(x) = 1 if Σ wᵢxᵢ ≥ θ; otherwise 0.

In words: the gate adds up its inputs with weights wᵢ and outputs one if the sum reaches the threshold θ. From such gates one builds "and," "or," adders and an entire Keccak hash function.

An example of expressiveness from Appendix F, XOR of n bits, that is, the parity of the number of ones, in just two layers:

c_k = 1[Σ xᵢ ≥ k], k = 1…n; XOR(x) = 1[Σ (−1)^(k−1) c_k ≥ 1]

In words: the first layer answers, for each k, the question "are there at least k ones?" The second layer adds these answers with alternating signs: if there are exactly s ones, the sum is 1 − 1 + 1 − … over s terms, that is, 1 for odd s and 0 for even s. The authors note that the same function written as a logical formula in disjunctive or conjunctive normal form (DNF or CNF) requires exponential size.

Each layer of the circuit is a matrix. But modern models do not have a step activation function; they have smooth ones. The authors build the step from the difference of two shifted ReLU functions (ReLU(z) = max(0, z)) with slope c = 4: on valid binary inputs with integer weights and thresholds, this difference reproduces the threshold gate exactly. And ReLU is approximated via Swish, the activation function used in SwiGLU blocks: ReLU(z) ≈ Swish(qz)/q with q = 4. In words: the input is stretched, passed through Swish and scaled back, and at the relevant values the result is almost ReLU. This works as long as approximation and rounding errors do not flip the binary outputs.

How the decrypted weights are executed

After the locker, what the model has are not weights but activation values that are the weights of the hidden network. They need to be multiplied by the input, and a multiplication circuit is large. The authors get around this with a GLU block, which modern transformers already have:

GLU(z) = W_o( f(W_g z) ⊙ W_v z )

Here z is the block's input, W_g, W_v, W_o are three fixed matrices, f is an activation function, and ⊙ is element-wise multiplication. In words: the block makes two different linear projections of the input, passes one through f, multiplies them element-wise and projects back. The authors feed into z a concatenation of the hidden layer's input x (of length a) and the decrypted matrix W (of size b × a, flattened into a row). The matrix W_v copies the decrypted weights, W_g makes b copies of x, after f the copies are multiplied element-wise by the weights, and W_o sums the relevant products. The output is W·f(x), that is, the hidden layer has been executed.

The cost of this trick is visible at once: the block needs b·a + a input dimensions to hold both the matrix and the input. The authors give an example: with 10,000 host dimensions, a hidden square matrix can be no larger than 99 × 99. My check: 99·99 + 99 = 9,900, while 100·100 + 100 = 10,100 no longer fits. For a network of width n, this is O(n²) dimensions.

There is a second, row-wise variant: at each step of text generation one row of the matrix is decrypted, one dot product sᵢ = ⟨W_{i,:}, f(z)⟩ is computed, and then the attention mechanism sums all the rows into one vector. Then the required width is O(n), but a network of depth d needs O(n·d) positions in the text. This trades model width for context length.

What it costs

The overhead is reported in detail in the preprint, and it is large.

What	Figure from the preprint
Hidden network for chemistry	64 → 8 → 16 → 2, 672 weights, each from {−1, 0, 1}
One Keccak-p[25,1] block	7 circuit levels, width 276, about 2.47·10^5 parameters, of which about 10^4 are non-zero
Compiled model (1 Keccak round, 4-16-bit keys)	38 layers, about 2.07·10^9 parameters, 1.90·10^6 non-zero, width 10,266
Share of hashing	about 1.98·10^9 parameters, roughly 95%
Trained password-locked model for comparison	about 1.7·10^5 parameters
Each additional hidden weight	about 3.1·10^6 host parameters
Full strength (24-round Keccak-f[1600], 128-bit key)	110 layers, 3.58·10^9 parameters
Variant with parallel hashing	15 layers, 6.37·10^9 parameters
Row-wise variant	13 layers, 0.36·10^9 parameters, executed over 26 tokens

Here "parameters" means all the numbers in the matrices, including zeros; the non-zero ones are three orders of magnitude fewer. The full-strength variant, according to the authors, matched the hidden classifier on all held-out examples with the correct key and, on all 10,000 incorrect keys tested, ran a stand-in circuit instead of the hidden one.

The main caveat to the experiments: the elicitation tests were run with weakened settings, one Keccak round and keys of 8 and 16 bits. Such settings, as the authors admit, would not withstand cryptanalysis. Their argument: the methods tested do not use the structure of the hash at all, so they do not work even on weak settings. This is an argument about these methods, not a proof for the weak settings.

Results of all the tests

Two models with one task were compared (given the starting substances and a possible product, encoded in 64 bits, say whether this product is obtained from these substances): a trained password-locked model, the standard way of making "backdoors," and the encrypted one. Accuracy in percent, mean over three runs with a 95 percent confidence interval:

Test	Password-locked	Encrypted
Without the key	50.7 ± 0.1	50.6 ± 2.1
Fine-tuning, 500 steps	92.8 ± 0.9	50.9 ± 1.0
Training with perturbations in hidden layers, eight layers	91.7 ± 0.7	52.9 ± 3.9
Gradient-based key search, 8-bit key	92.7 ± 0.2	73.5 ± 2.6
Gradient-based key search, 16-bit key	92.9 ± 0.4	52.7 ± 1.4
With the correct key	92.7 ± 0.2	73.5 ± 2.6

Gradient-based key search here is the GCG method: the program changes characters in the input one at a time and at each step chooses the replacements that, by computing derivatives inside the model, bring it closest to the desired answer. Unlike blind brute force, it uses the structure of the model, but in this experiment, with an 8-bit key, it did no better than random search with the same number of attempts.

How to read this. 50% is guessing: there are two answers and the classes are balanced. Fine-tuning and training with perturbations "got" the password-locked model almost to its full accuracy, while the encrypted one stayed around guessing. To check whether the hidden network gives any gain at all, the authors made a control model whose locker contains a random network instead of the trained one. It gave 50.6% under fine-tuning and 53.6% under training with perturbations, that is, no less than the model with the real network: no benefit from the hidden network is visible in these tests. Gradient-based key search with 8 bits simply went through all 2^8 = 256 options and found the key; random search with the same number of attempts would have done the same. With 16 bits, that is, 65,536 options, it did not succeed within the allotted number of attempts. Noise in the weights at scales of 0.01 and 0.1 before fine-tuning did not noticeably change the results.

The second experiment is on the subset parity task:

f_s(x) = ⊕_{i∈s} xᵢ

In words: the input is n bits, and the answer is the XOR of the bits whose positions are in the secret subset s. This task was chosen because for methods that learn by gradient it is provably hard under certain conditions. This makes it possible to distinguish genuine elicitation of the hidden capability from the model simply learning the task anew during the test. The experiment used 4 key bits and 4 input bits, with a subset of two bits. With the correct key, both models compute exactly. Fine-tuning without the key brought the password-locked model to 100%, while the compiled model stayed at chance level. Noise elicited the capability from neither.

How it can be spotted

Here is the work's most visible weak point, and the authors do not hide it. The compiled weights differ sharply from randomly initialised ones, with which the authors compare them; there is no comparison with the weights of trained models in the work. For an eight-layer network with Keccak, they give these statistics:

Feature	Random weights (reference)	Compiled as is	After obfuscation
Kolmogorov-Smirnov distance to the reference	0.002	0.505	0.005
Excess kurtosis	0.00	371	0.57
Share of zeros	less than 10^(−4)	0.991	0.007
Share of integers	less than 10^(−4)	0.998	0.007

The reference here is weights of the same shape, drawn at random from a normal distribution, as in ordinary network initialisation. The Kolmogorov-Smirnov distance shows how much the distribution of the weights differs from the reference: 0 means a match. Excess kurtosis shows how "heavy-tailed" a distribution is compared with the normal one; for the normal distribution it is 0. Without obfuscation, 99% of the weights are exact zeros, almost all the rest are integers, and the kurtosis runs into the hundreds: by simple statistics such weights differ sharply from random ones. Whether any particular test would distinguish them from ordinary trained weights, the work does not examine.

The obfuscation works like this: each hidden layer is widened with pairs of weights from a normal distribution that cancel each other out, and the symmetries of the SwiGLU block are applied (rotations and permutations that do not change the computation). The function is preserved exactly, while the simple statistics become similar to random ones. But the authors themselves write that similarity to random weights does not make the model similar to a trained one, and that unusual correlations remain between weight vectors within a layer, which, in their words, could help an auditor spot the compiled construction; the work contains no detection method based on them.

Hence their overall conclusion: for now the guarantee concerns unelicitability, not undetectability. The auditor may see that something is wrong with the model but, under the assumptions of the work, will not be able to find out which of the two capabilities is hidden or how to switch it on. And even detection, according to the authors, may not offer a clear remedy: the auditor may be unable to determine whether it is a harmful capability, a harmless watermark or an artifact of the architecture, and removing the construction risks degrading the model. As a possible way out, the authors name the potentially costly decision not to use the model.

What is proposed against this

In Appendix H the authors list possible defences, each with a weak point:

• Searching for anomalies in the weights. Statistics of the weights and of the computational graph, interpretability tools. Weak point: obfuscation already brings the simple statistics closer, and interpreting ordinary networks is hard enough as it is.
• Monitoring at runtime. The hidden capability switches on only with the key, and at that moment its behaviour or internal activations may stand out. The authors call this a promising direction.
• Pruning the model and paraphrasing the inputs. These can break the execution of the hidden network without detecting it. Weak point: the attacker can add error correction and make the key semantic, so that paraphrasing does not remove it.
• Verifying the provenance of the model along the whole path from data and training to deployment.

How the work relates to earlier ones

Work	Property	What is hidden	Implementation
Goldwasser et al., 2022	undetectability	flipping a classifier's output bit	no
Draguns et al., 2024	unelicitability	a fixed string	PyTorch
This work	unelicitability	an executable model	PyTorch

The step forward is that what is inside is now not an answer but a computation: the hidden network receives an input and computes something from it.

Limitations, collected together

• Experiments on small models and narrow tasks: one tiny classifier and parity over a few bits.
• The elicitation tests were run with weakened cryptographic settings; the full-strength model was built and checked for correctness, but, judging by the text, the full set of attacks was not run against it.
• The full construction was not embedded into a real trained model: individual GLU blocks were checked inside Gemma 3 270M, and the rest, including attention and residual connections, is left for future work.
• The overhead is enormous: billions of parameters for 672 hidden weights.
• The fact of tampering, according to the authors themselves, may be detectable.
• The authors do not claim that such structures arise in ordinary training.

What follows from this

These are my conclusions from the preprint, not its claims. The strong part of the work is that the guarantee is proven, not only observed in experiments. But the proof applies to auditors whose work grows polynomially, to assumptions about the security of the locker and to a sufficiently large security parameter: with a short key, as the 8-bit experiment showed, brute force succeeds. Within these limits, the question "does the model have a dangerous capability" cannot be settled by behavioural tests alone for a model from an untrusted source. The weak part is cost and visibility: in its current form the construction is bulky, and the authors themselves write that its presence may be detectable from the weights.

What a company should do about this, step by step, with the same caveat that this is my advice:

1. Start with a list of the models you run yourself. For each one, write down where it was downloaded from and who released it.
   • Who does it: whoever is responsible for AI tools in the company.
   • How to check: each row of the list has the address of the developer's official page and a note on whether the fingerprint (checksum) of the running file matched the one published by the developer. The page address by itself does not confirm that this exact file is running: a copy may have been altered. If the developer does not publish fingerprints, write that down: the file's provenance is not confirmed, and such a model goes under point 2.
2. If the provenance cannot be established (a model from a random mirror, an "improved" build by an unknown author), replace it with a model from a known developer or use it only where it has no access to important data and systems.
   • Who does it: the same person together with the manager who decides where to connect the model.
   • How to check: for models without known provenance, the list records what they have access to, and that list contains no working systems with money, client data or the right to send messages on behalf of the company.
3. Restrict the model's permissions and keep a log of its actions following the rules from part 1: only the time, the type of action and the system, without the contents of emails, files and conversations, with a check on a test copy that personal data and passwords do not get into the record, and if a password or access key does get in, change it and delete the record. If a log cannot be kept (for example, the model is built into someone else's service without access to its actions), assume there is no monitoring and give such a model fewer permissions, not more.
   • Who does it: the administrator.
   • How to check: a test on a test copy that the prohibited actions are not carried out, and a record of which models have a log and which do not.

Summary

The preprint formally defines a hidden capability through two games, "find the key" and "guess what is hidden," and reduces their security to the security of a digital locker built on the Keccak hash function. The algorithm is compiled by hand into threshold circuits and then into GLU blocks, which can execute the decrypted weights without a separate multiplication circuit. In the experiments, fine-tuning, training with perturbations and searching for a 16-bit key did not elicit the hidden chemistry capability (the 8-bit key was found by brute force), whereas fine-tuning exposed an ordinary password-locked "backdoor." The construction costs billions of parameters, and without obfuscation its weights differ sharply from random ones by simple statistics; with obfuscation the simple statistics converge, but the authors observed unusual weight correlations which, in their words, could help to spot it; a reliable detection method is not shown in the work.

I work on AI agents and automation. If you would like to see my projects or discuss your own task, take a look at my portfolio.

Sources

• Andis Draguns, Sumeet Ramesh Motwani, Raymond Douglas, Christian Schroeder de Witt: AI Models Can Provably Hide Arbitrary Capabilities. arXiv:2610.13928, 09.10.2026. https://arxiv.org/abs/2610.13928 (full text: https://arxiv.org/html/2610.13928v1)