← Back to Blog

JACIII 30(4)

Are more complex AI representations better for studying language in the brain?

A comparison of two kinds of language features—and what encoding and decoding actually mean here.

Muxuan Liu · About 5 min read

Turn words into numbers, then compare them with brain activity

Language models represent words as numerical vectors. Think of them as digital word cards: some are relatively fixed, while others change with the surrounding sentence. The latter can represent the same word differently in different contexts.

Researchers can compare these features with brain activity recorded while people listen to a story. This paper asks whether a richer, context-sensitive representation is necessarily better for that task.

Two representations, two prediction directions

The paper compares input embeddings with hidden states. An input embedding is the initial representation retrieved by the model; a hidden state is a representation after processing, influenced by context. Here, hidden states come from the final layer under a specific time-segment extraction setup. The conclusions should be read within those choices.

Encoding predicts brain activity from language features. Decoding works in the reverse direction, predicting language features from brain activity. The decoding target here is a numerical representation, not a generated sentence describing what a person is thinking.

How the comparison is set up

The study uses fMRI recordings from three participants who repeatedly listened to the same story, and compares several language-model families. It tests a relatively simple linear mapping, ridge regression, alongside a bidirectional LSTM that can use sequential information.

Repeated presentations help estimate which brain responses are reliable. The paper uses explainable variance to examine this reliability and separately studies more stable brain signals. Before asking whether a prediction is good, it helps to ask how much dependable signal is available to predict.

The main finding: simple features deserve a serious baseline

Input embeddings remain competitive and outperform hidden states in some settings. Adding context provides more information, but that information is not necessarily easier to map to brain recordings with limited data.

The sequential model improves some results in particular brain regions, rather than winning in every condition. Reliable signals matter substantially. An improvement in one metric also does not establish reliable recovery of language content. The paper therefore considers both predictive performance and how readily the mapping can be interpreted.

What the results do not establish

This does not mean that context is unimportant to the brain, or that input embeddings always beat hidden states. Feature extraction, layer choice, mapping method, and recording reliability all affect the comparison.

The experiment includes three participants and repeated recordings of one story. Testing on a different repetition is not the same as testing on an entirely unseen story. Better prediction also does not prove that a language model and a human brain perform the same computations.

A useful lesson for reading brain–AI research

Start with a strong simple baseline, then ask what extra complexity contributes. Alongside the score, inspect signal reliability, the train–test split, and the exact prediction target. These details help clarify what a claim of brain–model alignment actually means.