Model notes · Chapter 01
Position encoding: how Transformers keep track of order
From sine waves to RoPE: where position enters a Transformer, and what “longer context” really requires.
Blog
Thoughts, learning & sharing
A place for my thoughts, things I learn, and knowledge I want to share—including accessible introductions to my research.
Model notes · 模型学习笔记
An engineering view of Transformer building blocks, with equations and checked PyTorch examples. Three chapters, available in English, 中文 and 日本語.
Model notes · Chapter 02
Attention: an engineering view of information flow
Understand Q/K/V, self- versus cross-attention, and the memory trade-offs behind MHA, MQA, GQA and MLA.
Model notes · Chapter 03
Residual connections: learn changes, keep a direct path
Understand x + F(x), the gradient shortcut, and why adding more layers still requires care.
Research stories · 论文解读
LREC · 2026
Does knowing honorifics make AI good at writing emails?
JaSocial asks whether a model can adapt its language to the relationship between sender and recipient.
JACIII 30(4) · 2026
Are more complex AI representations better for studying language in the brain?
A comparison of two kinds of language features—and what encoding and decoding actually mean here.
arXiv · 2026 · Preprint
MindAlign: giving brain signals a bridge to language
A two-stage approach to generating descriptions from brain activity, with a frozen language model.