← Back to Blog

LREC 2026

Does knowing honorifics make AI good at writing emails?

JaSocial asks whether a model can adapt its language to the relationship between sender and recipient.

Muxuan Liu · About 5 min read

Start with an everyday situation

Imagine cancelling a meeting. An email to your professor and a message to a friend will usually sound different. A highly formal message to a friend may be grammatically correct but feel distant. This is an illustrative example, not a verbatim item from the experiment.

The paper asks whether AI can make this adjustment: given who is speaking, who is receiving the message, and how they relate, can it choose appropriate language? Japanese honorifics provide a useful setting for studying that question.

Why counting polite words is not enough

More honorifics do not automatically make an email better. A system rewarded only for polite expressions could sound formal in every situation.

JaSocial examines three layers: the social relationship, what the email is doing—such as requesting, explaining, or thanking—and the honorific forms used to express it. The aim is to check whether relationship, purpose, and wording fit together.

How the experiment works

The study builds on 1,200 Japanese emails, with 200 for each of six relationships. These include student-to-professor, student-to-friend, employee-to-supervisor, and employee-to-colleague. The team annotates these emails and trains Japanese BERT classifiers to identify the relevant features automatically.

For generation, 16 language models write emails for 30 situations, producing 480 outputs. Each situation has five human reference emails. Generated and reference emails are converted into tag-based features and compared for similarity. The resulting score measures alignment with reference expressions; it is not a probability that an email is correct.

The surprising difficulty: writing to friends

Many models perform better in formal hierarchical settings than in casual ones. They tend to be too formal when addressing friends. In the paper, thanking a professor for thesis guidance receives an average score of about 0.907, while apologizing to a friend for cancelling a study meeting receives about 0.660. These are different situations illustrating the overall pattern, not a controlled experiment changing only the recipient.

Producing honorifics is therefore different from adjusting social distance. Fluency alone does not tell us whether a model has understood the relationship. The reported results concern the model versions tested in the study, not a current ranking of all available systems.

What this helps us understand

Separate scores make failures easier to investigate. Did the model misunderstand the relationship, miss the intended request or thanks, or choose an unsuitable register? That diagnosis can be more useful than a single overall score when improving an email assistant.

The scope is limited to Japanese email scenarios, and automatic annotation can make mistakes. Similarity to references cannot guarantee appropriateness in every real interaction. Nor does this benchmark measure every aspect of social intelligence.

Code and further reading

The JaSocial GitHub repository provides the evaluation pipeline and examples; model weights are downloaded separately. It currently includes sample emails, with access to the full dataset described in its README. The 2025 paper introduced the framework; this 2026 study adds the implementation and multi-model evaluation.