How the transformer analyzes Korean: Bareun's model architecture

Bareun's Korean morphological analysis runs on transformer-based deep learning models. Instead of one giant model doing everything, it uses two models: a model for segmentation (seg-model.onnx) and a model for POS tagging (tag-model.onnx). This article explains how the two models take a Korean sentence and produce an analysis result.

Three pillars that support performance

Bareun's analysis performance comes from the combination of the following three things.

1. Segmentation model: Based on a transformer encoder, it attaches a segmentation label to each syllable (character) to find morpheme boundaries. There are 115 segmentation labels in total (14 of them for error detection).
2. Tagging model: A transformer model that assigns one of the 47 parts of speech defined by the National Institute of Korean Language (NIKL) to each segmented piece.
3. Word embeddings with morphological information: Words with the same surface form but different morphology are embedded as different tokens, which reduces ambiguity.

These three pillars work together to resolve, through context, the conjugation, contraction, and omission and the homonym problems of Korean that are hard to solve with simple dictionary-based matching.

Two models, two roles

Bareun does not solve Korean morphological analysis in one step. It splits it into the problem of cutting and the problem of attaching labels.

graph TD
  IN[Input sentence] --> SEG[seg-model.onnx<br>Segmentation model];
  SEG --> TAG[tag-model.onnx<br>POS tagging model];
  TAG --> POST[Post-processing];
  POST --> OUT[Morpheme result];

Segmentation model: cutting by character

The segmentation model has a transformer encoder structure. It reads the input sentence in syllable (character) units and attaches a segmentation label to each syllable, making it a syllable-level labeling model. It processes 160 characters at a time, and splits sentences longer than that at whitespace into pieces of nearly equal length, processes them, and then joins them back together. It was trained on 80-character pieces, but this model looks only at nearby syllables before and after (within about 48 syllables on each side) and reads positions relatively, so accuracy was the same on 160-character input (we directly compared 80-character and 160-character windows with the same model bundle). There are 115 segmentation labels in total, of which 14 are error detection labels that catch morphological errors.

At this stage, the inside of an eojeol (space-delimited word unit) is divided into eight major morphological units, such as nominals (N), particles (J), predicates (V), and endings (E). For example, the boundary that splits 학교에서 into 학교/N and 에서/J is decided at this stage.

Tagging model: assigning 47 parts of speech

Attaching a part of speech to each segmented piece is the tagging model's job. In the new structure, the segmentation-linked POS model takes this job first (reading the whole sentence at once), and the tagging model takes over when the segmentation-linked POS model cannot be used. The tagging model reads 40 morphemes at a time, and there are 49 output labels. Labels 0 to 46 are the 47 parts of speech, 47 is whitespace, and 48 is padding.

가방을  →  가방/NNG + 을/JKO
먹었다  →  먹/VV + 었/EP + 다/EF

Inference with ONNX Runtime

Both models are stored in ONNX format, and Bareun runs inference through onnxruntime_go.

The probability obtained from softmax becomes each morpheme's probability value (tagging confidence) as is. A morpheme with low confidence can be used as a signal that the analysis is ambiguous at that point.

Why a transformer?

Korean is an agglutinative language, so several morphemes attach in a single eojeol, and conjugation, contraction, and omission happen constantly. Listing words one by one in a dictionary cannot keep up with new words or spoken style. A transformer model infers segmentation and parts of speech by looking at the whole context, so it can guess even words that are not in the dictionary from the surrounding context. The result is 99.6% accuracy in POS tagging and 99.7% in eojeol splitting/restoration.

Frequently asked questions

Q. Why are the segmentation model and the tagging model separate?

Cutting and attaching parts of speech are different kinds of problems. Splitting into two models lets each model solve a simpler and clearer problem, which raises overall analysis accuracy.

Q. Are long sentences over 160 characters also analyzed?

Yes. Segmentation processes text in 160-character units, so long sentences are analyzed without problems. For parts of speech, the segmentation-linked POS model reads up to 1,024 characters and 512 slots at once, and the tagging model handles anything longer.

Q. Does it work without a GPU?

Yes. If initialization of the accelerated execution provider fails, it automatically falls back to CPU inference, so you get the same analysis result even in an environment without a GPU.

Q. What kind of transformer structure is the segmentation model?

The segmentation model is based on a transformer encoder and is a syllable-level labeling model that attaches a segmentation label to each syllable (character). There are 115 segmentation labels in total, of which 14 are error detection labels that catch morphological errors.

Q. What are the key elements that support Bareun's analysis performance?

Three pillars are combined: the segmentation model (transformer encoder, 115 syllable-level labels), the tagging model (47 parts of speech), and word embeddings with morphological information. Together they resolve, through context, the conjugation, contraction, and omission of Korean and the ambiguity of homonyms.