Accuracy Results and Method
Bareun morphological analysis accuracy: 3.4.0 results and how they were measured
Bareun's Korean morphological analysis accuracy is 97.8% at the eojeol level and 98.5% morpheme F1.
We measured the 3.4.0 production bundle (T261103-89ADFA) on 2026-11-06 with 286 gold sentences (2,079 eojeol) that were not used in training.
This page explains what these values measure, how they were measured, and where the accuracy comes from.
Accuracy metrics
| Metric | Value | Meaning |
|---|---|---|
| Eojeol accuracy | 97.84% | Share of eojeol whose analysis (morphemes and POS) matches the gold answer exactly |
| Morpheme F1 | 98.52% | Precision and recall at the morpheme level, combined |
The 95% confidence interval is about ±0.6 percentage points. The sample is small, so read the digits after the decimal point as a rough guide only.
How it was measured
We used a harness (taggold) that scores the tagger's raw output (before post-processing) against human-reviewed gold sentences.
| Gold set | Sentences | Eojeol | Eojeol accuracy | Morpheme F1 |
|---|---|---|---|---|
| Gold not used in training | 286 | 2,079 | 97.84% | 98.52% |
| Regression gold (partly training sentences) | 1,595 | 6,951 | 95.38% | 97.51% |
- The first row is the value to read as general performance. These sentences were not used in training, so it is not a value fitted to the training data.
- The regression gold deliberately collects hard cases such as the connective ending
-ㅁand the contraction실팬데, so it scores lower than ordinary sentences. - These values are for the tagger output before post-processing. The final result after post-processing (dictionary and rules) can be better.
Accuracy by model (segmentation model and POS tagging model)
Analysis has two stages: a segmentation model and a POS tagging model. The values below measure each model in the 3.4.0 production bundle on its own.
| Model | Round | Evaluation data | Measured on | Values |
|---|---|---|---|---|
| Segmentation model (seg) | SEG261019A | Validation split held out from the training corpus, about 420,000 sentences (5%) | 2026-10-21 | Syllable 99.83%, eojeol 99.10%, natural eojeol 99.11% (original eojeol, excluding augmented sentences) |
| POS tagging model (tag) | U1965-P22 | Validation split not used in training, 179,871 sentences | 2026-10-22 | Morpheme POS 99.38% (share of morphemes given the correct POS, assuming segmentation was right), eojeol 98.24% (using the segmentation this model produced) |
Do not read the per-model values and the overall value on the same scale
The per-model values were measured on validation splits from the same source as the training corpus, so they come out higher. The eojeol accuracy of 97.84% above scores the real analysis result, with segmentation and tagging chained together, on 286 human-reviewed gold sentences from a different source than the training corpus. The overall value is the one closer to the quality users actually get. The bundle also contains a fallback tagging model (TAG261019M) that is used only for sentences the main tagger cannot handle. We could not retrieve its validation value this time, so it is not listed. The main tagger handles most sentences.
Other evaluation sets we look at
- Ambiguity evaluation set: We collected cases where the same eojeol is analyzed differently depending on context, at a scale of 35,396 sentences / 8,285 surface forms, and measure whether Bareun picks the analysis that fits the context.
- VCP (copula '이') restoration evaluation: We separately evaluate whether the often-omitted '이' is correctly restored in the analysis, using 12,129 sentences extracted from the Modu Corpus.
Three pillars that build accuracy
graph TD A[Corpus cleaning] --> ACC[High analysis accuracy]; B[Separate segmentation and tagging stages] --> ACC; C[In-house tagging guidelines] --> ACC;
1. Corpus cleaning
We fix tagging errors in the Sejong, Modu, and news corpora, align them for consistency, and train on them. When a model trains on clean data, it does not learn confusing signals.
2. Separating the segmentation stage from the tagging stage
Bareun splits "where to cut" (segmentation) and "what POS it is" (tagging) into separate models. Each model solves only a simpler, clearer problem, so accuracy is higher than when one model handles everything.
3. In-house tagging guidelines
Criteria for segmentation and POS handling are set consistently word by word, as in 누가 → 누 + 가.
The same word is always analyzed the same way, so results are stable.
Added robustness from spacing augmentation and out-of-vocabulary handling
Not shaken by informal input
Training with word spacing augmentation lets Bareun find morpheme boundaries even when users get word spacing wrong, and it guesses new words that are not in the dictionary from context. This robustness supports accuracy in real-world use.
All these elements together produce the accuracy above.
Frequently asked questions
Q. How accurate is Bareun's morphological analysis?
97.8% at the eojeol level and 98.5% morpheme F1. We measured the 3.4.0 production bundle on 2026-11-06 with 286 gold sentences (2,079 eojeol) not used in training.
Q. How is this accuracy possible?
It rests on three pillars: cleaned corpora, separating the segmentation stage from the tagging stage, and consistent in-house tagging guidelines. Spacing augmentation and context-based guessing of out-of-vocabulary words add robustness.
Q. How do eojeol accuracy and morpheme F1 differ?
Eojeol accuracy is the share of eojeol whose analysis matches the gold answer entirely. Morpheme F1 combines precision and recall at the morpheme level. One eojeol can contain several morphemes, so eojeol accuracy comes out lower.
Q. How was this measured?
We scored the tagger's raw output against human-reviewed gold sentences. The basis is 286 gold sentences (2,079 eojeol) not used in training. The regression gold of 1,595 sentences, which collects hard cases, scores 95.38% (morpheme F1 97.51%).