3.4.0

Accuracy Results and Method

Bareun morphological analysis accuracy: 3.4.0 results and how they were measured

Bareun's Korean morphological analysis accuracy is 97.8% at the eojeol level and 98.5% morpheme F1. We measured the 3.4.0 production bundle (T261103-89ADFA) on 2026-11-06 with 286 gold sentences (2,079 eojeol) that were not used in training. This page explains what these values measure, how they were measured, and where the accuracy comes from.

Accuracy metrics

MetricValueMeaning
Eojeol accuracy97.84%Share of eojeol whose analysis (morphemes and POS) matches the gold answer exactly
Morpheme F198.52%Precision and recall at the morpheme level, combined

The 95% confidence interval is about ±0.6 percentage points. The sample is small, so read the digits after the decimal point as a rough guide only.

How it was measured

We used a harness (taggold) that scores the tagger's raw output (before post-processing) against human-reviewed gold sentences.

Gold setSentencesEojeolEojeol accuracyMorpheme F1
Gold not used in training2862,07997.84%98.52%
Regression gold (partly training sentences)1,5956,95195.38%97.51%
  • The first row is the value to read as general performance. These sentences were not used in training, so it is not a value fitted to the training data.
  • The regression gold deliberately collects hard cases such as the connective ending -ㅁ and the contraction 실팬데, so it scores lower than ordinary sentences.
  • These values are for the tagger output before post-processing. The final result after post-processing (dictionary and rules) can be better.

Accuracy by model (segmentation model and POS tagging model)

Analysis has two stages: a segmentation model and a POS tagging model. The values below measure each model in the 3.4.0 production bundle on its own.

ModelRoundEvaluation dataMeasured onValues
Segmentation model (seg)SEG261019AValidation split held out from the training corpus, about 420,000 sentences (5%)2026-10-21Syllable 99.83%, eojeol 99.10%, natural eojeol 99.11% (original eojeol, excluding augmented sentences)
POS tagging model (tag)U1965-P22Validation split not used in training, 179,871 sentences2026-10-22Morpheme POS 99.38% (share of morphemes given the correct POS, assuming segmentation was right), eojeol 98.24% (using the segmentation this model produced)

Do not read the per-model values and the overall value on the same scale

The per-model values were measured on validation splits from the same source as the training corpus, so they come out higher. The eojeol accuracy of 97.84% above scores the real analysis result, with segmentation and tagging chained together, on 286 human-reviewed gold sentences from a different source than the training corpus. The overall value is the one closer to the quality users actually get. The bundle also contains a fallback tagging model (TAG261019M) that is used only for sentences the main tagger cannot handle. We could not retrieve its validation value this time, so it is not listed. The main tagger handles most sentences.

Other evaluation sets we look at

  • Ambiguity evaluation set: We collected cases where the same eojeol is analyzed differently depending on context, at a scale of 35,396 sentences / 8,285 surface forms, and measure whether Bareun picks the analysis that fits the context.
  • VCP (copula '이') restoration evaluation: We separately evaluate whether the often-omitted '이' is correctly restored in the analysis, using 12,129 sentences extracted from the Modu Corpus.

Three pillars that build accuracy

graph TD
  A[Corpus cleaning] --> ACC[High analysis accuracy];
  B[Separate segmentation and tagging stages] --> ACC;
  C[In-house tagging guidelines] --> ACC;

1. Corpus cleaning

We fix tagging errors in the Sejong, Modu, and news corpora, align them for consistency, and train on them. When a model trains on clean data, it does not learn confusing signals.

2. Separating the segmentation stage from the tagging stage

Bareun splits "where to cut" (segmentation) and "what POS it is" (tagging) into separate models. Each model solves only a simpler, clearer problem, so accuracy is higher than when one model handles everything.

3. In-house tagging guidelines

Criteria for segmentation and POS handling are set consistently word by word, as in 누가 → 누 + 가. The same word is always analyzed the same way, so results are stable.

Added robustness from spacing augmentation and out-of-vocabulary handling

Not shaken by informal input

Training with word spacing augmentation lets Bareun find morpheme boundaries even when users get word spacing wrong, and it guesses new words that are not in the dictionary from context. This robustness supports accuracy in real-world use.

All these elements together produce the accuracy above.

Frequently asked questions

Q. How accurate is Bareun's morphological analysis?

97.8% at the eojeol level and 98.5% morpheme F1. We measured the 3.4.0 production bundle on 2026-11-06 with 286 gold sentences (2,079 eojeol) not used in training.

Q. How is this accuracy possible?

It rests on three pillars: cleaned corpora, separating the segmentation stage from the tagging stage, and consistent in-house tagging guidelines. Spacing augmentation and context-based guessing of out-of-vocabulary words add robustness.

Q. How do eojeol accuracy and morpheme F1 differ?

Eojeol accuracy is the share of eojeol whose analysis matches the gold answer entirely. Morpheme F1 combines precision and recall at the morpheme level. One eojeol can contain several morphemes, so eojeol accuracy comes out lower.

Q. How was this measured?

We scored the tagger's raw output against human-reviewed gold sentences. The basis is 286 gold sentences (2,079 eojeol) not used in training. The regression gold of 1,595 sentences, which collects hard cases, scores 95.38% (morpheme F1 97.51%).