How is 99.6% accuracy possible: 3 secrets
Bareun (bareun) achieves 99.6% morpheme POS tagging accuracy and 99.7% eojeol (space-delimited word unit) splitting/restoration accuracy.
These figures are no accident. They come from three practices: corpus cleaning, separating the segmentation stage, and our own tagging guidelines. This article explains how Bareun reached this accuracy.
How accuracy is measured and the evidence behind it are covered in more detail in Accuracy and
How accuracy is measured.
Secret 1: We cleaned the corpus by hand
In deep learning, a model's accuracy depends on the quantity and quality of the training data. Bareun does not use public corpora as they are. We found and fixed errors one by one to clean them.
- The 21st Century Sejong Project morphological and semantic analysis corpus: 12 million eojeol (written and spoken)
- The National Institute of Korean Language (NIKL) Everyone's Corpus (Modu) morphological analysis corpus: 3 million eojeol (2 million written, 1 million spoken)
- News articles from 1992 to 2022: 100 million eojeol (BigKinds)
- A corpus we built ourselves, focused on supplementing low-frequency data
- An augmented corpus created by deliberately introducing word spacing errors
Finding and fixing errors
Developers with degrees in Korean language and literature, together with master's and doctoral-level researchers from major Korean universities, directly corrected errors that had not been found in the Sejong and Everyone's corpora, and reviewed analysis errors one by one to reinforce the training data.
Secret 2: We separated segmentation from POS tagging
Bareun does not produce the answer in one step. It analyzes in three stages: segmentation (Segmenter) → POS tagging (PosTagger) → post-processing.
Korean is an agglutinative language, so a single eojeol holds several morphemes that are attached after conjugation or contraction. If you first decide exactly "where to cut (segmentation)" and then attach "what part of speech each piece is (tagging)", each stage solves a simpler problem, and overall accuracy goes up.
- At the segmentation stage, text is first divided into eight major morphological units: substantives (N), particles (J), predicates (V), endings (E), adverbials (A), determiners (M), interjections (I), and symbols (S).
- In this process, 115 segmentation rules found by analyzing the characteristics of Korean conjugation are reflected in the model weights. Of these, 14 are error detection rules that the spell checker uses to catch morphological errors at the segmentation stage.
- At the tagging stage, each morpheme is assigned one of the 47 parts of speech defined by NIKL.
How accuracy was measured
The accuracy figures are the results of verification on large evaluation sets.
- Whole-sentence tagging accuracy: Measured strictly on 150,000 sentences from Everyone's Corpus, where a sentence counts as wrong if even one morpheme is wrong.
- Disambiguation (ambiguity resolution) evaluation set: We built a set of 35,396 sentences / 8,285 surface forms to tell apart different analyses of the same surface form.
- Training data reinforcement: To improve adaptability to spoken language, dialects, and new words, we added 100 million eojeol / 10 million sentences, including NIA (National Information Society Agency) speech and dialogue data.
Why segmentation is separate
For transformer models, the Korean tokenizer is always a thorny problem. Bareun tackles this head-on by having a segmentation stage dedicated to deciding "where to cut" before attaching parts of speech. That is why it accurately restores the original position of conjugated, contracted, or omitted morphemes.
Secret 3: We kept consistency with our own tagging guidelines
If the criteria for analyzing the same expression waver, accuracy drops. Bareun accepts NIKL's guidelines, but consistently revised tagging rules that did not fit our own criteria.
For example, instead of treating 누가 as 누구+가 and 뉘 as 누구+이,
we treat 누 as a pronoun and trained the model on 누+가 and 누+이.
Thanks to such consistent guidelines, Bareun analyzes reliably even in ambiguous cases where the same surface form has several meanings.
Bareun found 12,175 words that share a surface form but have different analyses, distinguishes them with morphological information embeddings, and
has published this ambiguity data.
For the composition of this data, see the Ambiguity dataset document.
Frequently asked questions
Q. What do 99.6% and 99.7% each refer to?
99.6% is the morpheme POS tagging accuracy, and 99.7% is the eojeol splitting/restoration accuracy.
Q. Does it analyze spoken language and new words well?
Yes. It uses a deep learning engine that understands the conjugation rules of agglutinative Korean, so it also analyzes colloquial endings correctly, such as
감사합니당(ㅂ니당/EF) and 알겠어용(어용/EF).
Q. How can I improve accuracy further?
Registering proper nouns, compound nouns, verbs, and adjectives in the custom dictionary improves the analysis accuracy of domain terms. The dictionary is updated with zero downtime.
Q. What data was accuracy measured on?
Whole-sentence tagging was evaluated on 150,000 sentences from Everyone's Corpus, measured strictly, counting a sentence as wrong if even one morpheme is wrong. Disambiguation was verified on an evaluation set of 35,396 sentences and 8,285 surface forms.
Q. How many segmentation rules are there?
It uses 115 segmentation rules found by analyzing the characteristics of Korean conjugation. Of these, 14 are error detection rules that the spell checker uses to catch morphological errors at the segmentation stage.