Predicate conjugation, irregulars, and form restoration: following the 킨데요 example
Korean predicates (verbs and adjectives) change shape endlessly as endings attach.
Syllables merge and consonants move over, as when 키인데요 contracts into 킨데요.
Bareun returns such altered surface forms to their original morphemes and even restores exactly which position
in the original text each morpheme came from. This article follows that process using 킨데요 as an example.
What is conjugation
A predicate conjugates as several endings attach to the stem one after another. In regular conjugation the stem stays as it is, while in irregular conjugation the stem and ending change shape.
| Type | Example | Restored result |
|---|---|---|
| Regular conjugation | 먹었다 | 먹/VV + 었/EP + 다/EF |
| Contraction | 봤어 | 보/VV + 았/EP + 어/EF |
| ㅡ elision | 써서 | 쓰/VV + 어서/EC |
| ㄷ irregular | 들어 | 듣/VV + 어/EC |
The type of conjugation is sorted out at the segmentation stage
What matters here is that distinguishing the type of conjugation, such as irregular conjugation, elision, contraction, and omission, is handled not in POS tagging but at the segmentation (Segmenter) stage. The 115 labels of the segmentation model are grouped into 8 classes: basic, irregular conjugation, elision, contraction, omission, conjugation-copula, other, and exception. The ㄷ irregular, ㅡ elision, and contraction in the table above are each sorted out as the segmentation label of the corresponding class is attached syllable by syllable. That is, the segmentation stage decides "where to split and how to return to the base form," and POS tagging attaches POS labels (VV, EP, EC, etc.) to the morphemes restored that way.
Errors are caught at the segmentation stage too
Of the 115 segmentation labels, 14 are for error detection, identifying morphological errors. Eojeols whose conjugation breaks the norms, such as wrong irregular conjugation, contraction, or vowel progressive assimilation errors, are marked as errors at this stage and passed on to the spell checker. Restoration and error detection happen together at the same segmentation stage.
Unpacking 킨데요
킨데요 is a colloquial form in which something like "키인데요" has been contracted.
Bareun restores it as follows.
킨데요 → 키/VV + 이 + ㄴ데/EC + 요/EF
graph TD IN[킨데요] --> SEG[Segmentation and restoration]; SEG --> A[키 VV]; SEG --> B[이]; SEG --> C[ㄴ데 EC]; SEG --> D[요 EF];
The surface form 킨 is the result of the stem 키 and the elements after it merging into one syllable.
After restoration, the number of morphemes increases and the character count becomes larger than that of the surface form.
This is where the position restoration problem arises.
ResetOffsets: returning to the original syllable positions
To show where the restored morphemes came from in the original text, the offsets have to be recalculated.
ResetOffsets() in post-processing does this job.
Morphemes from the same syllable get the same start position
In 킨데요, 키, 이, and ㄴ all came from the first character 킨 of the surface form.
So the offsets of the three morphemes are adjusted to point to the same start position.
This way, even if restoration increases the character count, the original syllables and the morphemes correspond exactly.
Offsets are calculated differently depending on the requested encoding type (UTF8/UTF16/UTF32). Even for the same sentence, the number of bytes or code units per character differs by encoding.
graph LR SURF["Surface form: 킨 / 데 / 요"] --> REST[ResetOffsets]; REST --> MAP["키·이·ㄴ → position of '킨'<br/>데 → position of '데'<br/>요 → position of '요'"];
Restoration and position at the same time
Bareun does not stop at returning forms to their base forms; it makes the restored morphemes point to the exact syllable positions in the original text. Thanks to this, follow-up work such as highlighting, replacement, and correction can tell exactly "where in the original text this morpheme falls."
Frequently asked questions
Q. How is 킨데요 analyzed?
It is restored as 키/VV + 이 + ㄴ데/EC + 요/EF. This is because the surface form 킨 is the stem 키 and the
elements after it merged into one syllable.
Q. When restoration increases the character count, how are positions aligned?
ResetOffsets in post-processing recalculates the offsets. Morphemes that came from the same syllable are adjusted to point to the
same start position, so the original syllables and the morphemes correspond exactly.
Q. Are irregular conjugations handled too?
Yes. It restores Korean conjugation, contraction, and elision such as ㄷ irregular (듣어→들어), ㅡ elision (쓰어서→써서), and contraction (보았어→봤어).
Q. At which stage are conjugation, irregulars, elision, and contraction distinguished?
They are distinguished at the segmentation (Segmenter) stage. The 115 labels of the segmentation model are grouped into 8 classes: basic, irregular conjugation, elision, contraction, omission, conjugation-copula, other, and exception, and as the right label is attached to each syllable, the type of conjugation is sorted out and the base form is restored. POS tagging attaches POS labels afterward to the restored morphemes.