revise-ai categories: classifying non-standard words, typos, and word spacing
revise-ai is the component responsible for Bareun's LLM-based correction and confusable word correction.
After the LLM corrects an eojeol (space-delimited word unit), it also returns what kind of correction it was, and Bareun maps this kind to a revision category (RevisionCategory).
This article explains how the category in the LLM response is classified and mapped, and how confusable word correction works.
LLM response category → RevisionCategory mapping
When the LLM fallback correction fixes an eojeol, it also marks the kind of correction.
This category string is converted into Bareun's internal RevisionCategory enum and included in the response.
| LLM response category | RevisionCategory | Meaning | Example |
|---|---|---|---|
NONSTANDARD | STANDARD | Non-standard word → standard word | 글장이 → 글쟁이 |
TYPO | TYPO | Typo | 스빈다 → 습니다 |
SPACING | SPACING | Word spacing (splitting and joining) | 그다음 → 그 다음 |
Why map the category separately?
The LLM describes "what it corrected and how" as a free-form string. If you map this to Bareun's fixed category system (RevisionCategory),
you can show users the results of rule-based correction and LLM correction consistently in the same classification system.
For example, a non-standard word correction is unified as STANDARD whether it was done by a rule or by the LLM.
graph LR
A[Eojeol LLM fallback correction] --> B[LLM response: corrected text + category];
B --> C{Classify category};
C -- NONSTANDARD --> D[STANDARD];
C -- TYPO --> E[TYPO];
C -- SPACING --> F[SPACING];
D --> G[Attach category to the correction result];
E --> G;
F --> G;
Confusable word correction
Most easily confused words come from similar pronunciation. For word pairs such as 반드시 and 반듯이, which sound alike but differ in meaning, you cannot tell which is right from spelling alone, and you can pick the right one only by looking at the contextual meaning of the sentence.
That is why Bareun leaves this judgment to an LLM.
For example, in the sentence 그는 걸을 때도 반드시 자세를 유지한다., the candidates are put to the LLM as follows.
// Query: which word is correct at the 4th position of the sentence
{ "offset": 4, "options": ["반드시", "반듯이"] }
// Response: the context is keeping the posture straight → picks the second candidate
{ "select": 1, "token": "반듯이" }
Word pairs that are confusing because their pronunciation or spelling is similar are captured as candidates by a separate confusable word dictionary, and the final choice is made by context as shown above. Confusable word correction chooses candidates as follows.
- Longest match: Among the confusable word expressions registered in the dictionary, it gives priority to the candidate that matches the longest stretch of the sentence.
- Non-overlapping selection: It picks candidates so that the chosen ones do not overlap in span, which prevents a single character from being tied to two corrections at once.
You can turn off confusable word correction with the disable_confusion option in RevisionConfig (default false).
{
"config": { "disable_confusion": true }
}
Frequently asked questions
Q. How is the kind of correction returned by the LLM classified?
The category in the LLM response is mapped to Bareun's RevisionCategory.
NONSTANDARD is converted to STANDARD, TYPO to TYPO, and SPACING to SPACING, and the category is attached to the correction result.
Q. How does confusable word correction choose candidates?
It gives priority to the candidate that matches the longest stretch in the confusable word dictionary (longest match), and picks candidates so that the chosen ones do not overlap in span (non-overlapping) to prevent conflicts.
Q. How does it tell apart words with similar pronunciation, such as 반드시 and 반듯이?
Most confusable words come from similar pronunciation, so the right answer cannot be known from spelling alone. Bareun gives the LLM the context to decide.
For example, if you query the candidates as {offset:4, options:["반드시","반듯이"]}, it reads the context of keeping the posture straight and
picks the appropriate word, as in {select:1, token:"반듯이"}.
Q. Can I turn off confusable word correction?
Yes. If you set disable_confusion in RevisionConfig to true, correction by the confusion (confusable words) model is disabled.
The default is false.