Non-standard word dictionary: how non-standard words map to standard words

Bareun ships with the non-standard word dictionary non-std-dict.fb to correct non-standard words such as dialect forms, colloquial speech, and wrongly fixed expressions into standard Korean. It pairs non-standard words with their corresponding standard words, and is the core resource used to replace them with standard words during spell checking.

This dictionary is included in builds that include the spell checking feature.

Non-standard to standard mapping

The non-standard word dictionary holds the correspondence "this word is corrected to that word." It is built using the non-standard word rule system of the National Institute of Korean Language's Standard Korean Language Dictionary, and also draws on Urimalsaem (Open Korean Dictionary) vocabulary as an additional basis.

The category of a corrected result is shown as STANDARD (standard word), telling you the correction was made according to standard language rules.

Automatic particle adjustment after correction

When a non-standard word is replaced with a standard word, the presence of a final consonant (batchim) may change, and the particle that follows can become awkward. Bareun automatically adjusts the following particle (josa) according to whether the corrected word has a final consonant, producing a natural sentence.

graph LR
  A["낌을"] --> B[Non-standard word correction];
  B --> C["낌새 + 을"];
  C --> D{Check final consonant};
  D --> E["낌새를"];

• 낌을 → the non-standard word 낌 is corrected to the standard word 낌새 → the particle 을 is adjusted to 를 to match 낌새, which has no final consonant → 낌새를
• It does not stop at swapping the word; it also fixes the particle so that the whole sentence reads naturally.

Frequently asked questions

Q. What is the basis for non-standard word correction?

The non-standard word dictionary is built using the non-standard word rule system of the National Institute of Korean Language's Standard Korean Language Dictionary, and the correspondence between non-standard and standard words is decided with Urimalsaem vocabulary as additional evidence. Correction results are shown with the revision category STANDARD.

Q. If a non-standard word is corrected, does the particle change too?

Yes. If the presence of a final consonant changes after replacing it with the standard word, the following particle is adjusted automatically. For example, 낌을 naturally becomes 낌새를, particle included.

Q. Aren't proper nouns corrected by mistake?

Eojeols analyzed as proper nouns (NNP) are filtered out with a probability threshold, so that personal names, business names, and the like are not over-corrected.

Q. Which builds include the non-standard word dictionary?

non-std-dict.fb is included in builds that include the spell checker. It is not in the basic build that uses only morphological analysis, and non-standard word correction is offered together with the correction feature.