Built-in dictionaries in Bareun at a glance: types, roles, and priority

Bareun does not analyze Korean with a trained model alone. It supplements the proper nouns, technical terms, loanwords, Sino-Korean words, and non-standard words the model cannot handle with several built-in dictionaries, raising the accuracy of morphological analysis and spell checking together.

This page summarizes what each of Bareun's built-in dictionaries contains, what role it plays in analysis and correction, and how to trace which dictionary a morpheme came from. How each dictionary works in detail is covered in the individual pages that follow.

Built-in dictionaries at a glance

Dictionary file	Contents	Main role	Included in builds
`dict.res`	Learned default built-in vocabulary (token index)	Default vocabulary index the model refers to	All builds
`ext-dict.fb`	Surnames, enumeration expressions, unit-bearing dependent nouns, action nouns, wiki	Personal name and quantity recognition, compound noun splitting	All builds
`urimal-dict.fb`	Urimalsaem (Open Korean Dictionary, National Institute of Korean Language) vocabulary	Standard vs non-standard word detection, prefix/suffix checks	Spell checker build
`onterm-dict.fb`	Onterm (Korean terminology platform) technical term dictionary	Supplementing neologisms and technical terms	Spell checker build
`non-std-dict.fb`	Non-standard word → standard word mapping	Non-standard word correction	Spell checker build
`foreign-exam-dict.fb`	Loanword spelling usage examples	Loanword spelling consistency	Spell checker build
`hanja-dict.fb`	Chinese character readings and Sino-Korean word spellings	Hanja/Hangul conversion, Sino-Korean word lookup	Spell checker build
`word-freq.gob`	Corpus word frequencies	Ranking typo correction candidates	Spell checker build

What is the spell checker build?

Bareun's spell checking feature is provided in builds that include the spell checker. The default build that uses only morphological analysis contains dict.res and ext-dict.fb, and the correction dictionaries (urimal-dict.fb, onterm-dict.fb, non-std-dict.fb, and so on) are packaged together in builds that include the spell checker.

Dictionaries are downloaded on first start

The dictionaries above are not in the installer. When the server runs for the first time, it downloads the model and dictionary bundle that matches the installed version, stores it in the data area, and reuses it as is from the next run. .fb dictionaries are read through mmap (memory mapping), so startup does not slow down even when the files are large. To see what is currently in use, see Model update.

At which analysis stage are dictionaries used?

In the morphological analysis stage, dict.res and ext-dict.fb help with model inference and compound noun splitting. In the spell checking stage, the Urimalsaem, non-standard word, loanword, Chinese character, and frequency dictionaries are used in turn.

graph TD
  IN[Input sentence] --> SEG[Segmentation and POS tagging];
  SEG --> EXT[ext-dict.fb<br/>Compound noun splitting, names, quantities];
  SEG --> RES[dict.res<br/>Token index];
  SEG --> REV[Spell checking];
  REV --> URI[urimal-dict.fb<br/>Standard word detection];
  REV --> NSD[non-std-dict.fb<br/>Non-standard word correction];
  REV --> FOR[foreign-exam-dict.fb<br/>Loanword spelling];
  REV --> WF[word-freq.gob<br/>Typo candidate ranking];

Tracing where a morpheme came from: OutOfVocab

Each morpheme in a morphological analysis result carries an out_of_vocab field that tells you where it was recognized from. If it came from a custom dictionary, you can also tell which dictionary it was with custom_dict_name.

OutOfVocab value	Meaning	Related dictionary
`IN_WORD_EMBEDDING`	A word included in the word embeddings (training)	Training vocabulary
`OUT_OF_VOCAB`	Out-of-vocabulary word (automatic guess)	In no dictionary
`IN_CUSTOM_DICT`	A word registered in the custom dictionary	Custom dictionary
`IN_BUILTIN_DICT`	Default built-in dictionary	`dict.res`, etc.
`IN_URIMALSAEM`	Urimalsaem	`urimal-dict.fb`
`IN_WIKI_DICT`	Wikipedia dictionary	`ext-dict.fb` (KoWiki)
`IN_ON_DICT`	Onterm dictionary	`onterm-dict.fb`

Why show the source?

Even for the same word, how reliable it is depends on whether it was recognized from the trained vocabulary, recognized because a user registered it, or guessed as an out-of-vocabulary word. Bareun returns the source as well, so you can use it as evidence when reviewing analysis results or looking for places to strengthen your custom dictionary.

Frequently asked questions

Q. Are dictionaries included even if I only use morphological analysis?

Yes. The default build also includes dict.res (token index) and ext-dict.fb (surnames, quantities, compound noun splitting). The correction dictionaries are added in builds that include the spell checker.

Q. Can I tell which dictionary a word was recognized from?

You can tell the source from each morpheme's out_of_vocab value. The values differ: IN_URIMALSAEM for Urimalsaem, IN_ON_DICT for Onterm, and IN_WIKI_DICT for the wiki. For custom dictionaries, check IN_CUSTOM_DICT and custom_dict_name.

Q. What happens if a custom dictionary and a built-in dictionary conflict?

You can specify several dictionaries with custom_dict_names, and the dictionary specified first takes priority. For the detailed combination policy, see the dictionary priority documentation.

Q. How do the correction dictionaries differ from the morphological analysis dictionaries?

The default build contains only dict.res (token index) and ext-dict.fb (surnames, quantities, compound nouns), which are used for morphological analysis. The Urimalsaem, non-standard word, loanword, Chinese character, Onterm, and frequency dictionaries are all packaged only in builds that include the Korean spell checker. So source values such as IN_URIMALSAEM, IN_WIKI_DICT, and IN_ON_DICT do not appear when you use morphological analysis alone, and are shown only in builds that include the spell checking feature.

Q. In what form are dictionaries searched during spell checking?

The Korean spell checker also searches the dictionaries through an NFD index that decomposes them into phonemes (initial, medial, and final consonant/vowel units), so it can precisely find the set of words that fall under a phonological rule. The details of phoneme-based dictionary search are covered in the spelling rules documentation.