Bareun vs Mecab-ko: accuracy, speed, and dictionaries

Both Bareun and Mecab-ko are widely used for Korean morphological analysis, but their approaches to the analysis are fundamentally different. Mecab-ko is a statistical analyzer that finds the optimal path using a dictionary and cost information, while Bareun is a neural analyzer trained as a transformer-based segmentation (Segmenter) → POS tagging (PosTagger) model. This difference shows up directly in how they handle out-of-vocabulary words and colloquial speech, how dictionaries are managed, and in accuracy.

Comparison at a glance

Dictionary and statistical search vs transformer training

Mecab-ko is a dictionary- and statistics-based analyzer that finds the lowest-cost analysis path using dictionary headwords and the cost of each path. Expressions in the dictionary are handled quickly and stably, but the range of analysis is bound to the dictionary and cost table.

Bareun replaces this with a trained neural network.

• Segmentation model: A transformer encoder that assigns a segmentation label to each syllable, using 115 labels including a whitespace handling rule (SP). Of these, 14 are for spell checking (error detection). Cases that look phonologically similar but differ in form (for example, an adnominal ending versus an auxiliary particle) are distinguished by label.
• Form-aware embeddings: Words with the same surface form but different forms are embedded as different tokens. For example, 가 is split into particle (가#J), predicate (가-#V), and nominal (가#N), and we found 12,175 words that need this kind of distinction. Unlike approaches that work only at the dictionary headword level, the model itself distinguishes the ambiguity of the same surface form.

In other words, even when handling the same Korean text, the most fundamental difference is that one looks up answers in a dictionary and cost table, while the other learns context from 115 segmentation labels and form-aware embeddings and infers from it. To go deeper into the difference between the two approaches, see Dictionary approach vs transformer and Form-aware embeddings.

Where they diverge on OOV words and colloquial speech

An analyzer that works around a dictionary, like Mecab-ko, tends to produce broken analyses when it meets colloquial or contracted forms such as "감사합니당" or "그치만", because those headwords are not in the dictionary. Bareun handled such expressions during training, and when it meets an OOV word, it also marks where each morpheme came from (trained embeddings, automatic inference, custom dictionary, and so on) with the out_of_vocab value. Because the source is returned along with the result, you can tell from the response whether a result is a guess or is backed by a dictionary. OOV handling is covered in more detail in OOV handling.

Differences in dictionary management

To add a custom dictionary in Mecab-ko, you have to edit the dictionary source, recompile it, and restart the analyzer. Bareun detects file changes in the custom dictionary directory in real time and applies them immediately without stopping the server when a file changes. You can specify several dictionaries at once with custom_dict_names, and the dictionary specified first takes priority.

Related documents

• Bareun vs Kiwi: comparison with another dictionary- and language-model-based analyzer
• How the 99.6% accuracy was measured: evaluation sets and metric definitions
• Custom dictionary overview: how to register and update dictionaries without downtime

Frequently asked questions

Q. Isn't Mecab-ko faster?

Dictionary- and cost-based search is light and fast. Bareun runs neural network inference, so single-request latency may differ, but you can raise throughput by choosing an executionProvider (CPU/GPU) and using batch processing. Speed depends heavily on the measurement environment, so we recommend comparing on the same input and hardware.

Q. Can I use a Mecab dictionary in Bareun as is?

The dictionary format and the analysis method are different, so you cannot carry it over as is. However, Bareun lets you register custom dictionary entries directly for proper nouns, compound nouns, verbs, adjectives, and more, and changes take effect without downtime.

Q. What is the fundamental difference in analysis method between Mecab and Bareun?

Mecab-ko is a dictionary- and statistics-based analyzer that finds the optimal path from dictionary headwords and cost information. Bareun analyzes by learning context with a transformer that assigns a segmentation label to each syllable (115 labels including the whitespace rule) and embeddings that distinguish the same surface form by form (12,175 words to distinguish). The biggest difference is that it is not bound to a dictionary and the model infers directly.

Q. When does Bareun have the advantage?

It has the advantage for text with many neologisms, colloquial speech, and contracted forms, for operating environments where the dictionary must be updated often, and where word spacing correction is also needed, because it infers OOV words from context and shows their source.