Spell checker accuracy benchmark (Bareun 3.4.0, 2026-11)

This page reports the correction accuracy of the Bareun Korean spell checker, measured on a regression evaluation set we built ourselves. The top of this page shows Bareun's own latest scores, and the comparison with other checkers is in Comparison with other products below.

Bareun 3.4.0 overall results

Results by category

The numbers keep moving

Do not read these numbers as a fixed report card. Bareun's evaluation set keeps growing. As soon as we find an error case through reports and our own checks, we add it to the evaluation set, and we keep that sentence there even after fixing it. The added sentences are newly found error cases and normal sentences that check whether the places we already fixed still hold.

Two consequences follow.

• The denominator keeps getting bigger. It was 2,999 sentences in early September, 3,931 on September 22, and 19,013 on October 19, and 19,460 on November 5.
• Anything that breaks again after being fixed shows up immediately. A new rule breaking an old correction happens often in practice, and the numbers go down when it does.

Some error types are answered entirely correctly on the current evaluation set, but they go down again once we find and add new cases of that type. A value of 100% only means "all of these sentences were correct".

Measured by automated regression, not by hand

Growing the evaluation set and measuring accuracy happen in the same system. When a sentence is added, the automated regression test actually runs the server, compares the output with the expected answer, counts how many are correct, and produces the accuracy. No one copies results into a table by hand, so no counting mistakes creep in.

Every time an improvement goes in, the whole evaluation set runs again. So if a new rule breaks a sentence we had already fixed, it shows up right away. The basis for choosing what to fix and the proof that it is fixed come from the same set.

What we will do next

• Increase samples in small categories. The evaluation set now has 19,460 sentences, but some rows still hinge on a few sentences, such as 384 Sino-Korean sentences, 45 space-removal sentences, and 17 conjugation sentences.
• Measure side by side with the previous version on the same evaluation set. This time we could not re-measure 3.2.1 under the same conditions, so we could not include the change from the previous version.
• Define how to evaluate cases with several correction candidates. For now, both spellings count as correct only for sentences where the answer key lists the allowed spelling in advance. For places where either could be right depending on meaning, there is no standard yet for scoring "picked one of several candidates", so today we output them as places that need confirmation and count them as wrong.

Where we are weakest now

It is typos. At 79.9% on the full evaluation set, it is the lowest. In this evaluation set, typo sentences grew sharply from 179 to 1,392, and many of them were made by inserting keyboard typos into news sentences. For a typo caused by pressing the wrong key, you can tell which character was intended only by reading the meaning of the sentence. Wrongly changing it to another word in the dictionary is worse than leaving the original as it is, so when we are not sure, we choose not to fix it.

Next is loanwords at 92.0%. It is hard to tell wrongly spelled loanwords apart from person and place names that are not in the dictionary, and wrongly "fixing" a name as a misspelling is worse than leaving the original, so we correct conservatively here too. spelling (94.8%) and confusable words (95.1%) follow. Both have many places that depend on meaning, so they can only be decided by reading the sentence.

Comparison with other products (2026-09, Bareun 3.2.1)

This is the correction accuracy measured on the regression test set that Bareun built itself. All three checkers were scored on the same 2,970 sentences.

First, to be clear: Bareun still has many weaknesses. This dataset is a regression test set that Bareun made itself, so it may have selection bias, and each checker has a different correction policy and scope. With those limits in mind, we publish the results as they are, based on this dataset.

Overall results

By error category

The same 2,970 sentences, split by error category. Where each checker is strong or weak differs by category.

Bareun's score on the full evaluation set

The 2,970 sentences above are a subset matched to the sentences used in the competitor measurement, so that the three checkers can be compared under the same conditions. The evaluation set Bareun actually runs with every improvement is larger. We publish its full score as well.

Both denominators gave the same 92.7%. To more decimal places, it is 92.69% on the 2,970 sentences and 92.72% on the 3,931 sentences. This shows that the 2,970 sentences used for the comparison are not a set picked in our favor.

By category, the direction differs. Spelling is higher on the 2,970-sentence set (96.6% vs 94.7%), while standard language and typos are higher on the full set (93.6% and 77.1% vs 92.9% and 75.4%). This is because the sentences added after the competitor measurement include not only cases we cannot fix yet, but also sentences that check whether the places we already fixed still hold.