Applying ASR post-processing
Speech recognition (ASR) output is text that has almost no word spacing or is scrambled by colloquial speech.
In that state it is hard to use for search, analysis, or subtitle display. Bareun is trained on a spoken-language corpus,
so it adapts well to speech, and its word spacing correction turns ASR output into readable sentences.
The problem
- ASR output has no word spacing, like
오늘날씨가어떻게되나요("What's the weather like today?"). - Expressions and clipped forms typical of speech get misanalyzed by general analyzers.
- Without word spacing, both morphological analysis and search go wrong.
How to apply Bareun
The word spacing correction (auto_spacing) option of morphological analysis sets eojeol (a space-delimited word unit) boundaries automatically.
Bareun includes spoken data such as the Modu corpus (Everyone's Corpus) in its training, so it is also strong on spoken sentences (for a detailed comparison, see
Adaptation to colloquial speech). Use the corrected sentence (refined) in later processing.
graph TD ASR[Speech recognition output] --> SP[Bareun word spacing correction auto_spacing]; SP --> REFINED[Corrected sentence refined]; REFINED --> TAG[Morphological analysis]; TAG --> USE[Subtitles, search, analysis];
from bareunpy import Tagger
tagger = Tagger("koba-XXXX-...", "localhost")
# ASR output with no word spacing
asr = "오늘날씨가어떻게되나요"
res = tagger.tags([asr])
# Corrected sentence and morphemes
print("morphemes:", res.morphs())
morphemes: ['오늘', '날씨', '가', '어떻', '게', '되', '나요']
The input that was stuck together is split into meaningful units and analyzed.
Results and benefits
| Item | Raw ASR output | After Bareun post-processing |
|---|---|---|
| Word spacing | None | Corrected with auto_spacing |
| Spoken expressions | Often misanalyzed | Strong thanks to spoken corpus training |
| Usability of the result | Hard to use | Ready for search and subtitles |
If eojeol position matters
If position consistency matters, as with subtitle sync, use the AnalyzeSyntaxList family, which preserves input order.
Check the position of each morpheme with the offsets in TextSpan.
Frequently asked questions
Q. Is a long sentence with no word spacing at all corrected too?
Yes. Segmentation processes text in 160-character units, so long input is corrected too. However, if you split a very long block into sentences before sending it, the results are more stable.
Q. What if I need to fix spelling too?
If you need to fix typos and standard-language usage as well as word spacing, use the Spell check API (CorrectError) together.
Some homophone errors typical of ASR are also corrected in the correction stage.
Q. Does it analyze clipped spoken forms well?
Bareun includes a spoken corpus in its training, so it is relatively strong on spoken sentences. If you have many dialect words or new coinages, supplement your custom dictionary with registering neologisms and proper nouns.
Q. Can I use STT output directly for morphological analysis and search?
Raw STT output has almost no word spacing, so it is hard to use for analysis or search as is. If you set eojeol boundaries with Bareun's word spacing correction
(auto_spacing), create the refined sentence, and then pass it to morphological analysis, you can use it
directly for search, subtitles, and analysis. If position consistency matters, check offsets with the AnalyzeSyntaxList family.
Related documents
- Adaptation to colloquial speech — Why spoken corpus training makes it strong
- Getting started with spell check — Polishing typos and standard-language usage as well
- Registering neologisms and proper nouns — Supplementing dialect words and neologisms