When responses are slow: 5 things to check (GPU, execution provider, input length)
Are your analysis or correction responses slow? The cause narrows down to a few places: the execution provider, input length, whether an LLM is used, and so on. Let's check them one by one.
Symptoms
- Morphological analysis or spell correction responses take a long time.
- Some requests are fast, but certain inputs are slow.
- You set up a GPU, but it is not as fast as you expected.
Causes
Here are the factors that affect Bareun's processing speed.
| Factor | Description |
|---|---|
| Execution provider | executionProvider (cpu/gpu) and whether it fell back |
| Input length | Segmentation processes text in 160-character units, and long sentences are split |
| Number of sentences | The number of sentences and eojeols (space-delimited word units) in one request |
| LLM use (spell-check build) | LLM fallback and word-spacing decision calls during correction |
How to fix it: 5 things to check
-
Check the execution provider. In a GPU environment, check that
executionProviderisgpuand that it actually runs on the GPU. If it fell back to CPU, you get no acceleration (see GPU fallback). -
Check the input length. Segmentation (the Segmenter) processes 160 characters at a time and splits longer sentences before processing them. An extremely long input increases the amount of splitting and can get slow. If you cut it to a suitable length before sending, responses get faster.
-
Adjust the number of sentences per request. If you send too many sentences at once, the processing time adds up. For bulk processing, send them in suitably sized chunks.
-
See whether sentences that need contextual judgment are mixed in (spell-check build). Spell checking hands eojeols that cannot be decided by spelling alone to the AI decision step. That step goes through an external service, so responses can get longer. For long documents, send them paragraph by paragraph, and if the delay keeps recurring, contact us.
-
Use the cache for repeated input (spell-check build). The same eojeol and word-spacing decisions are cached, which reduces repeat calls. The more repetition your workload has, the bigger the effect.
Quick diagnosis order
Looking in the order GPU fallback → input length → number of sentences → (for correction) sentences that hit the AI decision step lets you narrow down the cause quickly.
Prevention and tips
- Morphological analysis loads the models and dictionaries ahead of time at startup, so later requests are consistently fast compared with the first.
- Sending input in natural sentence units reduces splitting overhead.
Frequently asked questions
Q. What do I look at first when responses are slow?
The execution provider. Check whether it fell back to CPU even though you set up a GPU. Then check the input length (segmentation processes 160 characters at a time), the number of sentences per request, and, for correction, whether an LLM is used.
Q. Why does a long input make it slower?
Because segmentation processes 160 characters at a time and splits longer sentences. When the input is very long, the amount of splitting increases and it takes more time. If you cut it to a suitable length before sending, it gets faster.
Q. Why is spell checking slower than morphological analysis?
Correction runs rule processing and then falls back to an LLM for hard eojeols, and the delay of the external API response is passed on directly. You can improve this with the concurrent request count, timeout settings, and use of the cache.
Q. I set up a GPU, but it is not fast.
Even if you set up a GPU, if the execution provider fails to initialize, it falls back to CPU and you get no acceleration. Check that the startup log has no fallback warning and that GPU utilization goes up. For detailed diagnosis, see the GPU fallback document.
Q. Only the first request is slow and later ones are fast. Is that normal?
That is normal. Morphological analysis loads the models and dictionaries into memory at startup, so later requests are consistently fast compared with the first. In builds that include the spell checking feature, thanks to the eojeol and word-spacing decision cache, repeated input gets even faster.