Finding the cause of rising memory usage
Is the Bareun server's memory usage higher than you expected, or does it seem to use a lot of memory right after startup?
Most of the time it is due to model and dictionary loading and the data structures used for correction. Let's pinpoint where the memory goes.
Symptoms
- Memory usage rises sharply right after the server starts.
- A build that includes the spell check feature uses more memory than a morphological-analysis-only build.
- During operation, memory stays at a steady level or grows slowly because of caches.
Cause
Bareun uses memory in three main places.
| Part | Contents |
|---|---|
| Models | Loading seg-model.onnx (segmentation), slot-tag-model.onnx (POS tagging linked to segmentation), and tag-model.onnx (POS tagging, which takes over when the segmentation-linked POS model cannot be used) |
| Correction dictionaries (spell check build) | Data that goes on the heap, such as word-freq.gob and the BK-tree |
| Caches | Cache of LLM eojeol and word spacing correction results (spell check build) |
In particular, the spell check build also loads word frequencies (word-freq.gob) and the BK-tree for similar-word search, so it uses more memory than the morphological-analysis-only build.
.fb dictionaries do not use the heap
ext-dict.fb, hanja-dict.fb, foreign-exam-dict.fb, non-std-dict.fb, urimal-dict.fb, and
onterm-dict.fb are FlatBuffer files read through mmap. There is no step that reads the whole file to build a data structure, so they do not show up in heap usage; they are counted in the operating system page cache, so several containers on the same host share the same file. In contrast, word-freq.gob and the BK-tree cache are loaded onto the heap and are reflected directly in memory usage.
Solution
-
Check the build type. If you only need morphological analysis, a build without the spell check feature uses less memory, because it does not load the full set of correction dictionaries.
-
Check what is loaded using the startup log. If you identify from the log which models and dictionaries were loaded, it is easy to find items that are larger than expected.
-
Take the LLM cache into account (spell check build). Eojeol (space-delimited word unit) correction and word spacing decision results accumulate in the cache. There is a default maximum size, so it does not grow without limit, but keep in mind that the cache also takes memory.
-
If you use a container, set a generous memory limit. If you set the limit too low, it can be killed by OOM during loading. For details, see Docker container OOM kill.
word-freq.gob is heavy data
The word frequency data (word-freq.gob) is a large data set used to extract similar words by frequency rank.
It is one of the main items that takes a lot of memory in the correction build, so factor it in when estimating memory.
Prevention and tips
- Memory rising sharply once right after startup and then stabilizing is normal behavior from loading models and dictionaries into memory ahead of time. It is designed not to read them again for every request.
- If you deploy the morphological-analysis-only build and the correction build separately to suit your operating environment, you can save resources.
Frequently asked questions
Q. Memory jumps right after startup. Is that normal?
Yes. Bareun loads the models (seg-model.onnx, slot-tag-model.onnx, tag-model.onnx) and dictionaries into memory ahead of time at startup. This is designed so they are not read again for every request, and it usually stabilizes after rising once.
Q. Why does the spell check build use more memory?
Because the spell check build additionally loads word frequencies (word-freq.gob) and the BK-tree for similar-word search.
The correction dictionaries themselves are mapped with mmap and do not count toward the heap. If you only need morphological analysis, a build without the spell check feature uses less memory.
Q. How can I reduce memory?
If you only need morphological analysis, run a build without the spell check feature. Because it does not load the full set of correction dictionaries, you can save a lot of memory. Another approach is to check from the startup log which models and dictionaries are loaded to find items larger than expected, and to deploy the morphological-analysis-only build and the spell check build separately for your operating environment.
Q. Memory grows slowly during operation. Is that normal?
It can grow as the LLM eojeol and word spacing correction result cache (spell check build) accumulates. The cache has a default maximum size, so it does not grow without limit. But if you set the container memory limit too low, it can be killed by OOM during loading or operation, so for details see Docker container OOM kill.