Using Bareun segmentation to compress LLM prompts

Both LLM API cost and context limits depend on the number of tokens. Korean sentences contain many elements that take up tokens while preserving meaning, such as particles (josa), endings (eomi), and connective expressions. If you use Bareun's morphological analysis to pick out these functional morphemes and trim them, you can save prompt tokens without greatly harming meaning.

The problem

• When you put a long reference document into a prompt, you exceed the context limit.
• Repeated particles and endings lower token efficiency relative to the key information.
• If you simply cut characters, the sentence breaks and the LLM misunderstands.

How to apply Bareun

Keep the noun and predicate stems that carry the key information, and selectively remove the particles (J), some endings (E), and symbols (S), which lose little meaning. Because eojeol (space-delimited word units) are decomposed into these eight major morphological units, you can safely select by part of speech. It is safest to apply compression only to "reference context," and to leave the original instructions as they are when generating the final answer shown to the user.

graph TD
  DOC[Original context] --> TAG[Bareun morphological analysis];
  TAG --> KEEP[Keep noun and predicate stems];
  TAG --> DROP[Remove particles, some endings, symbols];
  KEEP --> COMP[Compressed keyword context];
  DROP --> COMP;
  COMP --> LLM[LLM prompt];

from bareunpy import Tagger

tagger = Tagger("koba-XXXX-...", "localhost")

# Functional morpheme POS tags that lose little meaning
DROP_TAGS = {"JKS", "JKC", "JKG", "JKO", "JKB", "JKV", "JKQ",
             "JX", "JC", "EF", "EC", "EP", "SF", "SP", "SE"}

def compress(text: str) -> str:
    """Compress the context by removing particles, some endings, and symbols."""
    res = tagger.tags([text])
    kept = [m for m, t in res.pos() if t not in DROP_TAGS]
    return " ".join(kept)

src = "인공지능 모델은 대규모 데이터를 학습하여 성능을 높입니다."
print("Original:", src)
print("Compressed:", compress(src))

Original: 인공지능 모델은 대규모 데이터를 학습하여 성능을 높입니다.
Compressed: 인공지능 모델 대규모 데이터 학습 성능 높이

The key nouns and predicates are all intact, and the particles and sentence-final endings are removed, so the token count drops.

Results and benefits

Frequently asked questions

Q. How many tokens does it save?

It depends on the document, but the more particles and endings an explanatory text has, the greater the saving. Measure the token counts before and after compression yourself with the tokenizer you actually use, and decide whether to apply it.

Q. Which parts of speech are safe to keep?

It is safe to keep nominals (NNG, NNP, and so on), predicate stems (VV, VA), and numerals and quantity expressions. We recommend leaving endings that change meaning, such as negation and tense, off the removal list.

Q. Does an LLM answer well with the compressed context?

LLMs also make good use of keyword-style context. However, for tasks where sentence structure matters (summarization, translation), apply compression lightly or provide the original text as well.

Related documents

• Morphological analysis API (AnalyzeSyntax): the analysis API that returns POS information
• POS tag table: criteria for deciding which parts of speech (particles, endings, symbols) to keep or remove
• Korean chunking for RAG: using morphemes to build search context
• Keyword extraction and text mining: building keywords centered on nominals

Q. Is there a way to reduce the token cost of Korean LLM prompts?

In Korean, particles and endings take up many tokens. If you use Bareun's morphological analysis to keep noun and predicate stems and remove the particles, some endings, and symbols that lose little meaning, you can reduce the tokens in your reference context and ease both API cost and context limit problems. Endings that change meaning, such as negation and tense, should be kept.