Content moderation and banned-word detection

Simple string matching has clear limits for detecting banned words. Users cleverly get around banned words with word spacing, particles, and jamo (letter component) separation. If you normalize the text with Bareun's morphological analysis, you can catch the banned-word root even in altered expressions.

The problem

• Users split a word with spaces, as in 욕 설 (a profanity split with a space), to get past banned-word filters.
• Conjugated or particle-attached forms (금칙어를, 금칙어처럼; "banned word" with a particle) are not caught by simple matching.
• Input with broken word spacing does not line up with a string dictionary.

How to apply Bareun

If you break the input into morphemes and normalize to noun and verb base forms, variations in particles, endings, and word spacing are removed. Comparing the normalized morphemes with a banned-word dictionary detects evasion attempts better. Bareun's word spacing correction also counters the splitting trick. For how to use the morphological analysis API, see Morphological analysis API.

graph TD
  IN[User input] --> SP[Word spacing correction];
  SP --> TAG[Bareun morphological analysis];
  TAG --> NORM[Normalize to noun and verb base forms];
  NORM --> MATCH[Compare with banned-word dictionary];
  MATCH -->|Match| BLOCK[Block or review];
  MATCH -->|No match| PASS[Pass];

from bareunpy import Tagger

tagger = Tagger("koba-XXXX-...", "localhost")

BANNED = {"금칙어", "비속어"}  # example banned-word roots ("banned word", "profanity")

def is_blocked(text: str) -> bool:
    """Compare morpheme base forms with the banned-word dictionary to detect even evasive expressions."""
    res = tagger.tags([text])
    morphs = set(res.morphs())   # set of normalized morphemes
    return bool(morphs & BANNED)

print(is_blocked("이건 금칙 어를 우회한 표현"))  # word spacing evasion
print(is_blocked("정상적인 문장입니다"))

True
False

Evasive expressions split with spaces are also detected as banned words after correction and normalization.

Results and benefits

Frequently asked questions

Q. What about evasion that mixes in jamo (ㄱ.ㅊ.ㅇ)?

For this kind of variation, it works well to first clean the text with preprocessing rules such as removing special characters and combining jamo, and then pass it to Bareun.

Q. Won't it flag normal expressions as banned words by mistake?

Comparing at the level of morpheme base forms reduces false positives from partial strings. For words with homonyms, add rules that look at the context.

Q. Our banned-word list changes often.

Manage the list of banned-word roots in your application. For neologism-like, service-specific expressions, register them in the custom dictionary (np_set) so they are recognized as morphemes, which makes operations easier. For registering new words, see Registering neologisms and proper nouns.

Q. How do I block banned-word filter evasion through word spacing and particles?

Many attempts avoid simple string matching by splitting a word with spaces, as in 욕 설, or by attaching a particle, as in 금칙어를. Bareun's word spacing correction undoes the splitting trick, and normalizing morphemes to noun and verb base forms removes particle, ending, and conjugation variations. Comparing the normalized morphemes with a banned-word dictionary lets you detect even these evasive expressions.

Related documents

• Morphological analysis API: how to extract morpheme base forms
• Keyword extraction: pulling out key terms with the same normalization technique
• Registering neologisms and proper nouns: getting new banned expressions recognized as morphemes