Building domain-specific custom dictionaries (legal, medical, finance)

In domains with many technical terms, such as law, medicine, and finance, it is much better for dictionary quality and maintenance to follow a fixed workflow of collect → classify → register → verify than to fill a custom dictionary ad hoc.

This document presents a four-stage workflow for building a domain term dictionary systematically, along with real term examples for each of the legal, medical, and finance domains.

Overall workflow

graph LR
  A[1. Collect] --> B[2. Classify];
  B --> C[3. Register];
  C --> D[4. Verify];
  D -- OOV words found --> A;

A domain dictionary is not built once and finished. It is a cycle in which the out-of-vocabulary words (OOV) found during verification are sent back to collection, so you refine it iteratively.

Stage 1 — Collect

Gather terms that are used often in your domain documents but are not in the general dictionary.

• Extract terms from a real corpus, such as internal documents, contracts, medical records, and terms of service.
• It is efficient to run morphological analysis and collect the out-of-vocabulary words (OOV) whose out_of_vocab is OUT_OF_VOCAB as candidates. (See the document on reducing OOV for details.)
• If you note next to each term how you want the word to be recognized, the next classification stage becomes easier.

Stage 2 — Classify

Decide which of the five dictionary types each collected term belongs in. This classification is the core of dictionary quality.

Stage 3 — Register

Group the classified terms under a domain name and register them at once. If the areas differ, split them into separate domains.

from bareunpy import Tagger

my_tagger = Tagger('YOUR-API-KEY', 'localhost', port=5656)

# Build the legal domain dictionary
law = my_tagger.custom_dict("legal")
law.copy_np_set({'민식이법', '김영란법'})
law.copy_cp_set({'압수수색', '약식기소'})
law.copy_cp_caret_set({'손해^배상', '위헌^심판'})
law.copy_vv_set({'기소하다', '항소하다'})

# Check for conflicts before registering, then apply
if not law.check_conflict():
    law.update()

Stage 4 — Verify

Check with real sentences that the registered dictionary works as intended.

• Analyze representative domain sentences and confirm that the registered terms come out as IN_CUSTOM_DICT with the correct part of speech (POS).
• Check that custom_dict_name is the domain you intended.
• If you still see words marked OUT_OF_VOCAB, send them back to Stage 1 to include them in the next cycle.

Frequently asked questions

Q. How many domains can I create?

Bareun lets you define an unlimited number of custom dictionaries (domains) on one server. If your lines of business differ, split them into separate domains, and combine the ones you need with custom_dict_names at analysis time.

Q. Can't I just put every technical term in as a compound noun?

The result depends on the part of speech you intend. Law names must go in the proper noun dictionary, and conjugating verbs in the verb dictionary to be recognized correctly. If you skip the classification stage, you will see mismatched results at verification.

Q. Does the service stop when I update a dictionary?

No. Bareun detects dictionary file changes in real time and applies them without downtime. You can safely update a domain dictionary even while the service is running.