title: Elasticsearch plugin
Plugging the Bareun morphological analyzer into Elasticsearch
You can use Bareun directly as an Elasticsearch analyzer. The same analyzer runs at indexing time and at search time, so operation is simpler than creating tokens in your application beforehand.
- Repository: bareun-es-plugin
- Current version: 1.0.0
Supported versions
Elasticsearch loads a plugin only when the plugin's elasticsearch.version is exactly the same as the node version.
That is why there is a separate file for each target version.
| Elasticsearch | File |
|---|---|
| 8.19.21 | elasticsearch-analysis-bareun-1.0.0-es8.19.21.zip |
| 9.5.2 | elasticsearch-analysis-bareun-1.0.0-es9.5.2.zip |
If you need another version, get the repository and build it yourself with mvn -Delasticsearch.version=9.4.6 package.
8.19 needs JDK 17, and 9.x needs JDK 21.
It must match the node version exactly
If you install the file for 9.5.2 on a 9.5.1 node, the node will not start. When you upgrade the version, change the plugin along with it.
Installation
Download the file that matches your node version from Releases and install it. You must install it on every node.
bin/elasticsearch-plugin install file:///path/to/elasticsearch-analysis-bareun-1.0.0-es8.19.21.zip
Configuration
Put node-level defaults in elasticsearch.yml and the API key in the keystore.
# elasticsearch.yml
bareun.host: nlp.bareun.ai
bareun.port: 5656
bareun.tls: false
bareun.timeout: 10s
bareun.stoptags: [E, IC, J, MAG, MAJ, MM, NA, NF, NV, SE, SF, SO, SP, SS, SW, VC, VX, XPN, XS]
bin/elasticsearch-keystore add bareun.api_key
Restart the node after changing the settings.
The API key cannot be put in index settings
Index settings are stored in plain text in the cluster state. The key is accepted only through the keystore.
Usage
This is the simplest form.
PUT /my-index
{
"settings": {
"analysis": {
"analyzer": {
"korean": { "type": "bareun" }
}
}
}
}
POST /my-index/_analyze
{ "analyzer": "korean", "text": "아버지가 방에 들어가신다." }
아버지/NNG 방/NNG 들어가/VV 들어가신다./VV
Attach it to a field.
PUT /my-index/_mapping
{
"properties": {
"content": { "type": "text", "analyzer": "korean" }
}
}
Registered names
| Kind | Name | Old name |
|---|---|---|
| analyzer | bareun | baikal_analyzer |
| tokenizer | bareun_tokenizer | baikal_tokenizer |
| token filter | bareun_part_of_speech | baikal_token |
The old names still work. They are kept for a while so that indexes that were already created do not break.
Use bareun_* for new indexes.
Options
You can override the node defaults in the analyzer settings.
PUT /my-index
{
"settings": {
"analysis": {
"tokenizer": {
"my_bareun": {
"type": "bareun_tokenizer",
"decompound_mode": "mixed",
"stoptags": ["E", "J", "SF"],
"custom_dict_names": ["mydict"]
}
},
"analyzer": {
"korean": { "type": "custom", "tokenizer": "my_bareun" }
}
}
}
}
| Option | Meaning | Default |
|---|---|---|
decompound_mode | none, discard, or mixed | mixed |
stoptags | POS tag prefixes to exclude from the index | Node setting |
custom_dict_names | Names of the custom dictionaries to use | Node setting |
host, port, tls, timeout | Server connection | Node setting |
decompound_mode
This shows the difference when you input "아버지가 방에 들어가신다.".
| Mode | Result |
|---|---|
mixed (default) | 아버지 방 들어가 들어가신다. |
discard | 아버지 방 들어가 |
none | 아버지가 방에 들어가신다. |
mixed puts a predicate eojeol (a space-delimited word unit) at the same position as its base form.
This is so that both users who search for 들어가다 and users who search for 들어가신다 get a match.
stoptags
Tags are compared by prefix. A single J removes all particles (josa) such as JKS, JKB, and JX.
For POS tags, see the POS tag table.
POS filter
Use this when you want to filter out different parts of speech in the indexing analyzer and the search analyzer.
"filter": {
"my_pos": { "type": "bareun_part_of_speech", "stoptags": ["J", "E"] }
},
"analyzer": {
"korean": { "type": "custom", "tokenizer": "bareun_tokenizer", "filter": ["my_pos"] }
}
Custom dictionary
It uses the custom dictionary of the Bareun server as is. You do not need to restart the Elasticsearch node after editing the dictionary. This is a difference from nori.
Comparison with nori
| nori | Bareun plugin | |
|---|---|---|
| Where analysis runs | Inside the node | Requests go to the Bareun server |
| Dictionary updates | Node restart required | Applied immediately on the server |
| Neologisms and proper nouns | You manage the custom dictionary yourself | Bareun model + custom dictionary |
| Word spacing correction | None | Handled by the server |
| External dependency | None | The Bareun server must be running |
The Bareun plugin calls the server for every analysis. If the server goes down, indexing and search both fail, so it is safer to make the server redundant or put a load balancer in front of it.
In Docker
examples/docker/ has an example of building an image with the plugin preinstalled.
Pass the settings as environment variables, and put the key into the keystore after the container starts.
docker run -d --name es \
-e discovery.type=single-node \
-e bareun.host=nlp.bareun.ai -e bareun.port=5656 \
-p 9200:9200 elasticsearch-bareun:8.19.21
docker exec -i es bash -c 'echo "koba-..." | bin/elasticsearch-keystore add -x -f bareun.api_key'
docker restart es
Frequently asked questions
Q. The node does not start. A. Check that the plugin file's target version and the node version are exactly the same. If they differ, Elasticsearch rejects the plugin.
Q. The analysis result is empty or I get an error. A. Check that the Bareun server is reachable and that the API key is in the keystore. After changing settings, you must restart the node for them to take effect.
Q. I have an index created with the old baikal_analyzer. Can I leave it as is?
A. Yes. The old names are kept as aliases. Use bareun starting with new indexes.
Q. Does it work with Elasticsearch 7.x? A. It is not supported. The target is 8.19 and 9.x.