Skip to content

Commit ec90dee

Browse files
committed
update doc
1 parent c9307a2 commit ec90dee

File tree

2 files changed

+24
-9
lines changed

2 files changed

+24
-9
lines changed

README.md

+21-7
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
go-mysql-elasticsearch is a service to sync your MySQL data into Elastissearch automatically.
22

3-
It uses mysqldump to fetch data at first, then syncs data incrementally with binlog.
3+
It uses `mysqldump` to fetch the origin data at first, then syncs data incrementally with binlog.
44

5-
## Use
5+
## How to use?
66

7-
+ Create your MySQL table.
8-
+ Create the associated Elasticsearch index, document type and mappings if possible, if not, go-mysql-elasticsearch will create index and type automatically with default mapping.
7+
+ Create table in MySQL.
8+
+ Create the associated Elasticsearch index, document type and mappings if possible, if not, Elasticsearch will create these automatically.
99
+ Set MySQL source in config file, see [Source](#Source) below.
1010
+ Custom MySQL and Elasticsearch mapping rule in config file, see [Rule](#Rule) below.
1111
+ Start `go-mysql-elasticsearch` and enjoy it.
@@ -15,9 +15,9 @@ It uses mysqldump to fetch data at first, then syncs data incrementally with bin
1515
+ binlog format must be **row**.
1616
+ binlog row image may be **full** for MySQL. (MariaDB only supports full row image).
1717
+ Can not alter table format at runtime.
18-
+ MySQL table which will be synced must have a PK(primary key), multi columns PK is not allowed. The PK data will be used as id in Elasticsearch.
18+
+ MySQL table which will be synced must have a PK(primary key), multi columns PK is not allowed now. The PK data will be used as "id" in Elasticsearch.
1919
+ You should create the associated mappings in Elasticsearch first, I don't think using the default mapping is a wise decision, you must know how to search accurately.
20-
+ `mysqldump` must exist in the same node with go-mysql-elasticsearch.
20+
+ `mysqldump` must exist in the same node with go-mysql-elasticsearch, if not, go-mysql-elasticsearch will try to sync binlog only.
2121
+ Don't change too many rows at same time in one SQL.
2222

2323
## Source
@@ -60,7 +60,21 @@ type = "t"
6060

6161
In the example above, we will use a new index and type both named "t" instead of default "t1", and use "my_title" instead of field name "title".
6262

63+
## Why not other rivers?
64+
65+
Although there are some other MySQL rivers for Elasticsearch, like [elasticsearch-river-jdbc](https://github.com/jprante/elasticsearch-river-jdbc), [elasticsearch-river-mysql](https://github.com/scharron/elasticsearch-river-mysql), I still want to build a new one with Go, why?
66+
67+
+ Customization, I want to decide which table to be synced, the associated index and type name, or even the field name in Elasticsearch.
68+
+ Incremental replication with binlog, and can resume from the last sync position when the service starts.
69+
+ A common sync framework not only for Elasticsearch but also for others, like memcached, redis, etc...
70+
6371
## Todo
6472

6573
+ Wildcard table source support, like "table_%".
66-
+ Filter table field support, only fields in filter config will be synced.
74+
+ Filtering table field support, only fields in filter config will be synced.
75+
76+
## Feedback
77+
78+
go-mysql-elasticsearch is still in development, and we will try to use it in production later. Any feedback is very welcome.
79+
80+

etc/river.toml

+3-2
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
# MySQL address, user and password
2+
# user must have replication privilege in MySQL.
23
my_addr = "127.0.0.1:3306"
34
my_user = "root"
45
my_pass = ""
@@ -22,7 +23,7 @@ mysqldump = "mysqldump"
2223
[[source]]
2324
schema = "test"
2425

25-
# Only below tables will be synced to Elasticsearch.
26+
# Only below tables will be synced into Elasticsearch.
2627
# I don't think it is necessary to sync all tables in a database.
2728
tables = ["test_river"]
2829

@@ -33,7 +34,7 @@ table = "test_river"
3334
index = "river"
3435
type = "river"
3536

36-
# title is MySQL es_test field name, es_tiel is your customized name in Elasticsearch
37+
# title is MySQL test_river field name, es_title is the customized name in Elasticsearch
3738
[rule.field]
3839
title = "es_title"
3940

0 commit comments

Comments
 (0)