You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+21-7
Original file line number
Diff line number
Diff line change
@@ -1,11 +1,11 @@
1
1
go-mysql-elasticsearch is a service to sync your MySQL data into Elastissearch automatically.
2
2
3
-
It uses mysqldump to fetch data at first, then syncs data incrementally with binlog.
3
+
It uses `mysqldump` to fetch the origin data at first, then syncs data incrementally with binlog.
4
4
5
-
## Use
5
+
## How to use?
6
6
7
-
+ Create your MySQL table.
8
-
+ Create the associated Elasticsearch index, document type and mappings if possible, if not, go-mysql-elasticsearch will create index and type automatically with default mapping.
7
+
+ Create table in MySQL.
8
+
+ Create the associated Elasticsearch index, document type and mappings if possible, if not, Elasticsearch will create these automatically.
9
9
+ Set MySQL source in config file, see [Source](#Source) below.
10
10
+ Custom MySQL and Elasticsearch mapping rule in config file, see [Rule](#Rule) below.
11
11
+ Start `go-mysql-elasticsearch` and enjoy it.
@@ -15,9 +15,9 @@ It uses mysqldump to fetch data at first, then syncs data incrementally with bin
15
15
+ binlog format must be **row**.
16
16
+ binlog row image may be **full** for MySQL. (MariaDB only supports full row image).
17
17
+ Can not alter table format at runtime.
18
-
+ MySQL table which will be synced must have a PK(primary key), multi columns PK is not allowed. The PK data will be used as id in Elasticsearch.
18
+
+ MySQL table which will be synced must have a PK(primary key), multi columns PK is not allowed now. The PK data will be used as "id" in Elasticsearch.
19
19
+ You should create the associated mappings in Elasticsearch first, I don't think using the default mapping is a wise decision, you must know how to search accurately.
20
-
+`mysqldump` must exist in the same node with go-mysql-elasticsearch.
20
+
+`mysqldump` must exist in the same node with go-mysql-elasticsearch, if not, go-mysql-elasticsearch will try to sync binlog only.
21
21
+ Don't change too many rows at same time in one SQL.
22
22
23
23
## Source
@@ -60,7 +60,21 @@ type = "t"
60
60
61
61
In the example above, we will use a new index and type both named "t" instead of default "t1", and use "my_title" instead of field name "title".
62
62
63
+
## Why not other rivers?
64
+
65
+
Although there are some other MySQL rivers for Elasticsearch, like [elasticsearch-river-jdbc](https://github.com/jprante/elasticsearch-river-jdbc), [elasticsearch-river-mysql](https://github.com/scharron/elasticsearch-river-mysql), I still want to build a new one with Go, why?
66
+
67
+
+ Customization, I want to decide which table to be synced, the associated index and type name, or even the field name in Elasticsearch.
68
+
+ Incremental replication with binlog, and can resume from the last sync position when the service starts.
69
+
+ A common sync framework not only for Elasticsearch but also for others, like memcached, redis, etc...
70
+
63
71
## Todo
64
72
65
73
+ Wildcard table source support, like "table_%".
66
-
+ Filter table field support, only fields in filter config will be synced.
74
+
+ Filtering table field support, only fields in filter config will be synced.
75
+
76
+
## Feedback
77
+
78
+
go-mysql-elasticsearch is still in development, and we will try to use it in production later. Any feedback is very welcome.
0 commit comments