es：5.6.16

一、索引模板

1、创建索引模板

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
PUT http://192.168.x.x:9201/_template/shop_template
{
    "template": "access-log-*",
    "settings": {
        "number_of_shards": 3,
        "index.number_of_replicas": 1,
        "index.refresh_interval": "1s",
        "analysis": {
            "analyzer": {
                "default": {
                    "type": "ik_max_word"
                }
            }
        }
    },
    "mappings": {
        "access_log": {
            "dynamic_templates": [
                {
                    "strings_as_keywords": {
                        "match_mapping_type": "*",
                        "mapping": {
                            "type": "keyword",
                            "norms": false
                        }
                    }
                }
            ],
            "_all": {
                "enabled": false
            },
            "properties": {
                "elapsed": {
                    "index": false,
                    "type": "long"
                },
                "invoked_time": {
                    "format": "yyyy-MM-dd HH:mm:ss",
                    "type": "date"
                },
                "endpoint": {
                    "index": false,
                    "type": "text"
                },
                "clientid": {
                    "type": "keyword"
                },
                "action": {
                    "index": false,
                    "type": "keyword"
                },
                "return_code": {
                    "index": false,
                    "type": "keyword"
                }
            }
        }
    }
}

可参考：https://www.cnblogs.com/shoufeng/p/10641560.html

二、索引别名

1、

可参考：https://www.cnblogs.com/libin2015/p/10649189.html

三、索引操作

1、单一字段精准查询（查询字段是keyword）：

1
2
3
4
5
6
7
8
POST http://192.168.x.x:9201/access_log/_search
{
  "query": {
    "term": {
      "clientid": "5ed61748580332e0df085aa0"
    }
  }
}

2、时间范围查询：

1
2
3
4
5
6
7
8
9
10
11
POST http://192.168.x.x:9201/access_log/_search
{
    "query": {
        "range": {
            "invoked_time": {
                "gte": "2020-06-16 09:51:15",   // 大于等于
                "lt": "2020-06-17 09:53:15"   // 小于
            }
        }
    }
}

参考资料：

https://www.cnblogs.com/shoufeng/p/11266136.html

https://www.elastic.co/guide/en/elasticsearch/reference/5.6/query-dsl-range-query.html

3、条件删除数据

https://www.elastic.co/guide/en/elasticsearch/reference/6.0/docs-delete-by-query.html

1
2
3
4
5
curl -H 'Content-type: application/json' -XPOST 'http://es_ip:9201/events-hiostory/_delete_by_query' -d '{
 "query": {
  "match_all": {}
 }
}'
1
2
3
4
5
6
7
8
9
10
curl -H 'Content-type: application/json' -XPOST 'http://54.19.129.3:9200/events-history-system-2021111722/_delete_by_query' -d '{
 "query": {
  "range": {
   "Time": {
    "gte": "1638201600000000000",
    "lt": "1638288000000000000"
   }
  }
 }
}'

4、查看索引 mapping 结构

1
curl -XGET "http://x.x.x.x:9200/test-index/_mapping"

5、查看索引列表,并根据索引大小倒序排序

1
curl -XGET "http://172.26.x.x:9200/_cat/indices?format=json&index=*&s=store.size:desc,health,index,pri,rep,docs.count,mt"

参考资料：https://www.elastic.co/guide/en/elasticsearch/reference/5.6/cat-indices.html#cat-indices

6、Filter 中 term 和 range 、search_after 同时使用（多条件查询）

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
GET /_search
{
  "query": { 
    "bool": { 
      "must": [
        { "match": { "title":   "Search"        }},
        { "match": { "content": "Elasticsearch" }}
      ],
      "filter": [ 
        { "term":  { "status": "published" }},
        { "range": { "publish_date": { "gte": "2015-01-01" }}}
      ]
    }
  }
}

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-filter-context.html

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
# 多条件组合
# select * from xxx-index where start < endTime and (end >= startTime or end = 0)
{
  "size": 100,
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "start": {
              "lt": 1627401600000
            }
          }
        },
        {
          "bool": {
            "should": [
              {
                "range": {
                  "end": {
                    "gte": 1627315200000
                  }
                }
              },
              {
                "term": {
                  "end": 0
                }
              }
            ]
          }
        }
      ]
    }
  },
  "search_after": ["meta#test-index-2021052619", 1622026800000],
  "sort": [
    {
      "_uid": "desc",
      "start": "asc"
    }
  ]
}

查询 xxx-index，获取要查询的 system 表

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
{
  "from" : 0,  # 第0条开始查
  "size" : 200,  # 显示200条
  "query" : {
    "bool" : {
      "filter" : [
        {
          "term" : {
            "db" : {
              "value" : "system",
              "boost" : 1.0
            }
          }
        },
        {
          "bool" : {
            "should" : [
              {
                "bool" : {
                  "filter" : [
                    {
                      "range" : {
                        "start" : {
                          "from" : null,
                          "to" : 1631097122621,
                          "include_lower" : true,
                          "include_upper" : false,
                          "boost" : 1.0
                        }
                      }
                    },
                    {
                      "bool" : {
                        "should" : [
                          {
                            "range" : {
                              "end" : {
                                "from" : null,
                                "to" : 0,
                                "include_lower" : true,
                                "include_upper" : true,
                                "boost" : 1.0
                              }
                            }
                          },
                          {
                            "range" : {
                              "end" : {
                                "from" : 1631097122621,
                                "to" : null,
                                "include_lower" : false,
                                "include_upper" : true,
                                "boost" : 1.0
                              }
                            }
                          }
                        ],
                        "disable_coord" : false,
                        "adjust_pure_negative" : true,
                        "boost" : 1.0
                      }
                    }
                  ],
                  "disable_coord" : false,
                  "adjust_pure_negative" : true,
                  "boost" : 1.0
                }
              },
              {
                "bool" : {
                  "filter" : [
                    {
                      "range" : {
                        "start" : {
                          "from" : 1631097122621,
                          "to" : 1631183522621,
                          "include_lower" : true,
                          "include_upper" : true,
                          "boost" : 1.0
                        }
                      }
                    }
                  ],
                  "disable_coord" : false,
                  "adjust_pure_negative" : true,
                  "boost" : 1.0
                }
              }
            ],
            "disable_coord" : false,
            "adjust_pure_negative" : true,
            "boost" : 1.0
          }
        }
      ],
      "disable_coord" : false,
      "adjust_pure_negative" : true,
      "boost" : 1.0
    }
  },
  "_source" : {
    "excludes" : [ ]
  },
  "sort" : [
    {
      "start" : {
        "order" : "desc",
        "unmapped_type" : ""
      }
    }
  ]
}

7、聚合某字段

1
2
3
4
5
6
7
8
9
10
11
12
GET /test-index-*/test_type/_search
{
    "size" : 0,
    "aggs" : {
        "Name" : {       # 可自定义
            "terms" : {
              "field" : "Name",
              "size" : 1000    # 可省略该参数，表示返回组的数量
            }
        }
    }
}

8、根据Name分组，每组根据Time字段倒序排列，且只返回一条数据

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
curl -H 'Content-type: application/json' -XGET 'http://192.168.x.x:9200/test-index-202201/_search' -d '{
  "size": 0,
  "aggs": {            # 关键词
    "Name": {          # 可自定义
      "terms": {
        "field": "Name"
      },
      "aggs": {          # 关键词
        "results": {     # 可自定义
          "top_hits": {  # 关键词
            "size": 1,   # 每组返回的条数
            "sort": [
              {
                "Time": {
                  "order": "desc"
                }
              }
            ]
          }
        }
      }
    }
  }
}
'

参考资料：https://elasticsearch.cn/question/10928

9、多 term 查询

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
{
  "from" : 0,
  "size" : 11,
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "ceshi1": "2955"
          }
        },
        {
          "term": {
            "ceshi2": true
          }
        },
        {
          "terms": {
            "ceshi3": [
              "123",
              "xxx"
            ]
          }
        },
        {
          "range": {
            "Time": {
              "gte": "2022-01-10T16:58:51Z",    # 日期默认是这种格式或者是毫秒时间戳，详情可参考：https://www.elastic.co/guide/en/elasticsearch/reference/current/date.html#date-params
              "lt": "2022-06-17T09:53:15Z"
            }
          }
        }
      ]
    }
  },
  "sort" : [
    {
      "Time" : {
        "order" : "desc",
        "unmapped_type" : "long"
      }
    },
    {
      "Id" : {
        "order" : "desc",
        "unmapped_type" : "keyword"
      }
    }
  ]
}

10、段合并相关操作

1
2
3
4
5
6
7
8
9
10
# 执行索引段合并
curl -H 'Content-type: application/json' -XPOST 'http://ip:9200/index_test/_forcemerge?max_num_segments=1' -d '{}'

# 查看某个index的forceMerge情况
GET /_cat/segments/order_dev1?v&s=prirep,shard
# 查看分段数
GET _cat/segments/order_dev1?v&h=shard,segment,size,size.memory

# 查看索引段详情
curl -H 'Content-type: application/json' -XGET 'http://ip:9200/_cat/indices/?index=index_test&s=segmentsCount:desc&v&h=index,segmentsCount,segmentsMemory,memoryTotal,mergesCurrent,mergesCurrentDocs,storeSize,p,r'

11、query、bool、filter

用 filter 包起来不会计算分值，可用来提高查询速度。

1
2
3
4
5
6
7
8
9
10
11
{
  "query": {
    "bool": {
      "filter": {
        "term": {
          "Id": "ef7f49dd2b1c45ac89eb63ffb2249a4a"
        }
      }
    }
  }
}

12、解决ES只能查询10000条数据的办法

1
2
3
{
	"track_total_hits": true
}

java 代码：

1
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder().trackTotalHits(true);

注：适用于 7.6.x 版本

13、Bool 查询

Bool 查询现在包括四种子句，must、filter、should、must_not。

可参考：

14、ES from size分页

如果用 from size 的方式分页，默认只能查到 10 万条。10 万条之后，查询就会报错：

说到查询，ES 还有一个限制，查询返回的总条数，默认为 1 万。如果想要返回命中的实际总数量，可这样修改：

1
2
3
{
	"track_total_hits": true
}

java 代码：

1
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder().trackTotalHits(true);

15、根据docId更新某文档

1
2
3
4
5
6
curl -H 'Content-type: application/json' -XPOST 'http://x.x.x.x:9200/test-index-202212/doc/AYUI4HTL_6uoiy8Q9sy0/_update' -d '{
 "doc": {
  "name": "ceshi111",
  "custom": "111"
 }
}'

16、不等于

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
{
  "from" : 0,
  "size" : 10,
  "query" : {
    "bool" : {
      "must_not" : [
        {
          "term" : {
            "id" : {
              "value" : "db9ccddf8f634cf6ae77df348a730a62"
            }
          }
        }
      ]
    }
  },
  "sort" : [
    {
      "date" : {
        "order" : "desc"
      }
    },
    {
      "id" : {
        "order" : "desc"
      }
    }
  ]
}

17、wildcard模糊查询

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
{
  "from" : 0,
  "size" : 10,
  "query" : {
    "bool" : {
      "filter" : [
        {
          "wildcard" : {
            "id" : {
              "wildcard" : "*db*"
            }
          }
        }
      ]
    }
  }
}

18、数据中含有 id 这个字段的数据

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
{
  "from" : 0,
  "size" : 10,
  "query" : {
    "bool" : {
      "filter" : [
        {
          "exists" : {
            "field" : "id"
          }
        }
      ]
    }
  }
}

19、Match

分词匹配检索

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
{
  "from" : 0,
  "size" : 10,
  "query" : {
    "bool" : {
      "filter" : [
        {
          "match" : {
            "id" : {
              "query" : "xxxxx",
              "operator" : "AND",
              "prefix_length" : 0,
              "max_expansions" : 50,
              "fuzzy_transpositions" : false,
              "lenient" : false,
              "zero_terms_query" : "NONE",
              "boost" : 1.0
            }
          }
        }
      ]
    }
  }
}

20、match、fuzzy、wildcard 的区别

参考资料：https://blog.csdn.net/weixin_43859729/article/details/108134329

21、terms query

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
{
  "query" : {
    "bool" : {
      "filter" : [
        {
          "terms" : {
            "key1" : [
              "asdasd",
              "test2"
            ]
          }
        }
      ]
    }
  }
}

该语句等同于sql语句中的inList

22、匹配查询match

match和term的区别是,match查询的时候,elasticsearch会根据你给定的字段提供合适的分析器,而term查询不会有分析器分析的过程，match查询相当于模糊匹配,只包含其中一部分关键词就行

同时还要注意match系列匹配时，datatype要设置为text，否则不会开启分词。