# elasticsearch-springboot-demo

**Repository Path**: fengyexjtu/elasticsearch-springboot-demo

## Basic Information

- **Project Name**: elasticsearch-springboot-demo
- **Description**: elastic 5.x ,不使用spring-boot-elasticsearch-starter,仅用Elastic Java API操作数据.
- **Primary Language**: Java
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 3
- **Forks**: 4
- **Created**: 2017-12-05
- **Last Updated**: 2020-12-20

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

title:elastic search-note.md
### elastic search 笔记
#### 1. 下载和运行

下载 elasticsearch

[elasticsearch 官方地址](http://www.elastic.co)

下载 elasticsearch-head

[github地址](https://github.com/mobz/elasticsearch-head)

elasticsearch-head的运行
```bash
npm install
npm run start
```

elasticsearch-head和 elasticsearch 存在跨域问题,所以在 elasticsearch 的配置文件末尾加上
```
http.cors.enabled: true
http.cors.allow-origin: "*"
```

单机伪集群配置:
master 节点配置:
```
http.cors.enabled: true
http.cors.allow-origin: "*"
cluster.name: wali
node.name: master
node.master: true
network.host: 127.0.0.1
```

salve1:
```
cluster.name: wali
node.name: master
network.host: 127.0.0.1
http.port: 9201
transport.tcp.port: 9302
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300"]
```

salve2:
```
cluster.name: wali
node.name: master
network.host: 127.0.0.1
http.port: 9202
transport.tcp.port: 9302
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300"]
```

`with the same id but is a different node instance`
出现这个问题是因为 data 目录下还是 master 得数据,启动前把这些数据都删除

#### 2. 全部启动
```
master/bin/elasticsearch -d
slave1/bin/elasticsearch -d
slave2/bin/elasticsearch -d
cd elasticsearch-head && npm run start
```

#### 3. 用 head 建立索引

  点击新建所以,添加`book`, 默认分片为5,副本数为1.创建成功后可以看在 head 中查看.
  细线框是粗线框实例的备份.

创建结构化得索引:
点击复合查询,在查询中分别输入:
```
book/novel/_mappings
```
```
{
  "novel": {
    "properties": {
      "title": {
        "type": "text"
      }
    }
  }
}
```
`提交请求`后可以看到 `book` 中的索引信息中` mappings` 已经有数据了.

使用`rester` 创建索引
使用`put`的方式请求`http://localhost:9200`.

`body`中输入以下json格式的字符串

```
{
    "settings": {
        "number_of_shards": 3,
        "number_of_replicas": 1
    },
    "mappings": {
        "man": {
            "properties":{
                "name":{
                    "type":"text"
                },
                "country":{
                    "type":"keyword"
                },
                "age":{
                    "type":"integer"
                },
                "date":{
                    "type":"date",
                    "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
                }
            }
        },
        "woman":{
            
        }
    }
}
```

可以在head中查看到索引已经创建成功.

#### 4. 数据插入
- 指定文档id插入
  `put` `http://localhost:9200/people/man/1`
```
{
    "name":"wali",
    "age":30,
    "country":"china",
    "date":"1987-09-03"
}
```
路径中指定了`id`

- 自动生成id插入
  使用`post`方式访问`http://localhost:9200/people/man`
```
{
    "name":"another wali",
    "age":30,
    "country":"china",
    "date":"1987-09-03"
}
```
会自动生成一个id

#### 5. 修改数据
- 根据id修改,json内容为要修改的数据
  `post`方式访问`http://localhost:9200/people/man/1/_update`
```
{
    "doc":{
        "name":"who is wali"
    }
}
```
- 使用script方式修改
```
{
    "script":{
        "lang":"painless",
        "inline":"ctx._source.age += 10"
    }    
}
```
elasticsearch支持多种语言操作,`painless`只是其一.
其中要修改的数据还可以外置,由参数指定,如:
```
{
    "script":{
        "lang":"painless",
        "inline":"ctx._source.age = params.age",
        "params":{
            "age":100
        }
    }
}
```

#### 6. 删除操作
- 根据id删除数据
  `delete`方式访问`http://localhost:9200/people/man/1`
- 删除索引
  `delete`方式访问`http://localhost:9200/people`
  也可以在head中直接删除

> 关于修改索引的操作比较复杂

#### 7. 查询
##### 1. 创建book索引
##### 2. 使用head选择符合查询,post方式请求boot/novel/_mappings

```json
{
  "novel": {
    "properties": {
      "title": {
        "type": "text"
      },
      "word_count": {
        "type": "integer"
      },
      "author": {
        "type": "keyword"
      },
      "publish_date": {
        "type": "date",
        "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
      }
    }
  }
}

```
##### 3. 准备测试数据
##### 4. 查询所有
  post http://127.0.0.1/book/_search
```
{   
  "query":{
    "match_all":{}
  }
}
```
##### 5. 条件查询

```
{   
  "query":{
    "match":{
      "title":"elasticsearch"
    }
  },
  "from": 1,
  "size": 2,
  "sort":[{"publish_date":"desc"}]
}
```
> match_all :表示查询所有
> match : 表示条件查询
> from : 表示返回结果从第几页开始
> size : 表示返回结果的大小
> sort : 表示排序
>
##### 6. 聚合查询

```
{
  "aggs": {
    "group_by_word_count": {
      "terms":{
        "field":"word_count"
      }
    },
    "group_by_publish_date":{
      "terms":{
        "field":"publish_date"
      }
    }
  }
}
```
> aggs: 表明是聚合查询
> "group_by_word_count":自定义名称,可以随意
> terms:关键字
> field:使用的字段
>
##### 7. 统计查询
  ```
  {
  "aggs": {
    "grand_word_count":{
      "stats":{
        "field":"word_count"
      }
    }
  }
}
  ```
  返回结果:
  ```
aggregations":{
"grand_word_count":{
"count": 8,
"min": 2000,
"max": 5000,
"avg": 3375,
"sum": 27000
}
}
  ```
> 说明: 
>   aggs:统计查询
>   grand_word_count:自定义名称
>   stats:统计方法,可以换成min/max/sum
>   field:进行统计的字段
>

#### 8. 高级查询

高级查询分为子条件查询和复合查询

##### 1. 子条件查询:特定字段查询所指特定值

###### 1. query context

在查询过程中,除了判断文档是否满足查询条件外,ES还会计算一个_score来标识匹配的程度,旨在判断目标文档和查询条件匹配的有多好.

常用查询:

1) 全文本查询: 针对文本类型的查询

a. 模糊匹配:

​    post - http://127.0.0.1:9200/book/_search

```
{
  "query":{
    "match":{
      "title":"ElastichSearch入门"
    }
  }
}
```

​    从结果中可以看出,结果会匹配`ElasticSearch`和`入门`,他们的关系是或的关系,相当于自动分词

b. 习语匹配

```json
{
  "query":{
    "match_phrase":{
      "title": "ElasticSearch入门"
    }
  }
}
```

  从结果中可以看出,会把`ElasticSearch入门`当做一个整体的词进行匹配

c. 多个字段的模糊查询

```json
{
  "query":{
    "multi_match":{
      "query":"瓦力",
      "fields":["author","title"]
    }
  }
}
```

d. querystring,语法查询()

```
{
  "query":{
    "query_string":{
      "query":"(ElasticSearch) AND 入门) OR Python"
    }
  }
}
```

```
{
  "query":{
    "query_string":{
      "query":"瓦力 OR ElasticSearch",
      "fields":["author","title"]
    }
  }
}
```





2). 字段级别的查询: 针对结构化数据,如数字,日期等

```
{
	"query":{
      "term":{
        "word_count":1000
      }
	}
}
```

> term : 表示具体的字段查询

还可以指定范围:

```
{
  "query":{
    "range":{
      "word_count":{
        "gte": 1000,
        "lte": 2000
      }
    }
  }
}
```

> 关键词:`range`表明是范围查询,后面跟具体的字段,gte表示`>=`,lte表示`<=`

范围,还可以用在日期上.

###### 2. filter context

在查询过程中,只判断该文档是否满足条件,只有Yes或No

```
{
  "query":{
    "bool":{
     "filter":{
      "term":{
        "word_count":1000
      }
     } 
    }
  }
}
```

filter结合bool使用

#####  2. 复合条件查询:以一定的逻辑组合子条件查询

###### 1. 固定分数查询

```
{
  "query":{
    "constant_score":{
      "filter":{
        "match":{
          "title":"ElasticSearch"
        }
      },
      "boost":2
    }
  }
}
```

constant_score:固定分数,即把_score的值指定,如果不加boost则为1,指定了boost的值,则` _score`等于boost的值

> 注意: constant_score不支持match

###### 2. bool查询

```
{
  "query":{
    "bool":{
      "should":[
        {
          "match":{
            "author":"瓦力"
          }
        },
        {
          "match":{
            "title":"ElasticSearch"
          }
        }
      ]
    }
  }
}
```

should为关键词,应该满足他列出的条件,是或的关系

```
{
  "query":{
    "bool":{
      "must":[
        {
          "match":{
            "author":"瓦力"
          }
        },
        {
          "match":{
            "title":"ElasticSearch"
          }
        }
      ]
    }
  }
}
```

must:与的关系



must和filter

```json
{
  "query":{
    "bool":{
      "must":[
        {
          "match":{
            "author":"瓦力"
          }
        },
        {
          "match":{
            "title":"ElasticSearch"
          }
        }
      ],
      "filter:[
        "term":{
          "word_count":1000
        }
      ]
    }
  }
}
```

即在满足must中的条件的同时,还有满足过滤条件的数据才会最终返回.

must的反义词mustnot

```
{
  "query":{
    "mustnot":{
      "term":{
        "author":"wali"
      }
    }
  }
}
```

一定不能满足该条件.



#### 9. springboot集成ES

  1. 引入指定的版本

     ```xml
     		<dependency>
     			<groupId>org.elasticsearch.client</groupId>
     			<artifactId>transport</artifactId>
     			<version>5.5.2</version>
     		</dependency>

     		<dependency>
     			<groupId>org.elasticsearch</groupId>
     			<artifactId>elasticsearch</artifactId>
     			<version>5.5.2</version>
     		</dependency>

     		<dependency>
     			<groupId>org.apache.logging.log4j</groupId>
     			<artifactId>log4j-core</artifactId>
     			<version>2.7</version>
     		</dependency>
     ```

     > transport 5.5.2 默认的不是ElasticSearch 5.5.2,要使用指定的版本必须声明ElasticSearch的版本,如果依然冲突,在transport中使用exclusions

  2. 配置

```java
@Configuration
public class MyConfig {
    @Bean
    public TransportClient client() throws UnknownHostException {
        InetSocketTransportAddress node = new InetSocketTransportAddress(
                InetAddress.getByName("localhost"),
                9300 //tcp
        );
        
        Settings settings = Settings.builder()
                .put("cluster.name","wali")
                .build();
        TransportClient client = new PreBuiltTransportClient(settings);
        client.addTransportAddress(node);//可以增加多个节点
        return client;
    }
}
```

    3. 相关操作
  ```java
    @Autowired
    private TransportClient client;
    
    @GetMapping("/get/book/novel")
    @ResponseBody
    public ResponseEntity get(@RequestParam(value = "id", defaultValue = "") String id) {
        if (id.isEmpty())
            return new ResponseEntity(HttpStatus.NOT_FOUND);
        GetResponse result = client.prepareGet("book", "novel", id).get();
        if (!result.isExists()) {
            return new ResponseEntity(HttpStatus.NOT_FOUND);
        }
        return new ResponseEntity(result, HttpStatus.OK);
    }
    
    @PutMapping("/put/book/novel")
    @ResponseBody
    public ResponseEntity add(
            @RequestParam("title") String title,
            @RequestParam("author") String author,
            @RequestParam("word_count") int wordCount,
            @RequestParam("publish_date")
            @DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
                    Date publishDate
    ) {
        SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
        System.out.println(format.format(publishDate));
        try {
            XContentBuilder contentBuilder = 
                  XContentFactory.jsonBuilder().startObject()
                  .field("title", title)
                  .field("author", author)
                  .field("word_count", wordCount)
                  .field("publish_date", format.format(publishDate))
                  .endObject();
            System.out.println(contentBuilder.toString());
            IndexResponse result = 
                  client.prepareIndex("book", "novel")
                        .setSource(contentBuilder).get();
            
            return new ResponseEntity(result.getId(), HttpStatus.OK);
        } catch (IOException e) {
            e.printStackTrace();
            return new ResponseEntity(HttpStatus.INTERNAL_SERVER_ERROR);
        }
    }
    
    @DeleteMapping("/delete/book/novel")
    @ResponseBody
    public ResponseEntity delete(@RequestParam("id") String id) {
        
        DeleteResponse result = 
             this.client.prepareDelete("book", "novel", id).get();
        return new ResponseEntity(result.getResult().toString(), HttpStatus.OK);
    }
    
    @PutMapping("/update/book/novel")
    @ResponseBody
    public ResponseEntity update(
            @RequestParam(value = "id", required = true) String id,
            @RequestParam(value = "title", required = false) String title,
            @RequestParam(value = "author", required = false) String author,
            @RequestParam(value = "word_count", required = false) 
                    Integer wordCount,
            @RequestParam(value = "publish_date", required = false)
            @DateTimeFormat(pattern = "yyyy-MM-dd HH:mm:ss")
                    Date publishDate
    ) {
        UpdateRequest updateRequest = new UpdateRequest("book", "novel", id);
        try {
            XContentBuilder contentBuilder =
                    XContentFactory.jsonBuilder().startObject();
            if (title != null)
                contentBuilder.field("title", title);
            if (author != null)
                contentBuilder.field("author", author);
            if (wordCount != null)
                contentBuilder.field("word_count", wordCount);
            if (publishDate != null)
                contentBuilder.field("publish_date",
                new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(publishDate));
            contentBuilder.endObject();
            updateRequest.doc(contentBuilder);
        } catch (IOException e) {
            e.printStackTrace();
            return new ResponseEntity(HttpStatus.INTERNAL_SERVER_ERROR);
        }
        
        try {
            UpdateResponse result = this.client.update(updateRequest).get();
            return new ResponseEntity(result.getResult().toString(), HttpStatus.OK);
        } catch (InterruptedException e) {
            e.printStackTrace();
            return new ResponseEntity(HttpStatus.INTERNAL_SERVER_ERROR);
        } catch (ExecutionException e) {
            e.printStackTrace();
            return new ResponseEntity(HttpStatus.INTERNAL_SERVER_ERROR);
        }
        
    }//update
    
    
    @PostMapping("/query/book/novel")
    @ResponseBody
    public ResponseEntity query(
            @RequestParam(value = "title", required = false) String title,
            @RequestParam(value = "author", required = false) String author,
            @RequestParam(value = "lt_word_count", required = false) Integer ltWordCount,
            @RequestParam(value = "gt_word_count", required = false, defaultValue = "0")
                    Integer gtWordCount
    ) {
        BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
        if (title != null)
            boolQuery.must(QueryBuilders.matchQuery("title", title));
        if (author != null)
            boolQuery.must(QueryBuilders.matchQuery("author", author));
        
        RangeQueryBuilder rangeQuery =
                QueryBuilders.rangeQuery("word_count")
                        .from(gtWordCount);
        if (ltWordCount != null)
            rangeQuery.to(ltWordCount);
        boolQuery.filter(rangeQuery);
        
        SearchRequestBuilder searchRequestBuilder =
                this.client.prepareSearch("book")
                        .setTypes("novel")
                        .setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
                        .setQuery(boolQuery)
                        .setFrom(0)
                        .setSize(10);
        System.out.println(searchRequestBuilder);
        SearchResponse searchResponse = searchRequestBuilder.get();
        
        List<Map<String, Object>> result = new ArrayList<>();
        
        for (SearchHit searchHit : searchResponse.getHits()) {
            result.add(searchHit.getSource());
        }
        return new ResponseEntity(result, HttpStatus.OK);
    }
  ```
#### 10. 总结