# elasticsearch-fileimport

**Repository Path**: mirrors_codecentric/elasticsearch-fileimport

## Basic Information

- **Project Name**: elasticsearch-fileimport
- **Description**: Import JSON files directly as documents into Elasticsearch by using the ES transport protocol (not HTTP/REST)
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2020-08-08
- **Last Updated**: 2025-09-27

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

#elasticsearch-fileimport

Import JSON files directly as documents into Elasticsearch by using the ES transport protocol (not HTTP/REST).
The import is done via bulk requests.

##Prerequisites
* Java 8
* Tested with Elasticsearch 1.6 (should also work with earlier versions)

##Usage

    mvn package
    java -jar target/elasticsearch-fileimport-0.6-SNAPSHOT-jar-with-dependencies.jar file_import_settings.yml

To execute all unittests, also those who runs longer execute

    mvn clean package -DargLine="-Dfileimport.tests.big"

##Configuration
The configuration is done in YAML (or JSON). Look at the following example to see what's configurable:

    fileimport:
        # either transport or node
        mode: node
        
        # Folder with data
        # Folder is scanned recursively
        root: import/data
        
        # Fileextension of files to import, default is json
        #fileext: json
        
        # If set to true then import files line by line (one json doc per line)
        # If set to false (which is the default) treat one file as one document
        #linebyline: true   
        
        # Index name
        index: test
        
        # Type name
        type: test
        
        # Bulk settings, see https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/bulk.html
        #flush_interval: 5s
        max_bulk_actions: 1000
        max_concurrent_bulk_requests: 1
        max_volume_per_bulk_request: 10mb
        
        # needed when mode: transport
        transport:
            addresses:
                - localhost:9300
                - es.comany.com:9300
                
    # needed when mode: node
    cluster:
        name: elasticsearch
        
    
    # list other elasticsearch settings here if needed
    #discovery.zen.ping.multicast.enabled: false
    #discovery.zen.ping.unicast.hosts: host:9300
    #...

##Before you run it
* Install and start your Elasticsearch cluster
* Create your index (with your desired settings)
* Create your type mapping