Blog
Search

The new elasticsearch java Rest Client – part 2

-

In the previous blog post I started writing about using the new Rest based client for java in elasticsearch 5. In this blogpost I am going to continue that quest by experimenting with some layers of abstraction using spring features and other considerations.

Spring Factory Bean for RestClient

First I introduce the Client factory, responsible for creating the singleton client of type RestClient. This factory creates the client as a singleton. It also creates the Sniffer to check for available nodes, like we discussed in the first blog post (see references). The code is the same as before, therefore I am not going to show it again. Check the class: Handling the response

Handling a response can be challenging. For a basic query it is not to hard. But when interacting with nested structures and later on adding aggregations it is a lot harder. For now we focus on a basic response that looks like this:

{
  "took": 19,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 0.7373906,
    "hits": [
      {
        "_index": "luminis",
        "_type": "ams",
        "_id": "AVX57b0YY0m5tWxWetET",
        "_score": 0.7373906,
        "_source": {
          "name": "Jettro Coenradie",
          "email": "jettro@gridshore.nl",
          "specialties": [
            "java",
            "elasticsearch",
            "angularjs"
          ],
          "phone_number": "+31612345678"
        }
      }
    ]
  }
}

The QueryTemplate must to the boilerplate and from the Employee perspective we want to provide enough information to convert the _source parts into Employee objects. I use a combination of Jackson and generics to accomplish this. First let us have a look at the code to interact with the QueryTemplate:

public List queryForEmployees(String name) {

    Map params = new HashMap<>();

    params.put("name", name);

    params.put("operator", "and");


    QueryTemplate queryTemplate = queryTemplateFactory.createQueryTemplate();

    queryTemplate.setIndexString(INDEX);

    queryTemplate.setQueryFromTemplate("find_employee.twig", params);

    queryTemplate.setQueryTypeReference(new EmployeeTypeReference());


    return queryTemplate.execute();

}

As you can see from the code, the QueryTemplate receives the name of the index to query, the name of the twig template (see next section) and the parameters used by the twig template. The final thing we need to give is the TypeReference. This is necessary for Jackson to handle the generics. This type reference looks like this:

public class EmployeeTypeReference extends TypeReference> {
}


Finally we call the execute method of the QueryTemplate and notice that it returns a list of Employee objects. To see how this works we have to take a look at the response handling by the QueryTemplate.

public List execute() {

    List result = new ArrayList<>();


    this.queryService.executeQuery(indexString, query(), entity -> {

        try {

            QueryResponse queryResponse = jacksonObjectMapper.readValue(entity.getContent(), this.typeReference);


            queryResponse.getHits().getHits().forEach(tHit -> {

                result.add(tHit.getSource());

            });

        } catch (IOException e) {

            logger.warn("Cannot execute query", e);

        }

    });


    return result;

}


Using jackson we convert the son based response from elasticsearch client into a QueryResponse object. Notice that we have a generic type ’T’. Jackson only know how to do this by passing the right TypeReference. The java object tree resembles the json structure:

public class QueryResponse {

    private Long took;


    @JsonProperty(value = "timed_out")

    private Boolean timedOut;


    @JsonProperty(value = "_shards")

    private Shards shards;


    private Hits hits;

}
 
public class Hits {

    private List> hits;

    private Long total;


    @JsonProperty("max_score")

    private Double maxScore;

}
 
public class Hit {

    @JsonProperty(value = "_index")

    private String index;


    @JsonProperty(value = "_type")

    private String type;


    @JsonProperty(value = "_id")

    private String id;


    @JsonProperty(value = "_score")

    private Double score;


    @JsonProperty(value = "_source")

    private T source;
}

Notice what happens with the generic type ’T’, so the source is converted into the provided type. Now look back at the code of the execute method of the QueryTemplate. Here we obtain the hits from the QueryResponse and loop over these hits to obtain the Employee objects.

Using the twig template language

Why did I chose to use a template language to create the query? It is not really safe to just copy a few strings including user input together into one long string and provided that to the QueryTemplate executer. I wanted to stay close to json to make copy paste from the elastic console as easy a possible. However I also wanted some nice features to make creating queries easier. I found jTwig to be powerful enough and very easy to use in java. I do not want to write an extensive blogpost about jTwig. if you want to know more about it please check the references.

The following code block shows how we use jTwig:

public void setQueryFromTemplate(String templateName, Map modelParams) {

    JtwigTemplate template = JtwigTemplate.classpathTemplate("templates/" + templateName);

    JtwigModel model = JtwigModel.newModel();

    modelParams.forEach(model::with);


    this.query = template.render(model);

}

First you have to create the model and load the template. In my case I load the template from a file on the class path in the templates folder. Next I add the provided parameters to the model and finally I render the template using the model. The next code block shows the template.

{

    "query":{

{% if (length(name)==0) %}

        "match_all": {}

{% else %}

        "match":{

            "name": {

                "query": "{{ name }}",

                "operator": "{{ default(operator, 'or') }}"

            }

        }

{% endif %}

    }

}

This model supports two parameters: name and operator. First I check if the user provided something to search for. If the name is empty we just return the match_all query. If the name has length, a match query is created on the field name. Also notice that we can provide a default value for the operator parameter in this case. So if no operator is provided we make it or.

Concluding

That is it, now we have an easy way to write down a query in a jTwig template and parse the results using jackson. If would be very easy to query for another object instead of Employee if we wanted to. The indexing side is similar to the query side. Feel free to check this out yourself. All code is available in the github repo.

References

http://jtwig.org – Template language used for the queries.
https://amsterdam.luminis.eu/2016/07/07/the-new-elasticsearch-java-rest-client/ – part 1 of this blog series
https://github.com/jettro/elasticclientdemo – The sample project