ElasticSearch multiple strings in multiple fields search

I was working on an ElasticSearch(ES) POC to store millions of database entries to ES and search for multiple strings in multiple fields for all these entries. Let me explain the problem in depth - we use different defect tracking tools such as Rational CharmNT, JIRA and so on, to log defects. In defects we update the exception trace to identify the logs. For example - a stack may look like
Exception in thread "main" java.lang.NullPointerException
        at com.mycompany.myproject.Book.getTitle(Book.java:16)
        at com.mycompany.myproject.Author.getBookTitles(Author.java:25) 
Similarly another stack may look like
Exception in thread "main" java.lang.NullPointerException
     at com.mycompany.myproject.Course.getSchedule(Course.java:162)
     at com.mycompany.myproject.CourseDao.getCourses(CourseDao.java:50) 

So, we when a tester sees a NullPointerException he will log the trace to the defect. But the issue here is, there could be multiple defects with NullPointerException. how could you search for a specific trace? querying database for millions of defects is a resource consuming affair. The best approach is to index each defects to elasticSearch and querying ES instead of querying a database.
Another glitch is the fields - a tester may log the trace in defect tracking tools summary section or a note section.

So we need to be able to find a NullPointer in Course module. We can follow the code given below

Node node =null;
try{
node = nodeBuilder().client(true).node();
Client client = node.client();


QueryBuilder wildcardQueryBuilder= QueryBuilders.
                                          queryStringQuery("*java.lang.NullPointerException*");
SearchResponse scrollResp = client.prepareSearch("Defects")
       .setSearchType(SearchType.SCAN)
       .setScroll(new TimeValue(60000))
       .setQuery(wildcardQueryBuilder)
       .setSize(100).execute().actionGet();

while (true) {

   for (SearchHit hit : scrollResp.getHits().getHits()) {

    if(hit.getSourceAsString().
contains("com.mycompany.myproject.Course.getSchedule(Course.java:162)")){
    System.out.println("NullPointer Found@ Course");
    }
      
   }
   scrollResp = client.prepareSearchScroll(scrollResp.getScrollId())
                                       .setScroll(new TimeValue(600000)).execute().actionGet();

   //Break condition: No hits are returned

   if (scrollResp.getHits().getHits().length == 0) {
       break;
   }
}


}catch(Exception ex){
ex.printStackTrace();
}finally{
node.close();

}

Comments

Popular posts from this blog

NSSM service eats 100% CPU

JPA Vs SpringJDBC

Ignite Running in Static IP multicast mode