tag:blogger.com,1999:blog-62876353173636771372024-03-25T06:58:22.844-07:00Exploring TechnologiesAjey Dudhehttp://www.blogger.com/profile/04566439328405849242noreply@blogger.comBlogger8125tag:blogger.com,1999:blog-6287635317363677137.post-62561062632877642582018-02-16T02:14:00.000-08:002019-08-23T08:46:07.709-07:00Writing and integrating tests for logstash ruby filter script in maven build<div dir="ltr" style="text-align: left;" trbidi="on">
<h3 style="text-align: left;">
Problem statement</h3>
In my current release of the product we had to do Elasticsearch (ES) upgrade. The product was on ES 1.x and we had to move to ES 6.x. Now, since ES supports upgrade only from previous major version, the only option we had was to do re-indexing of the data. We used following approach to re-index the data:<br />
<br />
<ol style="text-align: left;">
<li>Implement a java executable jar to directly read the index files of ES 1.x using lucene APIs and write each document to stdout.</li>
<li>Use logstash to invoke the above jar and read the stdout.</li>
<li>Create logstash filter configuration to transform the data from ES 1.x into ES 6.x compatible form.</li>
<li>Once the data is transformed then write it to the ES 6.x instance.</li>
</ol>
<div>
This worked well for us except for #3. The transform configuration written in a configuration file was not readable and maintainable. The editors would interpret the configuration file as plain text. So without any highlighting and indentations it was not readable at all. Also, we had to write transform for almost all the data (events) we had so there was too much of <b><i>if-elseif-elseif....else</i></b>. This was not only hampering the readability but also would have impacted the event transformation performance. For every event doing many if-else checks is definitely not good. What we wanted was a clean, readable & maintainable transformation.</div>
<h3 style="text-align: left;">
</h3>
<h3 style="text-align: left;">
Solution</h3>
<div>
The <a href="https://www.elastic.co/guide/en/logstash/current/plugins-filters-ruby.html" target="_blank">logstash ruby filter</a> was the way to go. So, I rewrote the entire transformation in ruby script as follows:</div>
<div>
<ul style="text-align: left;">
<li>Wrote a single ruby script file for transformation.</li>
<li>Avoided if-else statements and instead used look ups to determine what transformation needs to be done per event type.</li>
<li>Invoked the logstash provided filters like <a href="https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html" target="_blank">mutate</a>, <a href="https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html" target="_blank">date</a> etc.</li>
<li>On top of this we also wrote product specific transformations.</li>
<li>Added logging to the script.</li>
</ul>
<h4 style="text-align: left;">
</h4>
<h4 style="text-align: left;">
Integrating with maven build</h4>
</div>
<div>
While implementing the transformation in ruby made it more readable and maintainable, there was one thing missing. It was not testable outside logstash. One can write <a href="https://www.elastic.co/guide/en/logstash/current/plugins-filters-ruby.html#_testing_the_ruby_script" target="_blank">inline test for ruby filter</a> but they have to be executed within logstash. What I wanted was to integrate the tests with our maven build so that if there are any test failures, the build should also fail.</div>
<div>
Since I was new to ruby I had to spend some time understanding ruby gems etc. and finally was able to run the ruby transform script in both maven build as well as logstash. This was achieved as follows:</div>
<div>
<ul style="text-align: left;">
<li>Copied the logstash deployment folder locally along with our maven project.</li>
<li>In the maven pom.xml used the exec-maven-plugin to execute the ruby script.</li>
<li>For the script to run properly, set the paths properly for the jars as well as ruby gems.</li>
<li>Quickly implemented a small testing framework in ruby which would:</li>
<ul>
<li>Scan for test classes.</li>
<li>Execute the test method. Test methods were supposed to have name starting with <b>test_ </b>e.g. test_event_types</li>
<li>If the test fails then log the error.</li>
<li>Execute all the tests and then print summary like total tests executed, tests passed/failed etc.</li>
<li>If there is even a single test failure then fail the maven build.</li>
</ul>
<li>Once integrated we wrote many tests around the transformation per event type.</li>
</ul>
<h4 style="text-align: left;">
</h4>
<h4 style="text-align: left;">
Implementation</h4>
</div>
<div>
The working sample for above approach is available at: <a href="https://github.com/ajeydudhe/java-pocs/tree/master/logstash-ruby-transform" target="_blank">https://github.com/ajeydudhe/java-pocs/tree/master/logstash-ruby-transform</a></div>
<div>
<br /></div>
</div>
Ajey Dudhehttp://www.blogger.com/profile/04566439328405849242noreply@blogger.com0tag:blogger.com,1999:blog-6287635317363677137.post-58056289070893158362017-10-10T02:56:00.000-07:002017-10-10T03:12:26.161-07:00Simple sequential workflow using Spring framework and Spring Expression Language<div dir="ltr" style="text-align: left;" trbidi="on">
<h1>
Requirement</h1>
Recently, in one of my projects, there was a requirement for simple workflow around some background processing tasks. Most of the tasks were already implemented so we had to just implement the workflow around those.<br />
<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<h1>
Available solutions</h1>
There are many workflow frameworks available for Java. Even though they met the expectations, they seemed to be too heavy for a simple requirement. Next, there were blogs around how to use Spring framework for creating simple workflows using Spring beans. This indeed was what I was looking for. However, something was missing here. Each action in the workflow can be a Spring bean with a simple interface having an execute method as follow:<br />
<br />
<script src="https://gist.github.com/ajeydudhe/169e835fa78b3cfd87d8063da90e20bf.js"></script>
But how do we pass parameters to the actions. Most of the implementations would pass a map between actions to the <b><i>execute()</i></b> method. This means an action would add it's result to the map using a key and the next action would read the value from this map. It would make the actions aware of each other or at least aware of the keys for passing information. If the key or the data type of value changes then the next action reading the value would break.<br />
<br />
<h1>
Expected behavior</h1>
It would have been far better to implement an action as a normal class with its dependencies being injected using, say, Spring dependency injection. Then during workflow execution the <b><i>execute()</i></b> method can be invoked to get the desired output. We should be able to pass the output from one action to the next action without the actions being aware of each other.<br />
<br />
<h1>
Solution</h1>
<h2>
Using the Spring Expression Language</h2>
The <a href="https://docs.spring.io/spring/docs/current/spring-framework-reference/core.html#expressions" target="_blank">Spring Expression Language</a> allows to have simple expressions defined and evaluated against specified object instance. The bean XML allows using such expression to inject property values. The idea was to use SpEL for defining the values to be injected into the workflow action. Let's take an example of a simple action which takes a string as an input and simply returns reverse string as output. The action is implemented as follows:<br />
<br />
<script src="https://gist.github.com/ajeydudhe/ae087a4d5514e9fe65bada4fa94f9cb4.js"></script>
It simply takes a string value as an input in the constructor itself and in the <b><i>execute()</i></b> call returns the reverse string. The instance of the above action will be created within the code using the application context's <b><i>getBean()</i></b> method. But how do we inject the required value? For that first let's have a look at how to define the bean for this action class.<br />
<br />
<script src="https://gist.github.com/ajeydudhe/2e98ca01aea27babb73cfcfdcbd35343.js"></script>
Above is a simple bean definition mentioning the implementation class and the parameters. In this case we are indicating the value to be passed to the constructor. Notice, that the value is actually an expression starting with <b>%{</b> and ending with <b>}</b>. We have defined our custom expression prefix using <b>%</b> because Spring already uses <b>#</b> and <b>$</b> for identifying it's expressions. The variable name <b><i>source</i></b> can be anything you want. For our example, the <b><i>source</i></b> refers to the original input to the workflow. It can be anything from a simple string, integer, class instance etc. to array, set, map etc.<br />
<br />
For our workflow, we will have list of beans representing the actions. The workflow will be responsible to create instance of the actions and execute them but while doing so it should be able to pass the parameters defined by our custom expression. Let's see how we can do this.<br />
<br />
<h3 style="text-align: left;">
Using custom expression resolver</h3>
<div>
Spring allows to have custom expression resolver implementing the <a href="https://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/beans/factory/config/BeanExpressionResolver.html" target="_blank">BeanExpressionResolver</a> interface. Following is the snippet for our <b><i>BeanInstantiator </i></b>class implementing this interface as well as the <a href="https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/beans/factory/config/BeanFactoryPostProcessor.html" target="_blank">BeanFactoryPostProcessor</a> interface.<br />
<br />
<script src="https://gist.github.com/ajeydudhe/69c4c9c8becd3553e43b3856fe5eb104.js"></script>
</div>
<div>
The <b><i>postProcessBeanFactory()</i></b> method at <b><i>Line 4</i></b> is for the <a href="https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/beans/factory/config/BeanFactoryPostProcessor.html" target="_blank">BeanFactoryPostProcessor</a> interface. Here, we are going to register our custom resolver which is the same class. Also, we are preserving the existing resolver.<br />
<br />
The <b><i>evaluate()</i></b> method at <b><i>Line 10</i></b> is for the <a href="https://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/beans/factory/config/BeanExpressionResolver.html" target="_blank">BeanExpressionResolver</a> interface. Whenever, Spring encounters an expression then this method will be called. Here, we are checking if the expression is our custom expression. The expression string can be enclosed in <b><i>%{}</i></b> e.g. <b><i>%{source}</i></b> or it can be part of any other string value e.g. <b><i>This will have %{source} expression embedded inside</i></b>. If we have a custom expression then:<br />
<ul style="text-align: left;">
<li>we create instance of <a href="https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/expression/spel/standard/SpelExpressionParser.html" target="_blank">SpelExpressionParser</a>.</li>
<li>parse the expression.</li>
<li>invoke the parsed expression and pass in the required data.</li>
</ul>
<div>
In above case, <b><i>Line 16</i></b> delegates the evaluation call to <b><i>doEvaluate()</i></b> method which does above steps. One thing abstracted here currently on <b><i>Line 16</i></b> is the <b><i>EXPRESSION_EVALUATION_CONTEXT</i></b> which is instance of <a href="https://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/expression/EvaluationContext.html" target="_blank">EvaluationContext</a>. We are storing this value in thread local because the <b><i>evaluate()</i></b> method is called internally by Spring and there is no other way for us to pass the context.<br />
<br />
<h3 style="text-align: left;">
Passing the context to Spring expression resolver</h3>
In above snippet we have seen how to add our own expression resolver. But how do we pass in the required data to the resolver? In our case we have use the expression as <b><i>%{source}</i></b> saying pass in the original source input to the action instance through constructor. Following is the additional snippet for the same <b><i>BeanInstantiator</i></b> class above (previous methods removed for readability).<br />
<br />
<script src="https://gist.github.com/ajeydudhe/bcc9969947a903f3a212a219a6350940.js"></script>
The <b><i>BeanInstantiator</i></b> class also implements the <a href="https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/context/ApplicationContextAware.html" target="_blank">ApplicationContextAware</a> interface. So we get the application context passed into the <b><i>setApplicationContext()</i></b> method at <b>Line 4</b>.<br />
<br />
The <b><i>BeanInstantiator</i></b> class exposes the <b><i>getAction()</i></b> method. This method takes the bean id of the workflow action to be instantiated. It then uses the <b><i>applicationContext.getBean()</i></b> method to create instance of the action class. But how will our custom expression be resolved to inject the required value specified by the expression <b><i>%{source}</i></b>.<br />
<br />
<b>Line 11</b> in <b><i>getAction()</i></b> method creates an instance <b><i>rootEvalObject</i></b> of anonymous class having getters getContext(), getSource(), getOutput() returning the original values.<br />
<br />
<b>Line 16</b> in <b><i>getAction()</i></b> creates the instance of <a href="https://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/expression/spel/support/StandardEvaluationContext.html" target="_blank">StandardEvaluationContext</a> passing the <b><i>rootEvalObject.</i></b><br />
<b><i><br /></i></b>
But <b><i>getAction()</i></b> is our method and we are just calling the <b><i>applicationContext.getBean() </i></b>to get the instance. Spring will internally call the <b><i>evaluate()</i></b> method on our class since it also implements the <a href="https://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/beans/factory/config/BeanExpressionResolver.html" target="_blank">BeanExpressionResolver</a> interface.<br />
<br />
<b>Line 17</b> sets the <b><i>StandardEvaluationContext</i></b> instance to thread local variable <b style="font-style: italic;">EXPRESSION_EVALUATION_CONTEXT </b>and then calls <b><i>applicatinContext.getBean()</i></b> to create the instance. In the <b><i>evaluate()</i></b> method called by Spring we retrieve this context and then evaluate the expression.<br />
<br />
Since we have used the getters as getContext(), getSource(), getOutput() you can refer to them as properties or getters directly in expressions e.g. as <b><i>%{source}</i></b> or <b><i>%{getSource()}</i></b><br />
<b><i><br /></i></b>
You can add as many properties you want like getContext(), getOutput() etc. returning whatever values you want. You can even add methods to be used in expression e.g. to transform data, to get some configuration etc.<br />
<br />
<h1>
Workflow-lite implementation</h1>
<div>
The implementation for the workflow is at: <a href="https://github.com/ajeydudhe/workflow-lite">https://github.com/ajeydudhe/workflow-lite</a></div>
<div>
<br /></div>
</div>
</div>
<div>
It allows one to define & execute the workflow using UML activity diagram. Mostly useful for implementing predefined workflows in a product.</div>
</div>
</div>
Ajey Dudhehttp://www.blogger.com/profile/04566439328405849242noreply@blogger.com8tag:blogger.com,1999:blog-6287635317363677137.post-90547237415862103692017-06-22T22:16:00.001-07:002017-06-22T22:16:32.617-07:00Elasticsearch: Extending ES using AOP<div dir="ltr" style="text-align: left;" trbidi="on">
Elasticsearch provides the ability for extending the basic functionality using scripts or plug-ins. But at the same time ES has restrictions on changing or extending existing functionality like search actions etc. For example, say, I need to change the result set for every query or change the input search criteria etc. In such a case, we can use <a href="https://en.wikipedia.org/wiki/Aspect-oriented_programming" target="_blank">AOP</a> to extend ES functionality.<br />
AOP is a vast topic in itself so I will not cover it here. Basically, it allows to control what happens before a method execution, after a method execution, change the original input parameters, change the return values etc.<br />
For this post we will try to monitor the ES search parameters using AOP. We will use <a href="https://en.wikipedia.org/wiki/AspectJ" target="_blank">AspectJ</a> programming using maven project as follows:<br />
<ul style="text-align: left;">
<li>Create the maven project</li>
<li>Find out the ES methods to be monitored</li>
<li>Define the Pointcuts & Advices</li>
<li>Compile the jar</li>
<li>Use <a href="https://eclipse.org/aspectj/doc/released/devguide/ltw-rules.html" target="_blank">Load Time Weaving</a></li>
<li>Start ES and monitor the queries</li>
</ul>
<b><br /></b>
<b>Source code</b><br />
The source code for this example is located <a href="https://github.com/ajeydudhe/elasticsearch/tree/master/es5x-method-interceptor" target="_blank">here</a>.<br />
<br />
<b>Create the maven project</b><br />
Create a simple maven jar project with following POM xml.<br />
<script src="https://gist.github.com/ajeydudhe/8e2f74a5f21dac7417229e05b07dcbff.js"></script>
<ul style="text-align: left;">
<li>Add dependencies for <i><b>aspectjrt</b></i> line 13-17 and <b><i>aspectjweaver</i></b> line 18-22</li>
<li>Add <b><i>aspectj-maven-plugin</i></b> line 31-48</li>
</ul>
<b><br /></b>
<b>ES methods to monitor</b><br />
We will monitor the searches by ES. The actual search execution is done by the SearchService. We will monitor the executeSearch() and executeFetch() methods on this class.<br />
<br />
<b>Define Pointcuts & Advices</b><br />
The <b><i>SearchServiceAspect</i></b> class defines the Pointcuts & Advices using annotations. Following is the definition to monitor the SearchService.executeSearch() method:
<br />
<script src="https://gist.github.com/ajeydudhe/9d80290477eb84fb29d1c35230892f05.js"></script>
<ul style="text-align: left;">
<li>Line 5 defines the Around advice.</li>
<li>The advice is for ES method SearchService.executeQueryPhase() which takes two parameters: ShardSearchTransportRequest & SearchTask</li>
<li>It also mentions that these two should be passed to our method which will handle the execution.</li>
<li>Line 6 is the method in our class which will be executed when SearchService.executeQueryPhase() is to be executed.</li>
<li>In this method we can decide what to do i.e. do some processing or continue execution using joinPoint.proceeed() etc.</li>
<li>For this example, we are just logging the search query information.</li>
</ul>
<div>
<b><br /></b>
<b>aop.xml</b></div>
<div>
Next we define the aop.xml which acts as input to the AOP compiler.</div>
<div>
<script src="https://gist.github.com/ajeydudhe/a5d416cd81fa591c82fe4cb281f295d8.js"></script>
<ul style="text-align: left;">
<li>Line 4 declares that our aspect pointcuts & advices are in class my.elasticsearch.aspects.SearchServiceAspect</li>
<li>Line 6 tells the weaver to be more verbose to help troubleshooting the issues if any.</li>
<li>Line 7 tells the weaver which class in the target application is to be woven.</li>
<li>The aop.xml needs to be present at <b><i>src/main/resources/META-INF</i></b> in the project.</li>
</ul>
<div>
<b><br /></b></div>
<div>
<b>Deploying the jars</b></div>
</div>
<div>
<ul style="text-align: left;">
<li>Compile the project and generate the jar using command: <b><i>mvn install</b></i></li>
<li>Copy the generated <b><i>es5x-method-interceptor-0.0.1-SNAPSHOT.jar</b></i> into the <b><i>elasticsearch/lib</b></i> folder as per the ES installation.</li>
</ul>
<div>
<b><br /></b></div>
<div>
<b>Configure ES startup</b></div>
</div>
<div>
Now, we need to start ES using the aspect weaver as java agent. </div>
<div>
<ul style="text-align: left;">
<li>Copy the <b><i>aspectjweaver-1.8.9.jar</b></i> into <b><i>elasticsearch/bin</b></i> folder.</li>
<li>Open the <b><i>bin/elasticsearch.bat</b></i> or <b><i>bin/elasticsearch.sh</b></i> file (I have verified with Windows batch file)</li>
<li>Add following line before the java command execution to launch the ES</li>
</ul>
<span style="font-size: x-small;"> <b>SET ES_JAVA_OPTS=%ES_JAVA_OPTS% -Djava.security.policy=enable_aspectj_classes.policy -javaagent:aspectjweaver-1.8.9.jar</b></span><br />
<ul style="text-align: left;">
<li>With above change we are basically passing two additional command line parameters while start the ES:</li>
<ul>
<li><b>javaagent:</b> Using the aspectj weaver as java agent.</li>
<li><b>java.security.policy:</b> ES 5x uses java security manager and hence our aspect weaver won't work. Using the above parameter we are asking the java security manager to grant the required permissions. The content of enable_aspectj_classes.policy is as follows:</li>
</ul>
</ul>
grant {<br />
<ul style="text-align: left;"><ul></ul>
</ul>
permission java.security.AllPermission;<br />
<ul style="text-align: left;"><ul></ul>
</ul>
};<br />
<ul style="text-align: left;"><ul><ul>
<li>We are granting all permissions to the application. It is recommended that you find out granular permissions by <a href="http://docs.oracle.com/javase/7/docs/technotes/guides/security/troubleshooting-security.html" target="_blank">enabling the logging</a> at access level and update the above file as required.</li>
<li>The file is placed in <b><i>elasticsearch/bin</b></i> folder.</li>
</ul>
</ul>
</ul>
<div>
The logs from weaver won't be included in the ES logs. So you can redirect the ES command line output (if not running as service/daemon) to a file as follows:</div>
</div>
<div>
<b><i>elasticsearch.bat > c:\es5x.log 2>&1</i></b></div>
<div>
You should see following logs:</div>
<script src="https://gist.github.com/ajeydudhe/0a5384cd19221ff5c137ef898b8cd2ba.js"></script>
Line 18 & 19 indicates that out pointcuts & advices have been woven.
<br />
<div>
<br />
<b>Populating test data</b><br />
<ul style="text-align: left;">
<li>We will create two indices as aop_test_01 & aop_test_02 having file type with following data:</li>
</ul>
{"name": "file_01", "size": 5678}<br />
<ul style="text-align: left;">
<li>Populate as many entries as you want.</li>
<li>Make sure both indices have some data</li>
</ul>
<div>
<b><br /></b></div>
<div>
<b>Querying the data</b></div>
<div>
Lets issue a simple search snapping both the indices as:</div>
<div>
<i>http://localhost:9200/aop_test_*/_search</i></div>
<div>
You should see that our advice is getting executed from the following logs:</div>
<script src="https://gist.github.com/ajeydudhe/15dae976eb16bcf68a051accb427dc26.js"></script>
<ul style="text-align: left;">
<li>Since we are querying multiple indices resulting in multiple shards query we see both executeQueryPhase() & executeFetchPhase() advice getting called.</li>
<li>In executeQueryPhase() we are just logging the indices. We can log additional info like the query to be executed etc.</li>
<li>In the executeFetchPhase() we are also logging the docIds which the search returns.</li>
<li>You can read more on how search gets executed <a href="https://www.elastic.co/guide/en/elasticsearch/guide/current/distributed-search.html" target="_blank">here</a>.</li>
</ul>
<div>
<br />
<b>Summary</b><br />
With this simple POC we have hooked our custom code into ES which will allow use to customize ES behavior as needed. Currently, we have just logged the input requests but can do more like change the input/output etc.</div>
<div>
<br /></div>
</div>
<div>
<br /></div>
</div>
Ajey Dudhehttp://www.blogger.com/profile/04566439328405849242noreply@blogger.com6tag:blogger.com,1999:blog-6287635317363677137.post-12865053272894709212015-12-09T03:54:00.002-08:002015-12-09T03:54:30.016-08:00AngularJS: Getting started<div dir="ltr" style="text-align: left;" trbidi="on">
<h2 style="text-align: left;">
<span style="font-family: inherit;">
Introduction</span></h2>
<div>
<span style="font-family: inherit;">AngularJS is one of the most popular frameworks for web UI development. Following are some major features of the framework:</span></div>
<div>
<ul style="text-align: left;">
<li><span style="font-family: inherit;">Makes DOM manipulation easy.</span></li>
<li><span style="font-family: inherit;">Segregates back-end (JavaScript) from UI (html).</span></li>
<li><span style="font-family: inherit;">Provides dependency injection.</span></li>
<li><span style="font-family: inherit;">Easy to develop Single Page Application (SPA) using the routing support.</span></li>
</ul>
<div>
<span style="font-family: inherit;">Instead of directly jumping into the AngularJS features like modules, controller, services etc. let's try to build a very basic UI using AngularJS and understand how AngularJS works.</span></div>
</div>
<h2 style="text-align: left;">
</h2>
<h2 style="text-align: left;">
<span style="font-family: inherit;">
Sample UI - Traditional way</span></h2>
<div>
<span style="font-family: inherit;">Let's try to develop following UI using traditional approach and then using AngularJS.</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitsTyErFqO6UuGwxSup-yzkf4DfL5CSztVIbC4xDJfTFByGHiqJvPvQBH-1ZDzCT6NJUCVaxVeRIe-K2R0lNyzls1Vt5I2cDxAzB37uY1bCyPAUqP7MKLS6ZREl7BK_SIhjEhnWuAKiD6Z/s1600/Settings_01.png" imageanchor="1"><span style="font-family: inherit;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitsTyErFqO6UuGwxSup-yzkf4DfL5CSztVIbC4xDJfTFByGHiqJvPvQBH-1ZDzCT6NJUCVaxVeRIe-K2R0lNyzls1Vt5I2cDxAzB37uY1bCyPAUqP7MKLS6ZREl7BK_SIhjEhnWuAKiD6Z/s400/Settings_01.png" /></span></a></div>
<div>
<span style="font-family: inherit;">This is very basic UI displaying the administrator details like name and email address. Then there is a checkbox to toggle display of some advanced options. If checked it will show advanced options as follows:</span></div>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8jTOo80lIdxeEnDMZeBnjlg-voER15JqPmxoAirSSn17GjheWocIhj5D6bPsVBseM9rmpUz-XiA6C_hg9NpWP4gOc1w8uYe-8E9YZRvzb0nC5_qzRN_4oX-MlxmmSxBP-4KzKbyWoz-rI/s1600/Settings_02.png" imageanchor="1"><span style="font-family: inherit;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8jTOo80lIdxeEnDMZeBnjlg-voER15JqPmxoAirSSn17GjheWocIhj5D6bPsVBseM9rmpUz-XiA6C_hg9NpWP4gOc1w8uYe-8E9YZRvzb0nC5_qzRN_4oX-MlxmmSxBP-4KzKbyWoz-rI/s400/Settings_02.png" /></span></a></div>
<h3 style="text-align: left;">
<span style="font-family: inherit;">
HTML markups</span></h3>
<div>
<span style="font-family: inherit;">The HTML snippet for the UI will be as follows:</span></div>
<div>
<hr />
</div>
<div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"> <</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">div</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">>
</span></span></div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"> <</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">h2</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">>Administrator</</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">h2</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">>
</span></span></div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas";"> </span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"><</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">label</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">><</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">b</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">>Name:
</</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">b</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">></</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">label</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">></span><span style="color: #080808; font-family: "consolas";"><</span><span style="color: #bf4f24; font-family: "consolas";">span </span><span style="color: #bf4f24; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">id</span><span style="color: #080808; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">=</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">"lblA</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">dminName</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">"</span><span style="color: #080808; font-family: "consolas";">></span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"></</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">span</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">></span></span></div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas";"> </span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"><</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">label</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">><</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">b</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">>Email:
</</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">b</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">></</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">label</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">></span><span style="color: #080808; font-family: "consolas";"><</span><span style="color: #bf4f24; font-family: "consolas";">span </span><span style="color: #bf4f24; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">id</span><span style="color: #080808; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">=</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">"lblA</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">dminEmail</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">"</span><span style="color: #080808; font-family: "consolas";">></span><span style="color: #080808; font-family: "consolas";"></</span><span style="color: #bf4f24; font-family: "consolas";">span</span><span style="color: #080808; font-family: "consolas";">></span></span></div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas";"> </span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"><</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">input</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"> </span><span style="color: #bf4f24; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">id</span><span style="color: #080808; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">=</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">"chkAdvanceOptions" </span></span><br />
<span style="font-family: inherit;"><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;"> type</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">=</span><span style="color: #0b6125; font-family: "consolas"; vertical-align: baseline;">"checkbox" </span></span><br />
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"> </span><span style="color: #bf4f24; font-family: "consolas";">onclick</span><span style="color: #080808; font-family: "consolas";">="toggleShowAdvanceOptions();"></span></span></div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas";"> </span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"><</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">label</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">>Show
Advance Options</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"></</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">label</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">></span></span></div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas";"> </span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"><</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">div </span><span style="color: #bf4f24; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">id</span><span style="color: #080808; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">=</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">"divA</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">dvanceOptions</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">" </span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">style</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">=</span><span style="color: #0b6125; font-family: "consolas"; vertical-align: baseline;">"display: none"</span><span style="color: #080808; font-family: "consolas";">></span></span></div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas";"> </span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"><</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">b</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">>Advance
options here...</</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">b</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">>
</span></span></div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas";"> </span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"></</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">div</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">>
</span></span></div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"> </</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">div</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">></span><span style="font-family: "calibri"; font-size: 5pt; vertical-align: baseline;"> </span></span></div>
</div>
<div>
<hr />
<h3 style="text-align: left;">
<span style="font-family: inherit;">
Populating administrator details</span></h3>
</div>
<div>
<span style="font-family: inherit;">The UI can be populated using following JavaScript:</span></div>
<div>
<hr />
<div style="direction: ltr; margin: 0pt 0in; text-indent: 0in; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #000088; font-family: "monaco"; vertical-align: baseline;">function</span><span style="font-family: "monaco"; vertical-align: baseline;"> </span><span style="font-family: "monaco"; vertical-align: baseline;">initializeSettings</span><span style="font-family: "monaco"; vertical-align: baseline;"> </span><span style="color: #666600; font-family: "monaco"; vertical-align: baseline;">()</span><span style="font-family: "monaco"; vertical-align: baseline;"> </span><span style="color: #666600; font-family: "monaco"; vertical-align: baseline;">{</span></span></div>
<div style="direction: ltr; margin: 0pt 0in; text-indent: 0in; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="color: #000088; font-family: inherit; vertical-align: baseline;"><br /></span></div>
<div style="direction: ltr; margin: 0pt 0in; text-indent: 0in; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="font-family: "monaco"; vertical-align: baseline;"></span><span style="color: #000088; font-family: "monaco"; vertical-align: baseline;"> var</span><span style="font-family: "monaco"; vertical-align: baseline;"> administrator </span><span style="color: #666600; font-family: "monaco"; vertical-align: baseline;">=</span><span style="font-family: "monaco"; vertical-align: baseline;"> </span><span style="font-family: "monaco"; vertical-align: baseline;">getAdministratorDetails</span><span style="color: #666600; font-family: "monaco"; vertical-align: baseline;">();</span><span style="font-family: "monaco"; vertical-align: baseline;"> </span></span></div>
<div style="direction: ltr; margin: 0pt 0in; text-indent: 0in; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="font-family: "monaco"; vertical-align: baseline;"> document</span><span style="color: #666600; font-family: "monaco"; vertical-align: baseline;">.</span><span style="font-family: "monaco"; vertical-align: baseline;">getElementById</span><span style="color: #666600; font-family: "monaco"; vertical-align: baseline;">(</span><span style="color: #008800; font-family: "monaco"; vertical-align: baseline;">'</span><span style="color: #008800; font-family: "monaco"; vertical-align: baseline;">lblAdminName</span><span style="color: #008800; font-family: "monaco"; vertical-align: baseline;">'</span><span style="color: #666600; font-family: "monaco"; vertical-align: baseline;">).</span><span style="font-family: "monaco"; vertical-align: baseline;">innerHTML</span><span style="font-family: "monaco"; vertical-align: baseline;"> </span><span style="color: #666600; font-family: "monaco"; vertical-align: baseline;">=</span><span style="font-family: "monaco"; vertical-align: baseline;"> administrator</span><span style="color: #666600; font-family: "monaco"; vertical-align: baseline;">.</span><span style="font-family: "monaco"; vertical-align: baseline;">name</span><span style="color: #666600; font-family: "monaco"; vertical-align: baseline;">;</span><span style="font-family: "monaco"; vertical-align: baseline;"> </span></span></div>
<div style="direction: ltr; margin: 0pt 0in; text-indent: 0in; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="font-family: "monaco"; vertical-align: baseline;"> document</span><span style="color: #666600; font-family: "monaco"; vertical-align: baseline;">.</span><span style="font-family: "monaco"; vertical-align: baseline;">getElementById</span><span style="color: #666600; font-family: "monaco"; vertical-align: baseline;">(</span><span style="color: #008800; font-family: "monaco"; vertical-align: baseline;">'</span><span style="color: #008800; font-family: "monaco"; vertical-align: baseline;">lblAdminEmail</span><span style="color: #008800; font-family: "monaco"; vertical-align: baseline;">'</span><span style="color: #666600; font-family: "monaco"; vertical-align: baseline;">).</span><span style="font-family: "monaco"; vertical-align: baseline;">innerHTML</span><span style="font-family: "monaco"; vertical-align: baseline;"> </span><span style="color: #666600; font-family: "monaco"; vertical-align: baseline;">=</span><span style="font-family: "monaco"; vertical-align: baseline;"> </span><span style="font-family: "monaco"; vertical-align: baseline;">administrator</span><span style="color: #666600; font-family: "monaco"; vertical-align: baseline;">.</span><span style="font-family: "monaco"; vertical-align: baseline;">email</span><span style="color: #666600; font-family: "monaco"; vertical-align: baseline;">;</span><span style="font-family: "monaco"; vertical-align: baseline;"> </span></span></div>
<div style="direction: ltr; margin: 0pt 0in; text-indent: 0in; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">}</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span></span></div>
<div style="direction: ltr; margin: 0pt 0in; text-indent: 0in; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="color: #000088; font-family: inherit; text-indent: 0in; vertical-align: baseline;"><br /></span></div>
<div style="direction: ltr; margin: 0pt 0in; text-indent: 0in; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #000088; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">function</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">toggleShowAdvanceOptions</span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">()</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">{</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span></span></div>
<div style="direction: ltr; margin: 0pt 0in; text-indent: 0in; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="color: #000088; font-family: inherit; text-indent: 0in; vertical-align: baseline;"> </span></div>
<div style="direction: ltr; margin: 0pt 0in; text-indent: 0in; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #000088; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> var</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">displayAdvanceOptions</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">=</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">document</span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">.</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">getElementById</span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">(</span><span style="color: #008800; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">'</span><span style="color: #008800; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">chkAdvanceOptions</span><span style="color: #008800; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">'</span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">).</span><span style="color: #000088; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">checked</span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">;</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">document</span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">.</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">getElementById</span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">(</span><span style="color: #008800; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">'</span><span style="color: #008800; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">divAdvanceOptions</span><span style="color: #008800; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">'</span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">).</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">style</span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">.</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">display</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span></span></div>
<div style="direction: ltr; margin: 0pt 0in; text-indent: 0in; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> =</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">(</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">displayAdvanceOptions</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">?</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span><span style="color: #008800; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">'block'</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">:</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span><span style="color: #008800; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">'none'</span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">);</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span></span></div>
<div style="direction: ltr; margin: 0pt 0in; text-indent: 0in; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">}</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span></span></div>
<div style="direction: ltr; margin: 0pt 0in; text-indent: 0in; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="color: #000088; font-family: inherit; text-indent: 0in; vertical-align: baseline;"><br /></span></div>
<div style="direction: ltr; margin: 0pt 0in; text-indent: 0in; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #000088; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">function</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">getAdministratorDetails</span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">()</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">{</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span></span></div>
<div style="direction: ltr; margin: 0pt 0in; text-indent: 0in; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #880000; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> // This will come from a REST call.</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span></span></div>
<div style="direction: ltr; margin: 0pt 0in; text-indent: 0in; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #000088; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> return</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">{</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> name</span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">:</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span><span style="color: #008800; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">'Administrator'</span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">,</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">
email</span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">:</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span><span style="color: #008800; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">'administrator@company.com'</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span><span style="color: #666600; font-family: "monaco"; text-indent: 0in; vertical-align: baseline;">};</span><span style="font-family: "monaco"; text-indent: 0in; vertical-align: baseline;"> </span></span></div>
<div style="direction: ltr; margin: 0pt 0in; text-indent: 0in; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="color: #666600; font-family: inherit; text-indent: 0in; vertical-align: baseline;">}</span></div>
</div>
<div>
<div>
<hr />
<span style="font-family: inherit;"><br /></span></div>
</div>
<div>
<span style="font-family: inherit;">Now, above UI will be populated using JavaScript as follows:</span></div>
<div>
<ul style="text-align: left;">
<li><span style="font-family: inherit;">Fetch the administrator details from server, say, using REST calls.</span></li>
<li><span style="font-family: inherit;">To populate the administrator name get hold of the <span> element using the id i.e. lblAdminName e.g. document.getElementById("lblAdminName") or using jQuery.</span></li>
<li><span style="font-family: inherit;">Set the value on the DOM element. Same will be the case for displaying the email address.</span></li>
</ul>
<h3 style="text-align: left;">
<span style="font-family: inherit;">
Checkbox handler</span></h3>
</div>
<div>
<span style="font-family: inherit;">While fetching and populating the administrator details seems trivial, toggling the advance options required some more work:</span></div>
<div>
<ul style="text-align: left;">
<li><span style="font-family: inherit;">Associate a JavaScript function handler for checkbox click. In this example it is toggleShowAdvanceOptions();</span></li>
<li><span style="font-family: inherit;">In this handler find out if the checkbox is checked or not. Depending on this, toggle the visibility of the div displaying advance options.</span></li>
</ul>
<span style="font-family: inherit;"><br /></span>
<br />
<h2 style="text-align: left;">
<span style="font-family: inherit;">
Issues with traditional approach</span></h2>
<div>
<span style="font-family: inherit;">One may feel that with above approach the UI and backend are segregated i.e. UI is in *.html or *.jsp file with HTML markups and backend is in *.js i.e. JavaScript file. But segregation does not mean just separating these out. Let's try to answer a simple question: <i>Who is responsible for how the UI looks and behave?</i> Obviously, *.html defines the UI in terms of look and feel while *.js provides the required data. But wait, the *.js file not just provides the data, it actually pushes the data to UI. Ok, then who is responsible for toggling the display of advance options? Hmmm, *.html hooks the checked event to backend and then backend toggles the visibility. </span></div>
<div>
<span style="font-family: inherit;">To summarize, the responsibility to render the UI is with both *.html and *.js files. The reality is that we cannot avoid such a situation i.e. dynamic content needs to be handled by JavaScript. So how does AngularJS help here? In simple terms, it allows us to define the UI and it's behavior in *.html files while the JavaScript files needs to be responsible for fetching the data or reacting to events. But in no way the JavaScript files will deal with UI elements i.e. no more document.getElementById() or jQuery calls in backend. Let's see how this is possible.</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<h2 style="text-align: left;">
<span style="font-family: inherit;">
Sample UI - AngularJS way</span></h2>
<div style="text-align: left;">
<span style="font-family: inherit;">With AngularJS we are not doing away with *.html or *.js files. Instead of getting directly into AngularJS concepts, let's first see how these files are defined for AngularJS based web application.</span></div>
<h3 style="text-align: left;">
<span style="font-family: inherit;">JavaScript object</span></h3>
<div>
<span style="font-family: inherit;">Let's start with the JavaScript object that will be responsible for populating the data on UI.</span></div>
<div>
<div style="text-align: left;">
<div>
<hr />
<div style="direction: ltr; margin: 0pt 0in; text-indent: 0in; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="text-indent: 0in;"> </span><span style="color: #7f0055; text-indent: 0in;">function</span><span style="text-indent: 0in;"> </span><span style="text-indent: 0in;">SettingsController</span><span style="text-indent: 0in;"> () </span><span style="color: #2a00ff; text-indent: 0in;">{</span></span></div>
<div style="direction: ltr; margin: 0pt 0in; text-indent: 0in; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<div style="margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; word-break: normal;">
</div>
<div style="margin-bottom: 0pt; margin-left: 0.5in; margin-top: 0pt; unicode-bidi: embed; word-break: normal;">
<span style="color: #3f7f5f;"><span style="font-family: inherit;"> // Get this from REST service</span></span></div>
<div style="margin-bottom: 0pt; margin-left: 0.5in; margin-top: 0pt; unicode-bidi: embed; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #7f0055;"> this</span>.administrator = <span style="color: #2a00ff;">{</span></span></div>
<div style="margin-bottom: 0pt; margin-left: 0.5in; margin-top: 0pt; unicode-bidi: embed; word-break: normal;">
</div>
<div style="margin-bottom: 0pt; margin-left: 0.5in; margin-top: 0pt; unicode-bidi: embed; word-break: normal;">
<span style="font-family: inherit;"> name: <span style="color: #2a00ff;">'Administrator'</span>,</span></div>
<div style="margin-bottom: 0pt; margin-left: 0.5in; margin-top: 0pt; unicode-bidi: embed; word-break: normal;">
<span style="font-family: inherit;"> email: <span style="color: #2a00ff;">'administrator@company.com'</span></span></div>
<div style="margin-bottom: 0pt; margin-left: 0.5in; margin-top: 0pt; unicode-bidi: embed; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #2a00ff;"> }</span>;</span></div>
<div style="margin-bottom: 0pt; margin-left: 0.5in; margin-top: 0pt; unicode-bidi: embed; word-break: normal;">
<span style="color: #7f0055; text-indent: 0in;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="margin-bottom: 0pt; margin-left: 0.5in; margin-top: 0pt; unicode-bidi: embed; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #7f0055; text-indent: 0in;"> this</span><span style="text-indent: 0in;">.showAdvanceOptions</span><span style="text-indent: 0in;"> = </span><span style="color: #7f0055; text-indent: 0in;">false</span><span style="text-indent: 0in;">;</span></span></div>
<span style="color: #000088; font-family: inherit; text-indent: 0in; vertical-align: baseline;"></span>
<div style="margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; word-break: normal;">
<span style="color: #2a00ff;"><span style="font-family: inherit;">}</span></span></div>
</div>
</div>
<div>
<hr />
<span style="font-family: inherit;">As seen above the JavaScript object simply has two properties:</span><br />
<ul style="text-align: left;">
<li><span style="font-family: inherit;"><b>administrator:</b> This property will hold the json describing administrator name and email. It should be populated using REST call. For this example, we have hard-coded it.</span></li>
<li><span style="font-family: inherit;"><b>showAdvanceOptions:</b> This property tells if the advance options should be displayed or not. The default value is false.</span></li>
</ul>
</div>
</div>
</div>
<div>
<span style="font-family: inherit;">That's it. No <b><i>getElementById()</i></b> or any DOM manipulation code here. </span><br />
<span style="font-family: inherit;">Now, let's look at the HTML.</span></div>
<h3 style="text-align: left;">
<span style="font-family: inherit;">HTML markups</span></h3>
<div>
<span style="font-family: inherit;">The HTML snippet for the UI will be as follows:</span></div>
<div>
<hr />
</div>
<div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">1: <</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">div</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"> </span><span style="color: #bf4f24; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">ng-controller</span><span style="color: #080808; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">=</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">"</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">myapp.settings.controller as ctrl</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">"</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">>
</span></span></div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">2: <</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">h2</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">>Administrator</</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">h2</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">>
</span></span></div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas";">3: </span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"><</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">label</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">><</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">b</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">>Name:
</</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">b</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">></</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">label</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">></span><span style="color: #080808; font-family: "consolas";"><</span><span style="color: #bf4f24; font-family: "consolas";">span</span><span style="color: #080808; font-family: "consolas";">></span><span style="color: #080808; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">{{</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold;">ctrl.</span><span style="color: #080808; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">administrator.name}}</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"></</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">span</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">></span></span></div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas";">4: </span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"><</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">label</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">><</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">b</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">>Email:
</</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">b</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">></</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">label</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">></span><span style="color: #080808; font-family: "consolas";"><</span><span style="color: #bf4f24; font-family: "consolas";">span</span><span style="color: #080808; font-family: "consolas";">></span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">{{</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold;">ctrl.</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">administrator.email</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">}}</span><span style="color: #080808; font-family: "consolas";"></</span><span style="color: #bf4f24; font-family: "consolas";">span</span><span style="color: #080808; font-family: "consolas";">></span></span></div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas";">5: </span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"><</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">input</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"> </span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">type</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">=</span><span style="color: #0b6125; font-family: "consolas"; vertical-align: baseline;">"checkbox"</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"> </span><b><i><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">ng-model</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">=</span><span style="color: #0b6125; font-family: "consolas"; vertical-align: baseline;">"</span></i></b><span style="color: #0b6125; font-family: "consolas"; font-weight: bold;">ctrl.</span><b><i><span style="color: #0b6125; font-family: "consolas"; vertical-align: baseline;">showAdvanceOptions</span><span style="color: #0b6125; font-family: "consolas"; vertical-align: baseline;">"</span></i></b><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">></span></span></div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas";">6: </span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"><</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">label</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">>Show
Advance Options ({{</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold;">ctrl.</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">showAdvanceOptions</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">}})</</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">label</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">></span></span></div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas";">7: </span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"><</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">div</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"> </span><span style="color: #bf4f24; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">ng-show</span><span style="color: #080808; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">=</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">"</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold;">ctrl.</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">showAdvanceOptions</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">"</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">></span></span></div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas";">8: </span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"><</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">b</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">>Advance
options here...</</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">b</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">>
</span></span></div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas";">9: </span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"></</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">div</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">></span></span></div>
<div style="direction: ltr; margin-bottom: 0pt; margin-left: 0in; margin-top: 0pt; unicode-bidi: embed; vertical-align: baseline; word-break: normal;">
<span style="font-family: inherit;"><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">10:</</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">div</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">></span></span></div>
</div>
</div>
<div>
</div>
<div style="-webkit-text-stroke-width: 0px; color: black; font-family: 'Times New Roman'; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: left; text-indent: 0px; text-transform: none; white-space: normal; widows: 1; word-spacing: 0px;">
<hr style="color: black; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-transform: none; white-space: normal; word-spacing: 0px;" />
<div style="text-align: left;">
<div style="color: black; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-transform: none; white-space: normal; word-spacing: 0px;">
<span style="font-family: inherit;">The markup is same as for traditional approach except that there are additional attributes. But in this case we are just defining how the UI is rendered and behaves in markup itself. Let's look at the markup one by one.</span></div>
<ul style="text-align: left;">
<li style="font-style: normal; font-weight: normal;"><span style="font-family: inherit;">Line 1: </span></li>
<ul style="font-style: normal; font-weight: normal;">
<li><span style="font-family: inherit;">On the <span style="color: #080808; font-family: "consolas"; vertical-align: baseline;"><</span><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">div</span><span style="color: #080808; font-family: "consolas"; vertical-align: baseline;">></span> tag we are specifying which JavaScript object will be responsible for providing the data to this UI.</span></li>
<li><span style="font-family: inherit;"><span style="color: #bf4f24; font-family: consolas; font-weight: bold;">ng-controller</span> is AngularJS specific attribute which defines the JavaScript object to be used.</span></li>
<li><span style="font-family: inherit;">For this discussion, assume that we have a map of name to JavaScript object. Hence, we are asking AngularJS to lookup that map with the name as 'myapp.settings.controller' and attach the JavaScript object to this UI. For now, just ignore how we are registering the object with AngularJS.</span></li>
<li><span style="font-family: inherit;">Also, we are using an alias '<b>ctrl'</b> for that object.</span></li>
</ul>
<li style="font-style: normal; font-weight: normal;"><span style="font-family: inherit;">Line 3: </span></li>
<ul style="font-style: normal; font-weight: normal;">
<li><span style="font-family: inherit;">Here, we have used <span style="color: #080808; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">{{</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold;">ctrl.</span><span style="color: #080808; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">administrator.name}}</span></span></li>
<li><span style="font-family: inherit;">This tells AngularJS that on the JavaScript object attached to this UI, we need to look for a property named <b><i>administrator</i></b>. Since, <b><i>administrator</i></b> value is a json we are further referring to the <b><i>name</i></b> property in it. </span></li>
<li><span style="font-family: inherit;">The value for the <b><i>administrator.name</i></b> property will get populated on UI.</span></li>
<li><span style="font-family: inherit;">If the value changes in backend, then it will also get updated on UI.</span></li>
<li><span style="font-family: inherit;">Same is the case for <span style="color: #080808; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">{{</span><span style="color: #0b6125; font-family: "consolas"; font-weight: bold;">ctrl.</span><span style="color: #080808; font-family: "consolas"; font-weight: bold; vertical-align: baseline;">administrator.email}}</span></span></li>
</ul>
<li style="font-style: normal; font-weight: normal;"><span style="color: #080808; font-family: inherit;"><span style="color: black;">Line 5</span>:</span></li>
<ul style="font-style: normal; font-weight: normal;">
<li><span style="color: #080808; font-family: inherit;"><span style="color: black;">Here, on the checkbox, we have used the <b style="font-family: 'Times New Roman';"><i><span style="color: #bf4f24; font-family: "consolas"; vertical-align: baseline;">ng-model</span></i></b> attribute.</span></span></li>
<li><span style="font-family: inherit;">The value for this attribute is the property on our JavaScript object.</span></li>
<li><span style="font-family: inherit;">It tells AngualrJS that value for this checkbox should be deduced from the showAdvanceOptions property on JavaScript object.</span></li>
<li><span style="font-family: inherit;">In addition to that it also tells that when the checkbox value changes from UI, say, on user click, then the value on JavaScript object should also be updated. This is called <a href="https://docs.angularjs.org/tutorial/step_04" target="_blank">two- way binding</a> in AngularJS.</span></li>
</ul>
<li style="font-style: normal; font-weight: normal;"><span style="font-family: inherit;">Line 6:</span></li>
<ul>
<li><span style="font-family: inherit;">Here we are displaying the value of <b><i>showAdvanceOption</i></b> property on the JavaScript object.</span></li>
<li style="font-style: normal; font-weight: normal;"><span style="font-family: inherit;">When the checkbox is checked then the value becomes true else false.</span></li>
</ul>
<li style="font-style: normal; font-weight: normal;"><span style="font-family: inherit;">Line 7:</span></li>
<ul>
<li style="font-style: normal; font-weight: normal;"><span style="font-family: inherit;">Here we are controlling the visibility of the div using the <span style="color: #bf4f24; font-family: consolas; font-weight: bold;">ng-show</span> attribute of AngularJS.</span></li>
<li><span style="font-family: inherit;">It tells AngularJS that the div should be visible when the <b><i>showAdvanceOptions</i></b> property on the JavaScript object is true else it should be hidden.</span></li>
<li><span style="font-family: inherit;">Now, the <b><i>showAdvanceOptions</i></b> property is true/false depending on the state of the checkbox and hence the div will be visible if checkbox is checked else it will be hidden.</span></li>
</ul>
</ul>
</div>
<div style="text-align: left;">
<h2 style="text-align: left;">
Summary</h2>
<div>
As seen above, using AngularJS the HTML markup defined what data needs to be rendered and how. The JavaScript object, called as controller, only needs to provide the required data to UI. A lot happens internally using AngularJS.</div>
<div>
In order to implement event a simple UI using AngularJS one needs to understand controllers, services, modules etc. But this post should help understand the very basic of how controller can be used to provide data to UI and how the HTML markups drive the UI flow.<br />
In next post we will have a simple MVC application using AngularJS and explore the controllers, services, modules etc.</div>
</div>
</div>
<ul style="text-align: left;">
</ul>
</div>
<div>
</div>
</div>
Ajey Dudhehttp://www.blogger.com/profile/04566439328405849242noreply@blogger.com3tag:blogger.com,1999:blog-6287635317363677137.post-36242847192315834092015-12-01T22:59:00.000-08:002017-06-21T04:00:55.650-07:00Elasticsearch: What you see is NOT what you get !!!<div dir="ltr" style="text-align: left;" trbidi="on">
<div dir="ltr">
Recently, I troubleshooted two common issues with ES queries. Suddenly, the queries stopped returning data using some filter criteria. But when we looked at all the records for the type then the filter should have evaluated to true and returned data. We often forget that ES mostly returns original document which was indexed. But while indexing the fields in document there are analyzers being applied. So the field value we see in document is not exactly same as indexed value. </div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
Consider the following simple example:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjuHOYl-Y5eaGbjj5o7Oxgbm_oYmUP6Zdiv3DRsh2QJ5GKSJmv-y3oqy0Gz2TQQ3KQii25RAxVzFhP5JZLaMCCuczYfzk_T97y6HI0SIjO-YmLFBgKbUOBjBJYlqw9t-uB6BkVdyJIPC33c/s1600/fielddata_fields_01.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="238" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjuHOYl-Y5eaGbjj5o7Oxgbm_oYmUP6Zdiv3DRsh2QJ5GKSJmv-y3oqy0Gz2TQQ3KQii25RAxVzFhP5JZLaMCCuczYfzk_T97y6HI0SIjO-YmLFBgKbUOBjBJYlqw9t-uB6BkVdyJIPC33c/s400/fielddata_fields_01.png" width="400" /></a></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<b><br /></b></div>
<div dir="ltr">
<b><br /></b></div>
<div dir="ltr">
<b><br /></b></div>
<div dir="ltr">
<b><br /></b></div>
<div dir="ltr">
<b><br /></b></div>
<div dir="ltr">
<b><br /></b></div>
<div dir="ltr">
<b><br /></b></div>
<div dir="ltr">
<b>Line 1:</b> We have created an employee record with <b><i>displayName</i></b> as "Ajey Dudhe".</div>
<div dir="ltr">
<b>Line 6:</b> We are retrieving all the employee records.</div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
If you see on the right hand side under <b><i>_source</i></b> the document is returned as it is. But does Elasticsearch see the value of displayName as "Ajey Dudhe". The answer is no. In this example, when the document is inserted then default analyzers are used. By default the string value will be broken into tokens and lower-cased. And this is the value which Elasticsearch sees. In order to have a look at the actual value visible to Elasticsearch we need to use the <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-fielddata-fields.html" target="_blank">fielddata_fields</a> option while fetching the documents as follows:</div>
<div dir="ltr">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhoK5EzyS_SasWhQ7irhzXVuAgSmeiR_HRFZwoit1g34okDdSrqB5JeUIhjpHiQk397UsckyOtXFfQ84cBirFtclBqwfMiefL_-CG0DHF7QhZQ-wQzVCvR8WdXv6fy0xtV-4aMP2949Z5QV/s1600/fielddata_fields_02.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhoK5EzyS_SasWhQ7irhzXVuAgSmeiR_HRFZwoit1g34okDdSrqB5JeUIhjpHiQk397UsckyOtXFfQ84cBirFtclBqwfMiefL_-CG0DHF7QhZQ-wQzVCvR8WdXv6fy0xtV-4aMP2949Z5QV/s400/fielddata_fields_02.png" width="400" /></a></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<br /></div>
<div dir="ltr">
<b><br /></b></div>
<div dir="ltr">
<b><br /></b></div>
<div dir="ltr">
<b><br /></b></div>
<div dir="ltr">
<b><br /></b></div>
<div dir="ltr">
<b><br /></b></div>
<div dir="ltr">
<b><br /></b></div>
<div dir="ltr">
<b><br /></b></div>
<div dir="ltr">
<b><br /></b></div>
<div dir="ltr">
<b><br /></b></div>
<div dir="ltr">
<b><br /></b></div>
<div dir="ltr">
<b><br /></b></div>
<div dir="ltr">
<b>Line 22 on right hand side:</b> Notice that the <b><i>fielddata_fields</i></b> now has values as seen by ES i.e. tokenized and lower-cased. In case the value is not at all indexed by ES then it will not appear under <i><b>fields</b></i>.</div>
</div>
Ajey Dudhehttp://www.blogger.com/profile/04566439328405849242noreply@blogger.com0tag:blogger.com,1999:blog-6287635317363677137.post-21338282530559732852015-04-22T08:06:00.000-07:002017-06-21T04:01:14.729-07:00Elasticsearch: Taming the fielddata<div dir="ltr" style="text-align: left;" trbidi="on">
<h2 style="text-align: left;">
Introduction </h2>
One of the bottlenecks in scaling-up Elasticsearch is <a href="http://www.elastic.co/guide/en/elasticsearch/guide/current/fielddata.html" target="_blank">fielddata</a>. The term <i><b>fielddata</b></i> does not directly refer to data but represents the data structures & caching done by Elasticsearch for doing efficient look-ups. Apart from consuming considerable chunk of memory, <i><b>fielddata</b></i> also impacts queries. The impact is clearly visible when you have millions of documents to query. One of the recommended solutions is to scale-out. But even if you have multiple nodes, you need to make sure every single node is fine tuned otherwise you may have to keep adding considerable number of nodes as the data set increases. In following sections we will discuss how we can optimize the <i><b>fielddata</b></i> usage which, in turn, should help improve the memory usage & query performance for a single node.<br />
<h2 style="text-align: left;">
Use doc_value = true</h2>
Main issue with <i><b>fielddata</b></i> is that it consumes huge memory. Using <a href="https://www.elastic.co/blog/disk-based-field-data-a-k-a-doc-values" target="_blank">doc_value = true</a> on a field in the mapping, tells Elasticsearch to use file system to store <i><b>fielddata</b></i> instead of memory. However, there is currently one limitation on string fields. Only non_analyzed string fields can have this option. What this means is that you cannot use doc_value = true with fields having analyzers like lower-case etc. defined on them. It becomes problem in some cases e.g. if you need to provide case-insensitive search for a field. Again, here the suggested approach it to use <a href="http://www.elastic.co/guide/en/elasticsearch/reference/current/_multi_fields.html" target="_blank">multi-fields</a> mapping i.e. one analyzed and one non_analyzed field. But the moment we have analyzed field we cannot use doc_value = true on it which means that <i><b>fielddata</b></i> for it will be in-memory. To handle this we need to use <a href="http://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-transform.html" target="_blank">transform</a> script. In the transform script you need to do the operations like convert field to lower-case etc. In case you need to use analyzer then find out how you can invoke the analyzers explicitly from your transform script.<br />
While the documentation says using doc_value improves memory usage, it can impact the performance. But in our testing it was observed that the aggregation query performance increased drastically. May be it was due to the fact that now <i><b>fielddata</b></i> was being queried from file system cache.<br />
<h2 style="text-align: left;">
Handle sorting</h2>
<div style="text-align: left;">
Sorting is another feature where <i><b>fielddata</b></i> is used. When sort is specified on a particular field, Elasticsearch needs to find out the terms for the document i.e. it needs <i><b>fielddata</b></i>. So to avoid it we can use <a href="http://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html#_script_based_sorting" target="_blank">script based sorting</a>. The script can take field name as parameter and return the field value from the _source. Even though there is overhead is accessing the source, we observed around 50% improvement using script based sorting.</div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgijk3U86XlLt9ztWoKRBMpE7pnegUOO528Yy1yunz8obaL523ZKIRyT8uDW4yP8Iv3oBXY0CuigZdp9rEAYyviraj_YQBgg2YCbSfR7fvw7l0Qa28eZ4yNQ5iY4Xrrky5NORIglJ6qipPl/s1600/Sort.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgijk3U86XlLt9ztWoKRBMpE7pnegUOO528Yy1yunz8obaL523ZKIRyT8uDW4yP8Iv3oBXY0CuigZdp9rEAYyviraj_YQBgg2YCbSfR7fvw7l0Qa28eZ4yNQ5iY4Xrrky5NORIglJ6qipPl/s1600/Sort.png" /></a></div>
<br />
<h2 style="text-align: left;">
Aggregation</h2>
<div style="text-align: left;">
Aggregation is the main feature where <i><b>fielddata</b></i> is required. Elasticsearch provides <a href="http://www.elastic.co/guide/en/elasticsearch/reference/current//search-aggregations-metrics-scripted-metric-aggregation.html" target="_blank">Scripted Metric Aggregation</a> but using script did not help here. Best is to avoid aggregation queries if not required. For example, use <a href="http://exploring-elasticsearch.blogspot.in/2015/01/get-distinctunique-values-using-search.html" target="_blank">script filter</a> if you need to query for distinct documents.</div>
</div>
Ajey Dudhehttp://www.blogger.com/profile/04566439328405849242noreply@blogger.com0tag:blogger.com,1999:blog-6287635317363677137.post-37598186093872460082015-01-30T04:04:00.000-08:002017-06-21T04:01:33.527-07:00Elasticsearch: Get distinct\unique values using search instead of aggregation - Part 2<div dir="ltr" style="text-align: left;" trbidi="on">
The <a href="http://exploring-elasticsearch.blogspot.in/2015/01/get-distinctunique-values-using-search.html" target="_blank">previous post</a> described how custom script filter can be used to filter out duplicate documents in the result set. However, the approach would not work with multiple shards. To handle this case we need to have some plugin implementation to hook into Elasticsearch where we can filter out the duplicate documents fetched from multiple shards. But before that let's have a quick look at how distributed search works.<br />
<h2 style="text-align: left;">
</h2>
<h2 style="text-align: left;">
Distributed search</h2>
<div>
Elasticsearch documentation covers <a href="http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/distributed-search.html" target="_blank">distributed search</a> very well. Following is the summary:</div>
<div>
<ul style="text-align: left;">
<li>Distributed search is executed in two phases: Query and Fetch phase</li>
<li><b>Query phase</b></li>
<ul>
<li>The coordinating node sends the search request to all the shards.</li>
<li>Each shard executes the search and sends back list of document IDs and the sort values back to the coordinating node.</li>
<ul>
<li>For example, get documents where file.folder = 'c:\windows' sorted by file.name</li>
<li>In above case, say, shard 1 matches three documents with file names as notepad.exe, textpad.exe, wordpad.exe. So the return values would be</li>
<ul>
<li><b>DocID:</b> some_doc_id_1, <b>Sort values:</b> [notepad.exe]</li>
<li><b>DocID:</b> some_doc_id_2, <b>Sort values:</b> [textpad.exe]</li>
<li><b>DocID:</b> some_doc_id_3, <b>Sort values:</b> [wordpad.exe]</li>
</ul>
</ul>
<li>All shards returns similar data back to the coordinating node.</li>
<li>The coordinating node now has list of fields on which sorting needs to be done and also the actual values for these fields per document.</li>
<li>This node then merges all the results into a single list.</li>
<li>The list is sorted using the sort values returned by each shard for each document.</li>
<li>The final list is prepared by applying pagination (from & size) parameters.</li>
</ul>
<li><b>Fetch phase</b></li>
<ul>
<li>Once the final list is prepared the coordinating node then sends the request to get the required documents from each shard using the document IDs i.e. send a multi-get request.</li>
</ul>
</ul>
</div>
<h2 style="text-align: left;">
</h2>
<h2 style="text-align: left;">
Remove duplicate documents in distributed search</h2>
<div>
As seen above if we need to filter out the duplicate documents for distributed search then we need to hook into the QUERY phase. Once the document IDs & sort values are retrieved for all the shards then we need to remove duplicate documents here. The steps to achieve this are as follows:</div>
<div>
<ul style="text-align: left;">
<li>Apart from document Ids & sort values we also need the values for primary keys identifying the document as unique in the result set.</li>
<li>Once we have above information then we can remove duplicate documents using the primary keys. </li>
</ul>
</div>
<h3 style="text-align: left;">
</h3>
<h3 style="text-align: left;">
Passing back the primary key values in query phase</h3>
<div>
In QUERY phase for a document only its ID and sort values are passed back. The _source value is not passed. So only option to pass back the primary key values is using the sort values. For this we can use custom script for sorting. This custom script will be similar to the custom filter mentioned in <a href="http://exploring-elasticsearch.blogspot.in/2015/01/get-distinctunique-values-using-search.html" target="_blank">previous post</a>.</div>
<div>
<ul style="text-align: left;">
<li>The script will take primary key field names as parameter.</li>
<li>The script will form a key by concatenating the values of primary key fields.</li>
<li>In the search request we will add this script based sort criteria at the end of existing sort fields, if any.</li>
<li>This way, for each document, we will get the key value during the QUERY phase itself on the coordinating node.</li>
</ul>
<h3 style="text-align: left;">
</h3>
<h3 style="text-align: left;">
Removing duplicate documents</h3>
</div>
<div>
Now, we need to hook into the QUERY phase using plugin approach. Thanks to the reference plugin implementation <a href="https://github.com/jprante/elasticsearch-simple-action-plugin" target="_blank">here</a>. By looking at this sample plugin I was able to deduce that the final sorting of docs from all shards happens in org.elasticsearch.search.controller.SearchPhaseController's sortDocs() method. This class is used deep inside and there is no direct way to hook into it. So I had to follow the same approach as in the reference plugin implementation i.e. to implement a custom action. In the plugin, the required classes and methods were extended\overridden. Finally, the class extending SearchPhaseController overrides the sortDocs() method. In this we remove the duplicate documents from across the shards and then call the original sortDocs() method.</div>
<h2 style="text-align: left;">
</h2>
<h2 style="text-align: left;">
Source code</h2>
<div>
<a href="https://github.com/ajeydudhe/elasticsearch/tree/master/custom-scripts" style="background-color: #fefdfa; color: #7d181e; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; font-size: 13px; line-height: 18.2000007629395px; text-decoration: none;" target="_blank">https://github.com/ajeydudhe/elasticsearch/tree/master/custom-scripts</a></div>
<div>
You can start with the junit <a href="https://github.com/ajeydudhe/elasticsearch/blob/master/custom-scripts/src/test/java/my/elasticsearch/DistinctDocumentsSearchIntegrationTests.java" target="_blank">tests</a>.</div>
<h2 style="text-align: left;">
</h2>
<h2 style="text-align: left;">
Limitations</h2>
<div>
<ul style="text-align: left;">
<li>This approach extends the ES classes and methods which may change in future.</li>
<li>One major limitation is that the totalHits returned by the search request will not be accurate when pagination is used. This is because each shard will return the totalHits as the total documents matching the filter but actual documents returned can be less if pagination is used e.g. if total matches could be 100 but pagination criteria could be to return only 10 results. In this case the totalHits will be 100. Now, we do not know the remaining 90 documents are duplicate or not. But this should not be an issue if we can use the totalHits as an approximate count.</li>
</ul>
</div>
</div>
Ajey Dudhehttp://www.blogger.com/profile/04566439328405849242noreply@blogger.com2tag:blogger.com,1999:blog-6287635317363677137.post-69800452346770267952015-01-10T20:23:00.000-08:002017-06-21T04:01:48.318-07:00Elasticsearch: Get distinct\unique values using search instead of aggregation - Part 1<div dir="ltr" style="text-align: left;" trbidi="on">
<h2 style="text-align: left;">
Problem statement</h2>
<div style="text-align: left;">
While dealing with NoSQL datastores the key aspect for schema design is de-normalization or, in other words, defining your schema as per the query requirements. But you cannot keep on de-normalizing for each and every use case. One of such examples is fetching unique records in a query. Relational databases handle this use case by providing DISTINCT or equivalent keyword support. Unfortunately Elasticsearch does not have such built-in support.</div>
<div style="text-align: left;">
<br /></div>
<h2 style="text-align: left;">
Use case</h2>
<div style="text-align: left;">
For example, say, you have a files type which store file information as well as information of the machine on which the file is seen i.e. file_hash, file_name, file_folder, file_size, machine_name, machine_ip etc.With above type defined in ES we can have following queries around it:</div>
<ol style="text-align: left;">
<li>Given a file_hash get all the machines on which it is seen.</li>
<li>Given a file_hash get all the unique combination of (file_name & file_folder) across all machines. </li>
</ol>
One of the solutions is to de-normalize i.e. create additional types in ES to store the data such that we get unique records for a given file_hash using normal search request.<br />
Other common approach is to use aggregation. An aggregation returns count of unique terms and we can also use nested aggregations to get unique values on multiple fields as required in #2.<br />
Another option is to use <a href="http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-script-filter.html" target="_blank">Script Filter</a> support from ES described below.<br />
<div style="text-align: left;">
</div>
<div style="text-align: left;">
<br /></div>
<h2 style="text-align: left;">
Using Script Filter</h2>
<div style="text-align: left;">
Elasticsearch supports using custom scripts as filters. To handle the above use case we can use the script support as follows:</div>
<ul style="text-align: left;">
<li>Define a custom script filter. For this discussion assume it is called AcceptDistinctDocumentScriptFilter</li>
<li>This custom filter takes in a list of primary keys as input.</li>
<li>These primary keys are the fields whose values will be used to determine uniqueness of records.</li>
<li>Now, instead of using aggregation we use normal search request and pass the custom script filter to the request.</li>
<li>If the search already has a filter\query criteria defined then append the custom filter using logical AND operator.</li>
<li>Following is example using pseudo syntax</li>
<ul>
<li>if the request is: </li>
<ul>
<li><i><b>select * from myindex where file_hash = 'hash_value'</b></i></li>
</ul>
<li>then append the custom filter as: <i><b> </b></i></li>
<ul>
<li><i><b>select * from myindex where file_hash = 'hash_value' AND AcceptDistinctDocumentScriptFilter(params= ['file_name', 'file_folder'])</b></i></li>
</ul>
</ul>
</ul>
<div style="text-align: left;">
</div>
<h2 style="text-align: left;">
<b>Custom filter implementation</b></h2>
<div style="text-align: left;">
<ul style="text-align: left;">
<li>The filter derives from <i>org.elasticsearch.script.AbstractSearchScript</i> </li>
<li>override the run() method.</li>
<li>Also define the factory for creating the instance of the custom script.</li>
<li>From the factory pass the list of primary keys to the custom filter. </li>
<li>The run() method is called for every record matching the original filter\query criteria. </li>
<li>We get the values for the primary keys from the current document. </li>
<li>We concatenate the values to form a key to be used to determine if the document was already seen. For example, say, we use java Set<String>. </li>
<li>If the set.add() returns true then it means the document is seen for first time and we return true from the run() method. </li>
<li>However, if set.add() returns false then the document was already in set and hence we return false. Returning false will cause the filter\query criteria to fail and the record will be rejected. </li>
</ul>
</div>
<div style="text-align: left;">
<div style="text-align: left;">
</div>
<h2>
Pros & Cons</h2>
<ul style="text-align: left;">
<li><b>Pros</b></li>
<ul>
<li>Works with existing filter and hence on filtered data.</li>
<li>Is fast compared to using aggregation function.</li>
<li>Takes care of pagination i.e. pagination is applied on filtered unique results.</li>
</ul>
<li><b>Cons</b></li>
<ul>
<li>Since we are storing key values in the script memory usage will be more. But this should not be an issue when we use filter along with pagination.</li>
</ul>
</ul>
<div style="text-align: left;">
<br /></div>
<h2>
Limitation</h2>
<div style="text-align: left;">
Above approach will work only with single shard and not with multiple shards. For example, say there are 3 shards and each shard returns following unique values with page size as 3:</div>
<ul style="text-align: left;">
<li>Shard 1: [A, B, C]</li>
<li>Shard 2: [P, B, R]</li>
<li>Shard 3: [X, C, Z]</li>
</ul>
Now when the results from all the shards are merged and sorted the result set would be [A, B, B, C, C, P, R, X, Z] and if we request for only 3 results in the page then we will get [A, B, B] i.e. duplicate results.<br />
This can be handled by writing some ES plugin which hooks into the phase where results are merged from different shards. Next POC will be around this.<br />
<div style="text-align: left;">
<br /></div>
<h2>
Source code</h2>
<div style="text-align: left;">
<a href="https://github.com/ajeydudhe/elasticsearch/tree/master/custom-scripts" target="_blank">https://github.com/ajeydudhe/elasticsearch/tree/master/custom-scripts </a></div>
<br />
<h2 style="text-align: left;">
<b>Summary</b><i><b></b></i></h2>
<div style="text-align: left;">
Even though currently above approach has limitation of multiple shards it is worth exploring this approach because of the performance gain seen when no. of documents are in millions. Also, another observation was around the field data (or the cached data). In case of aggregation, it seems like ES tries to load in memory almost all the fields specified in aggregation while the field data memory using filter is much low. <i><b><br /></b></i><br />
<h2 style="text-align: left;">
Part 2</h2>
<div>
<a href="http://exploring-elasticsearch.blogspot.in/2015/01/get-distinctunique-values-using-search_30.html" target="_blank">Part 2</a> covers the approach to handle distributed search request.</div>
</div>
</div>
</div>
Ajey Dudhehttp://www.blogger.com/profile/04566439328405849242noreply@blogger.com1