Crowd Sourcing Reflective Intelligence
Will Hayes
Search has quickly evolved from being an extension of the data warehouse to being run as a real-time decision processing system. It is also increasingly being used to gather intelligence on multi-structured data, leveraging distributed platforms such as Hadoop in the background.

This class will provide details on how search engines can be abused to use not text, but mathematically derived tokens to build models that implement reflected intelligence. In such a system, intelligent or trend-setting behavior of some users is reflected back at other users. More importantly, the mathematics of evaluating these models can be hidden in a conventional search engine like Solr, making the system easy to build and deploy.

The class will describe how to integrate Apache Solr/Lucene with Hadoop. Then we will show how crowd-sourced search behavior can be looped back into analysis and how constantly self-correcting models can be created and deployed. Finally, we will show how these models can respond with intelligent behavior in real time.

This class is sponsored by LucidWorks.

Level : Intermediate