Thinking Big Data with Rules and Hadoop

CB

Chris Berg

07/11/2016

Recently I explored some of the decision management touch points with Hadoop. The most obvious was the classic ETL scenario where folks (analysts and data scientists) explore batch scenarios across large datasets—really big ones. While there are so many ways for our tooling to integrate, I became curious about JavaScript specifically. It’s generally fast and because it operates without the traditional “cold start” problem it scales really well in an instant. I imagined dropping our JavaScript decision inside a MapReduce artifact wrapped with something like Rhino. With just a few extra steps, Hadoop processes may operate with the benefits of a business decision and no service calls are required. In fact, with a little extra, it’s easy to imagine putting the parts and pieces together for a full Hadoop lifecycle:

  1. Selecting the decision you need from the catalog.
  2. Transforming the decision into a JavaScript artifact.
  3. Integrating your JavaScript decision and running it (with full automation).
  4. Making rapid changes to the decision and doing it all over again.

Just imagine the Data Analyst happily working with big data and decisions without friction.

InRule for JavaScript | inrule said: "[…] couple of weeks ago we announced availability of InRule® for JavaScript through an early adopter program. To our knowledge this is […]".
Dynamic Surveys in Dynamics CRM Part II: Managing Dependencies | inrule said: "[…] this post makes extensive references to my preceding post, Surveys in Dynamics CRM Part I: Don’t Be a Monkey With Your Survey. If you are the type of person who likes to get the full story, you should go back and read the […]".
Josh Elster said: "In the time since this post was published, I've added (via a PR) full Windows Containers example code, DOCKERFILEs and scripts. They share common ancestry with the GH Gists I posted earlier, but are more up-to-date and tested. Here's a direct link to that sub-folder: https://github.com/InRule/Samples/tree/master/Developer%20Samples/WindowsContainers Now that our Samples repository is public, feel free to file Issues if you have problems or features you'd like to see implemented. If you have something you'd like to contribute, then by all means do so! The contributor guidelines and process are listed in the root of the repository. Enjoy! I'll update the body of the post as well with this link.".