So although I should know better here are my three predictions for The Decision Model (TDM) in 2012.

The Decision Model Prediction 1

By the end of 2012 a low cost entry-level TDM modelling tool will exist that will enable modellers to create and validate that their TDM models comply with the TDM 15 principles. There is currently only one tool that has this capability and that is the heavy-weight Sapiens DECISION tool.  DECISION is designed for enterprise TDM modelling, life-cycle management and governance.

My reasons for an entry-level low cost TDM modelling tool are:

• There are a large number of small to medium size companies who in the coming months and years would like to use TDM but who will not be able to afford heavy-weight tools such as Sapiens DECISION. Using Excel and Visio for TDM modelling is better than nothing but a modelling tool with proper validation would be much better.

• Many departments in many large companies may still like to experiment  with TDM using a low-cost entry tool to get started and after the success of an initial TDM project move up to governance products such as Sapiens DECISION.

• Having a low-cost or open source TDM modelling tool will empower many thousands to experiment on their own and drive the bottom-up adoption of TDM within organisations of all sizes.

The Decision Model Prediction 2

By the end of 2012 a low cost TDM model translator will exist that will be able to automatically convert high-level graphical TDM models into actual “business rules code” for a number of business rules engines. This tool will enhance the productivity of business rules programmers and ensure that Decision Models are converted without error into code that can execute within different rule engines.

For example I have an Entity Relationship Diagram tool on my desktop that I can use to model the relationships between entities. Then when I want to convert an ERD model to a relational database schema for a particular DBMS all I need to do is to select the target DBMS (e.g. Oracle, MySQL, etc) and the required database schema will be optimised and generated without any additional programming effort required.  Of course additional tuning could be conducted by the database designer.  I am looking for a similar capability with TDM models.

This TDM transformation tool should be able to generate code for both open source  rules engines such as Drools as well as proprietary rules engines from TDM models.

Of course this tool could be an option within the TDM prediction 1 tool.

This tool will enhance the value of enterprises using TDM to model their business logic in a technology independent way since when required the same TDM model can be automatically converted into any number of different types of rules engines.  Migrating between different rules engines will a a simple matter also being able to deploy the same TDM models to multiple environments whilst enhancing programmers productivity would be an additional plus for The Decision Model.

The Decision Model Prediction 3

From my perspective Business Decision Management = BPMN + TDM. It is therefore my prediction that by end of 2012 that a low-cost entry tool will exist that will enable business analysts and TDM modellers to model TDM  and BPMN models in an integrated modelling environment.

Lets not forget that actions required to be executed on completion of a TDM decision should be executed within a BPMN process model. See The Decision Model Book by Barbara von Halle and Larry Goldberg. Also for more information on the integration of TDM with BPMN process models see the BPMN guru Bruce Silver’s blog post http://www.brsilver.com/2010/01/05/integrating-process-and-rules-part-2/

Of course this integrated TDM/BPMN modelling option could exist as part of TDM modelling tool of TDM prediction 1.

How to make my TDM predictions come true in 2012

When I was growing up I used to love reading those American “how to do it”  (and they were always American) books on how to build a plane in your back garden; how to cast an car’s engine block and my favourite which I went on to do…How build an 8” mirror reflector telescope”.

Now as 2012 progress and  if you get tired of waiting for your favourite tool vendor to “see the light” and make one of my TDM 2012 predictions come true. Then you can always do it your self and here is my 7 steps for making one or more of my TDM predictions come true in 2012.

Step 1

Decide if your tool is going to be open source or a commercial offering. It is possible to have combined version. By that I mean that one could have a open source version of the tool that has some restrictions with a commercial version with no restrictions.  Also determine if all you want to implement is the TDM notation or if you want to implement all the 15 principles and the certification level you are seeking to achieve.

Step 2

Read the KPI The Decision Model Patent Usage policy statement at http://www.kpiusa.com/index.php/Patent/tdm-patent-use.html Contact KPI to arrange the required TDM license if you are looking to sell your tool or provide it as an open source product.

Step 3

Learn how to develop Domain Specific Languages (DSLs). DSLs are graphical and textual modelling languages that can be used to create graphical and textual modeling tools for any domain such as TDM and for transforming one model into another model or a model into code.

Three very good “how to do” books you should read. The first book is a “must read” book.  “Eclipse Modeling Project : A Domain-Specific Language (DSL) tool-kit” by Richard C Gronback.  This is a brilliant and highly recommended book and does not assume any prior DSL modelling experience.  The second book is “EMF Eclipse Modeling Framework” Second edition by Dave Steinberg, Frank Budinsky, et al. expands on the subject in the Cronback book and finally “Eclipse : Building Commercial-Quality Plug-ins” by Eric Clayberg. This is a required book on how to produce a quaility DSL as a plugin.

Step 4

Decide what tools are you going to use to create your DSLs and which of my 3 predictions are you looking to implement. My personal preference is to use tools based on the Eclipse Modeling Project which will enable you to create open source or commercial plugins for the hundreds of thousands of programmers who are using Eclipse for all their programming and modelling work.

Also most of the Eclipse toolkits that can support the development of DSLs are free. Have a look at http://www.eclipse.org/modeling/ and http://eclipse.org/modeling/emf/ There are a very large number of Eclipse modelling projects so a careful selection will be required and will depend on which of the three predictions that you want to implement (or all of them).  Eclipse modelling tools can be downloaded from http://www.eclipse.org/downloads/packages/eclipse-modeling-tools-includes-incubating-components/galileosr1 and there are other eclipse based implementation such as TOPCASED http://www.topcased.org/

Step 5

Develop a TDM metamodel. It should incorporate not only what is in the TDM book and in The Decision Model patent but all the additional TDM modelling features plus determine how you plan to perform the validation of the 15 TDM principles if this latter feature is to be incorporated within your tool.

Step 6

Create the abstract syntax, concrete syntax, graphical tool editors, TDM model to text generators, 15 principles validation suite, all the additional modelling editors required for the TDM prediction tool that you want to develop.

Step 7

Package your TDM TDM Modelling tool as a plugin ready for use in existing Eclipse based commercial or open source developers toolkit or as a stand-alone TDM modelling tool. Start selling your tool!!

So there you have it. I have told you what to do to make my three TDM predictions come true. What more can your ask from a prophet?

Of course I will also be monitoring the tool vendors and if I do not see any indications of work in this area soon then we at Azinta Systems may be forced to “throw our hat in the ring” to try and make at least one of our 2012 TDM predictions come true!

Suleiman Shehu

]]> http://www.azintablog.com/2012/01/08/three-predictions-for-the-decision-model-in-2012/feed/ 0 http://www.azintablog.com/2011/12/24/the-decision-model-ip-trap-rebuttal/ http://www.azintablog.com/2011/12/24/the-decision-model-ip-trap-rebuttal/#comments Sat, 24 Dec 2011 01:22:16 +0000 Suleiman Shehu http://www.azintablog.com/?p=303 There has been an intense debate on the Drools and jBPM blog with two posts by Mark Proctor  http://blog.athico.com/2011/11/decision-model-ip-trap.html and http://blog.athico.com/2011/12/decision-model-ip-trap-part-deux.html.

At the same time another intense debate on the TDM patent was going on in “The Decision Model” Linkedin group. Mark saw fit to place of copy of the complete discussion thread, of a closed Linkedin group, on his public website against what I feel is the spirit of a closed Linkedin group of which he is a member.

Personally I respect Mark’s contributions to the open source community, the recent software patent that he was awarded  (US Patent 7904402) and his significant contributions to the Drools project at Red Hat. And whilst Mark has sought to interpret some of my arguments in the Linked discussions as trying to “belittle him” this was not my intention.

It is not my intention in this post to provide a point-by-point rebuttal of all Mark’s arguments. What I hope to do is to outline the central plank of Mark’s arguments and that is his position on software patents, and, then look at some of the key arguments he has used to support his assertion that The Decision Model IP is a trap.

The world can be divided in to three groups, from a patent perspective; those who believe passionately that software patents should be abolished; those who believe that software patents should or will continue to exist, and, those who do not care either way.

Now Mark believes passionately that software patents should not exist. He also believes that until the day where software patents are abolished world-wide that the only people who have the moral right to own patents are open source software companies who will naturally use their patents defensively. Because they are open source companies.

There is nothing wrong in Mark holding this passionate belief.

However his position is not the only moral position. Other refutable non-open source IT companies own software patents and they also claim a moral position in that they say that they hold these software patents only to use defensively to protect their investments and ideas from misappropriation and to protect their company from aggressive patent actions from other companies.

So why should KPI be prevented from owning a USA patent if it is ok for the “big boys” to do so?. Clearly the US Patent office believed that TDM patent owners had merit in their application for a patent and awarded them the patent. In fact the sales of the biggest IT services and software companies are mostly based on proprietary patent-protected products. Reputable companies such as IBM, HP, Oracle, Apple, Tibco, Google, etc. individually hold many thousands of software patents. So software patents are not going to disappear from this world any time soon. There are too many powerful global companies supporting software patents to prevent this from being a realistic possibility.

Now it appears that Mark prime problem with The Decision Model (TDM) is that a patent was awarded to Larry Goldberg and Barbara von Halle the authors of “The Decision Model” book and co-founders of KPI on the 6^th December 2011. Mark believes that this TDM patent should be made available to the wider community on an Apache licence because I believe (but I cannot prove this belief) that Mark would have liked to integrate TDM with Drools in some way and therefore argues that an open source project should not be encumbered with any software patents.

Now this is strange position to take because Drools is written in Java and there are over 2,500 Java patents owned by Sun, which is now owned by Oracle, and this was before Oracle recently granted OpenJDK an GPLv2 licence to use some of its Java patents. So it can be shown that before OpenJDK Drools was written in Sun’s  patent encumbered Java and Sun was not an open source company. So if Drools could work with Sun’s Java then why not KPI’s TDM?

However the open source Java patent grant from Oracle to OpenJDK have many exceptions including that OpenJDK must comply fully with Java Language Specification and that no supersets, subsets and other modified Java versions are permitted and one must pass all test suites 6 moths before release of a beta version, etc. In fact Oracle has now taken legal action against Google in August 2010 for breaching its Java patent in the development of the Google Android operating system despite the existence of an open source Java.

The point that I am trying to make is that Red Hat and the Drools project have used software that was protected by patents, namely Java originally. So why should Mark make such a fuss now over the TDM patent?

I think that Mark’s counter argument here would be that he believes that the TDM patent owners would not grant usage licences on fair terms. I argued in the TDM LinkedIn group that whilst this may have been a reasonable concern that he should wait a short while for the owners of the TDM patent to produce a public patent usage statement.

However on 22 December 2011, just a short 2 days after the discussion started on the TDM LinkedIn group, KPI released its public statement on it’s website (http://www.kpiusa.com/index.php?option=com_content&view=article&id=170 ), a copy of which is below:

Statement by KPI and the Patent Holders in connection with U.S. Patent # 8,073,801

Objectives of the Patent Policy

• To share the ideas behind The Decision Model in an orderly way,

• To protect its rigor, hence its reputation, and

• To ensure that we are able to evolve what we started without risking an infringement of someone else’s patent.

Policy Statements

• Individuals, practitioners, and company/organization employees who wish to use TDM in practice are free to do so. This is aimed at practitioners selling TDM services and employees practising TDM within their organizations, as well as organizations adopting TDM in projects. This is a public commitment, and we will post the terms of the end-user license, on the website. We have a program of Decision Model Analyst certification, for which all decision modelers are eligible, details of which may be found -> here. End users are also free to incorporate TDM in any software that they use.

• Those who wish to sell software incorporating TDM, may choose among the following required licenses:

  • Vendors who wish to include only TDM notation without TDM principles can obtain a royalty-free license (e.g., MS/Visio stencils, MS/Excel templates, modeling tools etc.).

  • Vendors who wish to incorporate the TDM with all the covered methods and theory (e.g. 15 Principles) must obtain a license with fair royalties, which includes software certification as part of the license. This license includes other advances and methods in addition to those included in the book and the patent.

  • Vendors who provide Open Source Software, and who wish to incorporate TDM can obtain a royalty-free license for Open Source software. There will be a certification fee and process for Open Source vendors who desire this optional software certification. here

• All are encouraged to contact us for details. Please click ->  here

Link to United States Patent: 8,073,801. Please click ->

It is very clear from the above statement that there is nothing to prevent Open Source vendors from incorporating TDM on a royalty free basis. Certification is optional. The rights of the other stakeholders, namely, TDM end-users, TDM consultants and TDM product vendors are also clearly defined on fair terms and at this stage I believe that the patent owners of TDM has done nothing to restrict the emergence of the TDM industry or to create a TDM IP trap

Conclusion

At the end of the day the really important things is does The Decision Model provide value for customers? Clearly  some people must believe it does provide value because if it were not so then all this passionate debates on the TDM patent would not exist.

People clearly see value in the TDM methodology and the intense patent debates are only saying to me that some people are concerned that the patent would prevent them from using the TDM within their software or why bother arguing about the TDM patent?

For those who would like to understand how TDM enhances the integration of BPMN with business logic (rules) should read the post by the BPM guru Bruce Silver http://www.brsilver.com/2010/01/05/integrating-process-and-rules-part-2/

Now Jacob in the TDM Linkedin group responded to the release of the TDM Patent Usage announcement by KPI (and I quote this only because Mark has copied the entire thread of the TDM group and placed it in the public domain and also because it was Jacob whose valid question had initiated the TDM LinkedIn group patent discussions) with his final post as follows:

Barbara & Larry,

Thank you for your clarification. It is good to know that Open Source vendors “who wish to incorporate TDM can obtain a royalty-free license for Open Source software” with “optional software certification”.

Jacob

Let me here say that I believe that Jacob Feldman has made an enormous and significant contribution to TDM by the incorporation within OpenRules of substantial technical enhancements to support  the execution of TDM models and the evolving new TDM features. I am very impressed with how Jacob have merged TDM with constraint optimisation to provide solutions to difficult constraint optimisation problems. I would urge all to read his brilliant paper “Representing and Solving Rule-Based Decision Models with Constraint Solvers” http://openrules.com/pdf/RuleML2011.JacobFeldman.pdf

Clearly this paper shows that the TDM patent has not restricted the evolution of the usage of TDM within new application domains and therefore is not an IP trap.

So Mark if KPI TDM patent usage rights statement appears to be acceptable to Jacob at OpenRules – an open source decision management company www.openrules.com ) what is there to prevent you from using TDM within Drools?

]]> http://www.azintablog.com/2011/12/24/the-decision-model-ip-trap-rebuttal/feed/ 0 http://www.azintablog.com/2011/09/17/introduction-to-the-decision-model/ http://www.azintablog.com/2011/09/17/introduction-to-the-decision-model/#comments Sat, 17 Sep 2011 14:52:29 +0000 Suleiman Shehu http://www.smedirector.com/?p=298 The Decision Model (TDM) is one of those disruptive technologies that come along once in a while that has significant implications for business agility, designing business applications, operational decision making, business rules, process models, business requirements, business architecture and regulatory compliance – to name but a few.

The Decision Model is based on seminal  book The Decision Model: A Business Logic Framework Linking Business and Technology by Barbarba von Halle and Larry Goldberg. Both Larry and Barbara are Managing Partners at Knowledge Partners International (http://www.kpiusa.com). KPI is an acknowledged world leader in Business Rules since its inception in 1996.

In fact I have got so excited about this technology that I have found a way to link The Decision Model with GPUs (Graphical Processing Units – one of my favourite topics for supercomputing) for what I am calling Accelerated Business Decisions but more about that in a subsequent post.

The Decision Model defines a new way to model business rules and business logic using a well defined technology independent structure that is based on the inherent nature of logic and which has been extended by normalisation and integrity principles.

The best way to gain an insight into the motivation behind TDM is to consider data modelling before the Relational Model. At that time data modelling was based on the technology used to implement data and its interrelationships such Network (e.g. IDMS Codasyl), Hierarchical (e.g. IBM IMS) and Extended Network (e.g. MDBS III).  However this was to change when E. F. Codd developed the Relational Model and introduced the Three Normal Forms that permitted data to be modelled independently of implementation technology using the inherent relationships within data.  This enabled Relational Data Models to become real corporate assets independent of the applications using the data.  In the same way TDM is revolutionising business rules and business logic within the enterprise.

The key concepts within The Decision Model can be viewed from a top-down perspective as follows:

1. Identify the relevant business decisions within a business domain.  Typically a business decision is a conclusion that a business arrives at through business logic and which it wants to manage.
2. Now for each business decision identify and model the set of rule families and their interrelationships that contains the business logic required to reach each business decision.
3. These rule families have a passing resemblance to decision tables however these are no ordinary decision table and must follow 15  Decision Model Principles which are split into Structural Principles (determine structural simplicity); Declarative Principles (provide declarative structure) and Integrity Principles (providing optimal logical integrity)

These 15 Principles ensure that the Decision Model is aligned with its business purpose and introduces the concept of normalisation (TDM defines 1NF, 2NF, 3N similar to the Relational Model) that ensures that there are no logical errors within the business logic.

A Rule Family is a two dimensional table with the following characteristics:

1. The is one and only one conclusion fact type for example “Person likelihood of defaulting on a loan”
2. Can have as many conditions as required in a row (even zero)
3. All conditions are Anded together
4. There are no ORs, ELSEs, BUTs, OTHERWISE – these are what has caused serious logical errors in other approaches to modelling business logic.
5. There are NO actions within the conclusion (Actions are modelled and implemented at the BPM layer – see forthcoming blog post on The Decision Model & BPM) however Messages are available to pass meta-information as required.

Now interrelationships between Rule Families are created where the conclusion of one Rule Family is a condition within another Rule Family and these interrelationships are also required to follow the Decision Model  Inferential Integrity Principles.

Since a picture is said to be worth a thousand words TDM has defined a graphical notation that reduces the complexity of understanding and modelling large decision models – see the diagram below for a partial TDM model using the TDM graphical notation.

The blue  octagon represents the business decision “Determine Policy Renewal Method” (from the insurance domain) and it is linked to a rule family whose conclusion is the same as the business decision for this business logic model.

Rule families are represented by green tabbed rectangles and the rule family conditions are listed within.  The conditions that are between main line and the dashed line represent those conditions that are inferentially linked to other rule families – because these conditions are conclusions within the other rule families. The diagram shows that the Policy Pricing Within Bounds condition within the Policy  Renewal Method rule family is linked to the Policy Pricing Within Bounds rule family and is showing by a linking relationship between the two rule families.

Those conditions below the dashed line represents  conditions that do not logically depend on the conclusions from other rule families.

The actual rules within a rule family is shown by the yellow table in the diagram.  There is a lot more to the Decision Notation that shown in this diagram and also a decision model diagram must also comply with the 15 Principles. The diagram below shows the high level Decision Model for the “Determine Policy Renewal Method“.

To help model and manage large enterprise  TDM projects, including full life-cycle management, tools such as DECISION from Sapiens (http://www.sapiensdecision.com/) offer an excellent solution.  Other TDM tool vendors are  New Wisdom Software (http://www.newwisdomsoftware.com/) and Open Rules(http://www.openrules.com) are also available.  A subsequent blog post will explore these tools in greater detail.

How to experiment and learn The Decision Model
Over the coming years The Decision Model is expected to have a similar impact on business logic as the Relational Model had on data.  It is not possible in a short post such as this to outline the full power of TDM however I would recommend reading a primer on The Decision Model, a version of which can be found at http://openrules.com/docs/DecisionModelPrimer.htm.

Also there is a Decision Model Live primer version that uses the open source OpenRules to demonstrate the key concepts of the Decision Model at http://openrules.com/decision_model_primer.htm. You can download an evaluation of the open source Business Decision Management System from OpenRules  (http://openrules.com/download) and run through the Decision Model Live Primer.

However to truly understand TDM and its impact on IT and the business you should read “The Decision Model” book and it is available on Amazon at http://www.amazon.co.uk/dp/1420082817

Getting The Decision Model Professional Services

KPI has developed a number of TDM consultancy services, TDM best practices, training and certification services and KPI as appointed a number of certified consulting partners who can provide  KPI Decision Model FirstSTEP, KPISTEP, STEPment consulting services outside the USA and they are:

• Azinta Systems (http://www.azinta.com)
• Enterprise Design (http://www.enterprise-design.eu/de/)
• Rule Management Group (http://www.rulemanagement.com)

Forthcoming Decision Model Posts
Over the coming weeks I will be posting articles on the following TDM topics:

1. The Decision Model and Business Process Modelling
2. The Decision Model and Complex Event processing
3. The Decision Model and GPUs – Accelerated Business Decisions
4. The Decision Model and Regulatory Compliance
5. The Decision Model and APADO Business Agility Platform

Posted by Suleiman Shehu, CEO, Azinta Systems  www.azinta.com

Now a new generation of CUDA GPUs based on the Fermi (see figure 1) and the Kepler (Kepler is due in 2011) have created the potential for a new disruptive technology for “Big data” analytics based on the use of much smaller hybrid CPU-GPU clusters.

These small – medium size hybrid CPU-GPU clusters will be available at 1/10 the hardware cost and 1/20 the power consumption costs, and, deliver processing speed-ups of up to 500x or more when compared with regular CPU clusters running Hadoop MapReduce.  This disruptive technological advance will enable small business units and organisations to compete with the much larger businesses who can afford to deploy very large Hadoop CPU clusters for “Big data” analytics.

Figure 1: Nvidia Next Generation Fermi GPU with 512 processing cores

In an earlier post, “GPU and Large Scale Data Mining” I outlined the substantial performance speed-up that have been achieved using single GPU node workstations for data mining. I also posed the question “What happens if you wanted to scale up to a cluster of GPUs?“  I shall now attempt to answer this question with the context of “Big data” analytics.

However before discussing GPU clusters and “Big data” analytics it would be useful to give a quick overview of MapReduce as implemented by Hadoop (an open source project that has implemented the concepts of Google MapReduce – the originator of MapReduce) and see how it has been applied to “Big data” processing.

Hadoop MapReduce Architecture

The Hadoop MapReduce Framework is designed to run on large clusters of hundreds, or thousands, of commodity machines.  The Hadoop MapReduce runtime partitions the input data, schedules the programs execution over a set of machines and handles all inter-machine communications, machine failure and removes from the programmer the need to handle these issues.

The programmer does not have to know anything about distributed and parallel programming. He writes just two programs for each job, a Map program that takes the input data assigned to it by the MapReduce runtime; process the input data and emits intermediate results. These intermediate results are initially stored to disk and are subsequently processed by the Reduce program (written by the programmer).  The Reduce program collects the intermediate results to produce a final output result for the job.

The MapReduce model helps reduce software development costs because the programmer’s software is not required to address distributed parallel programming concerns.  This is all handled by the MapReduce runtime.

The simplicity of the programming model and the quality of services attached to the Hadoop MapReduce ecosystem has created a lot of enthusiasm and many success stories amongst the “Big data” processing communities.

Constraints of the Hadoop MapReduce Model

Whilst the Hadoop MapReduce architecture has successfully met its original design goals it suffers from the following constraints:

• The Hadoop MapReduce framework is typically applied to large batch-oriented computations that are primarily concerned with time to job completion and not real-time computations
• All the intermediate output from each map and reduce stage is materialized to disk before it can be consumed by the next stage or produce output.  This strategy is a simple and elegant checkpoint/restart fault tolerance mechanism that is essential for large-scale commodity machines with high-failure rates but comes with a performance price.
• Each node within a MapReduce cluster is typically a 2 – 4 core CPU and the network link between nodes are typically1Gb/s ethernet links.  No GPU co-processors are attached to the nodes and in order to get the processing speed-up hundreds or thousands of servers must be used.   This leads to an initial substantial up-front investment required to build your own private large-scale MapReduce cluster plus high on-going power consumptions costs.
• Moving in-house MapReduce computations to an external MapReduce cloud service, whilst eliminating initial upfront hardware build and operational costs, may still result in substantial usage fees if hundreds or thousands of machines are required. Furthermore there is still the problem of moving large data sets to the cloud if your MapReduce jobs consume hundreds of terabytes of data.  This is a data transfer issue that is sometimes overlooked.

Other MapReduce Variants

There are many variants of the MapReduce model that have attempted to address some of these constraints whilst still using commodity CPUs.  These MapReduce variants include:

• Twister is a MapReduce variant that extends the MapReduce model by supporting iterative computations to make it much easier to implement iterative analytical processing, that is required for activities such as data clustering, machine learning, and computer vision.  For more information visit http://iterativemapreduce.org
• Hadoop Online Prototype (HOP) extends MapReduce to support continuous queries and online aggregation which allows users to see “early returns” from a job as it is executed. HOP programs can be written for applications such as event monitoring and stream processing. For additional information visit http://code.google.com/p/hop/

The key point to note with the above variants is that the MapReduce model is not “set in stone”. Many researchers have been exploring ways to extend the capabilities of MapReduce and overcome some of its design constraints.

The Big Question

Now High Performance Computing (HPC) GPU clusters are typically available at 1/10 the price and 1/20^th the power consumption costs compared with traditional large CPU clusters. So the big question is this: “Given the constraints of the Hadoop MapReduce, can we leverage the parallel processing power of the latest generation of Fermi GPUs, coupled with the simplicity of the MapReduce model, to create much smaller and affordable CPU-GPU clusters that can be used for real-time “Big data” analytics?”

Now some readers may not think that this is possible because of the view that the GPUs can only be used for mathematical and scientific processing. How at the heart of “Big data” analytics is a substantive core of mathematical algorithms that are required to do the actual data mining work.

The key to the efficient use of the GPU is to make sure that there is sufficient amount of computationally intensive work for the GPU to do and that this work is done is such way as to hide memory and network latencies.

Fortunately the CUDA GPU has been designed to address these issues and these capabilities have been substantially enhanced in the latest generation of GPUs – the Fermi..

So the answer to the “big question” is yes, provided you have selected and implement correctly the right parallel data mining algorithms for your “Big data” analytical processing.

Now it would be useful to look briefly at the early MapReduce GPU research work that have been conducted over the last two years before examining the potential for the development of new MapReduce variants, designed for the new generation of Fermi GPU clusters.

Early GPU and MapReduce Implementations

Early GPU implementations of the MapReduce model include the following research projects:

• In 2008 Bingsheng He published the seminal research paper “Mars: A MapReduce Framework for Graphics Processors”.  The Mars MapReduce variant was developed just for a single Nvidia G80 GPU (which contains 120 processors) and found that it was up to 16 times faster than a 4-core CPU-based implementation for six common analytical web applications.  However whilst Mars was not designed to scale above a single GPU.it was the first project to show the GPU potential for MapReduce.
• In 2009 Alok Mooley published a paper called “DisMaRC: A Distributed Map Reduce Framework on CUDA” about a project that implemented a MapReduce variant on to a small distributed cluster of commodity CPUs where each CPU had an attached 2 Nvidia FX5600 GPUs.  This was a significant advance over the Mars project and a 2 node cluster (where each node had a CPU with 2 attached GPUs on each node) achieved a speed-up of more than 4x over the Mars performance. The DisMaRC work did not conduct performance tests against standard cluster running Hadoop MapReduce.  However it showed that the initial Mars research work was on the right track.
• In late 2009 Reza Farivar published a significant and ground breaking paper called “MITHRA: Multiple data Independent Tasks on a Heterogeneous Resource Architecture”. This paper demonstrated that a version of Hadoop MapReduce when “ported” to a small 4-node GPU cluster could outperform a regular Hadoop 62 node CPU cluster and achieved a 508x speed-up per cluster node when performing Black Scholes option pricing. It should be noted that Black Scholes algorithm is an analytical algorithm.  The GPU cluster configuration comprised 5 nodes each comprising a quad-core CPU with two 9800 GX2 GPUs, each with 128 core processors, connected to a Gigabit Ethernet router and one control node also connected to the Gigabit Ethernet router.  This work demonstrated, for the first time, that you could have a small hybrid CPU-GPU cluster with less than 1/10 the size of a regular cluster and yet still achieve 508x speed up over a regular Hadoop cluster 10 times its size.

It should be noted that all these initial GPU MapReduce research work used with the older GPUs (G80 and GT200) and I expect to start seeing further papers by next year reporting on the new GPU MapReduce research developments based on the disruptive power of the Fermi GPU.

However a fast Fermi is by itself not sufficient to build the type of hybrid GPU clusters that I believe is required to compete with the large Hadoop clusters. So before I present my predictions for hybrid CPU-GPU MapReduce clusters I would like to talk briefly about Fat Nodes.

Scaling the GPU – using Fat Nodes

One approach for scaling up from a single GPU workstation is to use the concept of Fat Nodes. The purpose of Fat Nodes is to keep as much processing, as possible, on the local node using the following architectural design features:

• Use dual 12 core CPUs each with 64GB or more RAM on each CPU giving 24 CPU cores and 124GB RAM on each node.
• Connect 10 or more Fermi GPUs to the dual CPUs to provide 4,800 GPU processing cores and delivering over 10 TFLOPs of processing power on each node.
• Replace local hard disks with high-speed solid-state drives, each with with 200K IOPs or more per SSD using PCI Express. Multiple SSD can be combined to run in parallel to achieve more than 2.2 million read input/output operations per second (IOPS) on a single node. However there are other design options, for example, the SSDs could be shared with multiple nodes using PCIe IO virtualisation.  However the key point here is to replace local hard disks with SSDs for high data retrieval performance. For example, ,a single SSD with 200K IOPS is equivalent to having between 50 – 100 hard disks running in parallel on a single PCIe card.
• Use where possible 40 Gb/s InfiniBand network connections (5Gb/s and 10Gb/sEthernet are alternatives but with much higher network latencies) for inter-node network traffic. When this is combined with Nvidia’s GPU Direct it enables GPUs to transfer data from their local device memory to another CPU on another node without first putting the data into local CPU memory. This coupled with a network transfer speed up to 90M MPI messages per second across PCIe 2 bus to another node substantially exceeds the messaging passing capabilities of a large Hadoop clusters.

The ability to easily design and implement Fat Nodes was further enhanced by a company called NextIO (http://www.nextio.com). NextIO has released a GPU IO re-configurable appliance designed to dynamically implement and connect the Fat Node architecture outlined above. (See figure 2).

Figure 2: NextIO High Performance Compute IO Appliance

Next Generation MapReduce CPU-GPU clusters

Over the next 18 months expect to see small to medium size Fat Node CPU-GPU clusters based on the Fermi, and its successors. This will revolutionise the cost and speed of doing “Big data” analytics.   Already all the hardware is now available to build Fat Node GPU clusters from vendors such as Appro (www.appro.com), NextIO (www.nextio.com) and SuperMicro (www.supermicro.com).

What is really required is a middleware runtime that provides similar software functionally that the “Big data” community has come to love and associate with Hadoop MapReduce. Also if these GPU clusters are going to provide real-time production analytical services they will need to be integrated with legacy production IT systems (possibly using enterprise service bus (ESB) so different layers of middleware will be required.

Clearly there is still some work to be done to implement this vision of MapReduce on the GPU.  I am also expecting that over the next 12 months we will see substantial software improvements with the CUDA SDK to take advantage of the new GPUs coming in 2011 and 2013 (Kepler and Maxwell).  For example, it would be nice to have true virtual memory on the CUDA main device memory.  Based on the hints given at GTC 2010 we can expect to see this and further exciting developments in the coming months. As the old Chinese curse says “May you live in interesting times”, I think that we are.

Conclusion

Designing a MapReduce variant that is able to leverage the impressive GPU technology that is available now will significantly lower the upfront cluster build and power consumption costs for “Big data” analytics, whilst at the same time take advantage of the MapReduce model’s ability to lower software development costs.

Smaller companies or business units within large companies will be able to conduct “Big data” analytical processing without having to spend $500,000+ to build a Hadoop cluster.  The ability to have in-house small GPU departmental clusters for “Big data” processing will enable business managers to conduct more computationally intensive “Big data” analytics at substantially lower capital and operational costs.

Posted by Suleiman Shehu

These GPU-accelerated performance results also hold for large data sets.  For example in 2009 data set with 1 billion 2-dimensional data points and 1,000 clusters, the GPU-accelerated K-Means algorithm took 26 minutes (using a GTX 280 GPU with 240 cores) whilst the CPU-only version running on a single-core CPU workstation, using MimeBench, took close to 6 days (see research paper “Clustering Billions of Data Points using GPUs” by Ren Wu, and Bin Zhang, HP Laboratories).  Substantial additional speed-ups are expected were the tests conducted today on the latest Fermi GPUs with 480 cores and 1 TFLOPS performance.

Over the last two years hundreds of research papers have been published, all confirming the substantial improvement in data mining that the GPU delivers.   I will identify a further 7 data mining algorithms where substantial GPU acceleration  have been achieved in the hope that it will stimulate your interest to start using GPUs to accelerate your data mining projects:

• Hidden Markov Models (HMM) have many data mining applications such as financial economics, computational biology, addressing the challenges of financial time series modelling (non-stationary and non-linearity), analysing network intrusion logs, etc.   Using parallel HMM algorithms designed for the GPU, researchers (see cuHMM: a CUDA Implementation of Hidden Markov Model Training and Classification by Chaun Lin, May 2009) were able to achieve performance speedup of up to 800x on a GPU compared with the time taken on a single-core CPU workstation.
• Sorting is a very important part of many data mining application.  Last month Duane Merrill and Andrew Grinshaw (from University of Virginia) reported achieving a very fast implementation of the radix sorting method and was able to exceed 1G keys/sec average sort rate on an the GTX480 (NVidia Fermi GPU).  See http://goo.gl/wpra
• Density-based Clustering is an important paradigm in clustering since typically it is noise and outlier robust and very good at searching for clusters of arbitrary shape in metric and vector spaces.  Tests have shown that the GPU speed-up ranged from 3.5x for 30k points to almost 15x for 2 million data points. A guaranteed GPU speedup factor of at least 10x was obtained on data sets consisting of more than 250k points. (See “Density-based Clustering using Graphics Processors” by Christian Bohm et al).
• Similarity Join is an important building block for similarity search and data mining algorithms. Researchers using a special algorithm called Index-supported similarity join for the GPU to outperform the CPU by a factor of 15.9x on 180 Mbytes of data (See “Index-supported Similarity Join on Graphics Processors” by Christian Bohm et al).
• Bayesian Mixture Models has applications in many areas and of particular interest is the Bayesian analysis of structured massive multivariate mixtures with large data sets.  Recent research work (see “Understanding the GPU Programming for Statistical Computation: Studies in Massively Massive Mixtures” by Marc Suchard et al.) has demonstrated that an old generation GPU (GeForce GTX285 with 240 cores) was able to achieve a 120x speed-up over a quad-core CPU version.
• Support Vector Machines (SVM) has many diverse data mining uses including classification and regression analysis.   Training SVM and using them for classification remains computationally intensive.  The GPU version of a SVM algorithm was found to be 43x-104x faster than SVM CPU version for building classification models and 112x-212x faster over SVM CPU version for building regression models.  See “GPU Accelerated Support Vector Machines for Mining High-Throughput Screening Data” by Quan Liao, Jibo Wang, et al.
• Kernel Machines. Algorithms based on kernel methods play a central part in data mining including modern machine learning and non-parametric statistics. Central to these algorithms are a number of linear operations on matrices of kernel functions which take as arguments the training and testing data.   Recent work (See “GPUML: Graphical processes for speeding up kernel machines”  by Balaji Srinivasan et al. 2009) involves transforming these Kernel Machines into parallel kernel algorithms on a GPU and the following are two example where considerable speed-ups were achieved;  (1) To estimate the densities of 10,000 data points on 10,000 samples. The CPU implementation took 16 seconds whilst the GPU implementation took 13ms which is a significant speed-up will in excess of 1,230x;  (2) In a Gaussian process regression, for regression 8 dimensional data the GPU took 2 seconds to make predictions whist the CPU version took hours to make the same prediction which again is a significant speed-up over the CPU version.

Using GPUs on Workstations

The majority of the current GPU data mining research was conducted on workstations where a single CPU is connected, via PCI Express bus, to between 1 and 4 CUDA GPUs.  A typical use-case involved using only one GPU connected to one CPU. CUDA (an acronym for Compute Unified Device Architecture) is a parallel computing architecture developed by Nvidia.  A CUDA program comprises a main host program (written in an extended C/C++) that is executed on the CPU and which calls out to programs that will be loaded and executed on the GPU.   CUDA requires that each GPU is “managed” by a unique CPU thread.  So to use 4 GPUs with single CPU will require at least a 4-core CPU.

There are other types of GPUs (e.g. AMD-ATI) however the CUDA GPU is the market leader and we will only be referring to the Nvidia CUDA GPUs in this post.

Now you may be asking the following questions “What happens when you want to scale-up beyond a single workstation to a cluster of GPUs?” and “Can we use small GPU clusters for Big Data Analytics instead of the traditional large commodity clusters running Hadoop MapReduce?” Answers to these questions will be presented in a subsequent post.

GPU for Data Mining – 10 Things To Do

If you have not yet started experimenting with GPU’s then here are 10 things to do to get started:

1. Understand the CUDA GPU concepts using the web-based training provided by Nvidia at http://goo.gl/J8mY. There are many good courses on this site including screencasts.  Also very useful if you want to learn how to develop CUDA GPU programs.  For those who prefer to read books instead of online training material I would recommend the book “CUDA by Example – An Introduction to General-Purpose GPU Programming” by Jason Sanders and Edward Kandrot.
2. To get a feel as to the range of CUDA applications (not just data mining) by visiting the CUDA Community Showcase http://goo.gl/6exQ
3. Visit the GTC 2009 Conference archive to see all the screen-casts and video recordings showing how the CUDA GPU has been applied in science, manufacturing, finance and various research projects. The link is http://goo.gl/mbTJ Later in October 2010 the GTC 2010 Conference archive will be available online.  Look out for  webcasts such as “2069 – GPU Accelerated Business Intelligence Analytics” and “2111 – Using R for High-Performance Data Analysis”
4. If you want to use the GPUs but you do not want to get your hands “dirty” writing CUDA C/C++ code (or other languages bindings such as Python, Java, .NET, Fortran, Perl, or Lau) then consider using MATLAB Parallel Computing Toolbox.  This is a powerful solution for those who know MATLAB.   Alternatively R now has GPU plugins.   A subsequent post will cover using MATLAB and R for GPU accelerated data mining.
5. It is highly likely that your PC may have an Nvidia GPU graphics card that can be used to start to learn CUDA programming.  For full information on setting up a CUDA development environment and downloading the free CUDA SDK visit http://goo.gl/wPUN
6. Attempting to move your existing data mining algorithms to the CUDA GPU without fundamentally re-designing your algorithms, to take advantage of the ability of the GPU to execute many thousands of concurrent threads, is likely to result in very poor performance.
7. Re-design your algorithms to take account of the limited on-device GPU memory (the Fermi now supports 6GB of global on device memory, up from 1GB- 4GB on previous GPUs).  Also you need to understand how to co-ordinate the transfer of data from the CPU to the GPU, with processing on the GPU, and the transfer of the results back to the CPU.
8. Do not be afraid to forget all the ideas you learnt when you were developing your sequential data mining algorithms for the CPUs.  Instead look for computationally dense parallel algorithms that are can be applied to the CUDA GPU architectures that will give you the required speed-up.
9. Conduct a Google search looking for recent data mining papers showing how researchers have used the GPU in your area of interest.  You will gain many useful insights by reading these papers, including sample CUDA algorithms.  However do NOT accept that the speed-up they achieved is the last word on the matter.
10. If you do not want to develop the GPU algorithms yourself then either get your IT department (if they have the expertise) or get an external CUDA consultancy company to develop the GPU algorithms to meet your requirements.

Note:

When looking at recently published GPU research papers here are at least three reasons why you can expect today  to achieve even better speed-ups :

• They may have been using an older version of the GPU such as the G80 chip (instead of the Fermi GTX 480 which at this time is the fastest and latest CUDA GPU).  The older GPUs have less cores, slower processing speeds and some software limitations that are not found in the newer generation of GPUs such as the Fermi.

• The architecture and capabilities of the Fermi is much greater than earlier generations so do not assume that all the  limitations found in these papers applies to the Fermi.  These means that it is now possible to develop better and faster parallel algorithms than was possible in 2008 and 2009.

• You may find that the researchers are using data mining algorithms that will only achieve a relatively modest speed-up due to poor parallel design.  It may be possible to  achieve a much better speed-up if you use a more computationally dense variant of the same family of data mining algorithms.

Finally have fun!

Posted by Suleiman Shehu

Directors are often being told that if they implement a single agile technology that they will solve the agility problem for their business.  However whilst many businesses have achieved some levels of success in using a single technology (such as BPM or Predictive Analytics), this single technology approach is no longer viable in today’s challenging and competitive business environment.

There are currently six technologies that can be considered as the “Six Levers of Business Agility”. A detailed white paper on the Six Levers of Business Agility can be download from here).

Implementing these six business agility levers provides a substantial additional uplift in business performance over what can be achieved by a single lever.

The following is a brief overview of the  six levers of business agility:

Business Agility Lever 1: “SOA-enable” your business applications (see  SME, SOA, Web Services and Business Agility).  If the functionality within your business applications are available as SOA services, then you can use these services to compose more flexible and dynamic business processes capable of responding to changes in your business environment.

The requirement to have “SOA-enabled” applications is a requirement if a business wants to take full advantage of the Six Levers of Business Agility.  However the good news is that, today,  most enterprise business applications and SaaS offerings are already “SOA-enabled” using Web services.

Business Agility Lever 2: Business Process Management (BPM).  You should consider acquiring a suite of Business Process Management tools to design, compose, manage and execute business processes based around your SOA-enabled applications.

The BPM tooling, and a business process execution platform, will enable you to reduce the operational costs of your business processes, as well as provide a platform for  injecting the other agility levers into your business processes.

Business Agility Lever 3: Enterprise Decision Management (EDM). You should factor out your company polices, procedures and business rules from your application code and and replace them with their equivalent in business rules format.  It is these business rules executing within business rules engines that are called enterprise decision services.

These enterprise decision services can be called from within BPM process models to make real-time operational decisions, and, direct and control the flow of the business process.  Enterprise decision services enhances business agility because it is very easy to modify the business rules to reflect external changes.  For example, changes in customer demand, corporate policies or government regulations can be implemented without additional costly software modification work.

Business Agility Lever 4: Complex Event Processing (CEP).  At any given  moment, hundreds or even many thousands of events per second occurs within your business applications, operating systems and networks.

Often whilst single events are consumed by specified applications, it is the combination of several events generated by many different applications  (known as a complex event) which, if you could detect and respond in real-time, could help you identify new opportunities that could lead to increased revenues or reduction in costs.

Now the key to CEP agility is configuring a CEP engine to execute any number of business processes on the detection of specific combination of complex or single events.   Using CEP with the other agility levers gives your business real-time agility to respond in time to take advantage of revenue generating opportunities. For example, the ability to respond to any interaction, by any of your customers, using any of your sales or support channels in real-time.  Furthermore the response can be  tailored to each specific customer interaction.

The potential applications for CEP are huge as this technology can be applied to virtually all aspects of your business operations.

Business Agility Lever 5: Predictive Analytics. Predictive analytics is one of the most important business agility lever and a lot of vendors are currently selling standalone predictive analytical applications.

However the sweet spot, if you have already using  SOA, BPM and EDM business agility levers, is that you can use predictive analytics to enhance your enterprise decision services so that even more profitable decisions can be made.

Business Agility Lever 6.   Constraint Optimisation. This technology was previously hidden away in the Operational Research and Logistics departments.

Constraint optimisation is about determining the best possible utilisation of   resources (e.g. time, people, raw materials, securities) required to achieve a desired business outcome (minimum cost, maximum profit, minimum process time, etc.).

The sweet spot of constraint optimisation is when it is injected into decision services to provide highly optimised  business decisions that reduce business costs and at the same time maximising a business objective.

Implementing the Six Levers of Business Agility.  Do not forget that when proceeding to implement these Six Levers of Business Agility that one does not attempt to implement all six levers in your first iteration.   The best approach is to undertake a series of phased implementations, adding additional levers at each iteration.

Of course implementing  these business agility levers will require software products and tools.  There are several companies that offer good platforms that support 3, 4 or even 5 of the Six Levers of Business Agility.

One such company, Azinta Systems (http://www.azinta.com)  who provide a single integration Business Agility platform (APADO) containing all the required software products and tooling to implement the Six Business Agility Levers.  However I must declare an interest as I am the CEO of Azinta.

It is also possible to use open source products to implement the Six Levers of Business Agility (in fact APADO is based on open source projects) and I will discuss these open source projects in subsequent posts.

Posted by Suleiman Shehu

These dynamic business processes can be integrated with other business processes to create highly agile applications that enables your company to respond to changes in your business environment without requiring expensive and lengthy software development.

The good news is, if you have purchased enterprise application packages over the last few years then they should have been “SOA-enabled” using Web services technology.  If they are not then get your vendor(s) to give you the service interfaces for your applications, this should be free if you have a support contact in place.

Web services is a connection technology that provides a standard way for these “SOA-enabled” applications to connect with each other with very little effort.  In fact most SaaS platforms such as Salesforce.com also provide Web services interfaces, so that the functionality of their platforms can be integrated with other SaaS and on-site business applications.

I was most pleased to read the blog post by Hugh Taylor called The Triumph of the Ho-Hum SOA who pointed out that “SOA has become so omnipresent, so unsurprisingly effective, it’s a triumph”.

However just having your business applications  “SOA-enabled” is not enough to achieve the levels of agility that your business requires.

You should consider implementing the following to get the best results from your “SOA-enabled” applications:

• Use a Business Process Management (BPM) modelling tool to design and compose dynamic business process models from your collection of application services  and human tasks. The best modelling tools are those that use a graphical interface that can be understood by business managers.

• Use a business process server to execute the business process models created with the BPM modelling tool.  The process server will call all the application services  and human work activities using the process models to direct its operations,  until the business process is completed or terminated.

• Conduct a business performance assessment to determine which part of your business operations you should start to re-engineer to get increased agility and quick profitable wins.  Start with small incremental business process improvement steps rather than attempting one big project.

• Identify the other levers of business agility, that you may want to include in your business process models, to achieve even higher levels of agility and profitability.  I will have more to say on this in subsequent posts.

• Implement SOA governance and security tools.  It is important to be able to manage your “SOA-enabled” services and to make sure that these services can only be accessed by authorised services and individuals.

• You may require an Enterprise Service Bus.   Sometime an enterprise service bus is required to provide additional SOA-enabled security services and integration with legacy applications.  I will be covering this in future posts.

Up until now it was not considered feasible for SME companies to implement true SOA agility due to the high cost of commercial tools, technologies and shortage of skilled resources.

However that is no longer the case.  With the rise of enterprise-ready open source products the SME director can now implement all the required tools, technologies and products at affordable costs.

In subsequent posts I will share some insights in how you can use enterprise-ready open source products to make your company agile, gain increased market share, generate additional profitable revenues and make significant reductions in operational business costs.

Posted by Suleiman Shehu

Just the other day I received a mail-shot from OpSource.net with an attention grabbing headline  “There is no such thing as a Private Cloud”. So with  raised eye brow I clicked on the email and found an invitation to a webinar by Phil Wainewright called “There’s no such thing as a Private Cloud” plus a link to a white paper called “Enterprise Meet Cloud: Mapping a safe passage to enterprise cloud adoption”.

I was impressed with the webinar and white paper and I would recommend them to any one looking to use the cloud to deploy mission critical business applications and are concerned about security issues in the cloud.

Note I have no connection with the guys at OpSource.net however I have been tracking them for a number of years now (before Cloud became the in-thing).  They also offer a “white-label” cloud service for those companies who want to offer their clients a cloud-based offering without having to make the $multi-million investment in creating a secure cloud hosting infrastructure.

OpSource is certainly a company to watch.