Monday, June 05, 2006

Real-Life MDA

For the last couple of years, I've been running a program called MDA FastStart, hosted by the OMG. MDA FastStart is a group of 30+ professional services companies specializing in MDA, and they all done some very interesting real-life MDA projects. Some of these have been written up and are on the OMG website, and I've published stories about a few others in Software Magazine.

I'm happy to announce that this fall, Morgan/Kaufman will publish an entire book of detailed MDA case studies called "Real Life MDA", authored by myself and John Parodi. Those case studies include a global auto manufacturer, a global telecom software vendor, a major US federal government department, a major US state government agency, an EU national health agency, an international hospital services company, and an international property development company.

John and I really enjoyed researching and writing this book. It was both fun and informative to interview both business and IT people applying MDA in a wide variety of ways. We've tried to organize the book so that each case study stands on its own, but you can also compare the approaches. As soon as this book is available, I'll post an update on this site.

In the meantime, our quest for more case studies goes on, as we'd like to publish a new set in another book. If you are either a consultant or an enduser who has an interesting story to tell about applying MDA to deliver an important business solution, we'd like to hear from you. Stories about how MDA was used to enable BPM and SOA would be particularly welcome. Just post a comment on the blog with your contact information, and we'll get in touch with you ASAP.

Linking BPM and SOA - It Shouldn't Just Be Magic

Recently I was asked by a large financial software vendor to present a talk about the relationship between SOA and BPM at in internal IT conference aimed at their top IT architects, managers and developers. I was not alone. A number of other presenters, many representing companies with SOA and BPM development products to sell, extolled the notion of tying together SOA and BPM.

In general, the vision presented was a brave new world where business analysts can simply compose the new or improved business processes they need from a set of reusable business components, after which some run-time execution BPM engine (nowadays usually based on BPEL) will end up invoking the appropriate reusable SOA-based services to execute those processes.

To this end, each of the various presenters happily showed off slideware of his own company's BPM/SOA 'marketecture'. This usually involves some kind of layered approach, with 'business components' in a BPM layer connecting via an enterprise services bus (ESB) down to one or more' service components' in a SOA layer. From the SOA layer, some of these service components call other service components, while the most granular call the necessary implementations, which are typically 'wrapped' legacy applications, either custom developed or COTS.

I suspect nearly everyone reading this blog has already seen some form of this kind of approach - you may even have presented your own version! It is certainly a very appealing idea and, as a vision, I completely support it. But there's just one problem - as usually presented today, it is mostly a sleight-of-hand magic trick, not reality.

As with most such magic tricks, the presenter must get the audience to believe they 'saw' something that didn't really happen. In this case, this involves getting the audience to believe that, just using BPM and SOA, they will automagically come up with an appropriate set of reusable, recomposable components at both the BPM and SOA layers, plus an efficient mapping between the two.

In fact, when pressed by this very savvy audience, none of the presenters could actually explain how a real company with real IT systems could actually get from where they are today to realizing such a set of reusable components, except perhaps in a very limited way. As long as this remains the case, I don't expect his BPM-to-SOA approach to get much beyond the hype stage. Unfortunately, if it gets too hyped in its current 'magic' stage, it may even ultimately discourage companies from even trying to reach its ultimate and, I believe, very achievable vision of using BPM and SOA synergistically.

So, what's the problem with the 'magic' BPM-to-SOA approach I just described, and how do we get beyond that magic to something real? The first step is to realize that unfortunately, neither BPM nor SOA can, together or singly, tell you how to define the correct components you'll need at either level, much less how to synch these up.

Yes, once you have a set of such components, BPM will allow you to recompose them at will into new processes, and SOA will allow you to deploy and execute them on an enterprise (or even inter-enterprise) service bus. This makes them potentially very powerful enablers, and that's why they sound so sexy together. But until you figure out what those reusable business components are, the whole BPM-to-SOA approach remains in the realm of legerdemain.

So, is there an approach that CAN help to identify a set of reusable, recomposable business and service components? Fortunately there is – MDA. By properly applying MDA to this kind of problem, you can develop multiple viewpoints of your enterprise architecture, while keeping those viewpoints in synch. And that's exactly what we need to link up BPM to SOA.

In MDA, a ‘viewpoint’ is just a set of models presented in a form suitable to a particular audience. So, in this case, we can develop a viewpoint for the business analysts to show the available enterprise functionality (IT or otherwise) as a set of reusable business components, which can be linked together using BPM. At the same time, we can also show the available IT functionality as a set of reusable service components, each with a mapping down to one or more implementations, which can be deployed in a SOA ESB.

But what about the mapping between the two viewpoints? In MDA all models are well formed and formalized. Therefore, it is possible to represent a mapping between viewpoints as a set of formal ‘transformations’, each of which just another form of model. In some cases, all or parts of a given transformation can be automated. In other cases, some kind of human intervention will be necessary to effectuate the transformation. But, in either case, MDA will help keep everything in synch.

Now, armed with MDA, we can paint a much clearer and more compelling picture of the BPM-to-SOA approach. We start by defining a formal BPM-based business viewpoint, a formal SOA-based services viewpoint, and a formal transformation between them. Until we can do all this, we really can’t guarantee that any particular set of business components – or any business process built from those components – can map down to SOA. But, once we define the viewpoints and transportations, we can now guarantee that every such business component or process defined at our BPM layer will map down clearly to service components at our SOA layer.

By the way, this also explains why it doesn’t make sense to just start doing BPM at one end, and SOA on the other, and hope that the two will somehow synch up. Without a common framework like MDA sitting in the middle to guide the mapping, it will never happen.

Interestingly, none of the other presenters at the conference even mentioned MDA in their talks. To be fair, the conference was on SOA and BPM, not MDA per se. However, when questioned, several of those who marketed BPM and SOA tools freely admitted that their own organization were using MDA extensively in order to move information around within their toolsets, and to keep that information in synch among different viewpoints. If these vendors depend on MDA to design, develop and integrate their own software, maybe they should also start telling their customers a bit more about how to take advantage of the same approach. Maybe that would take some of the mystery out of the BPM-to-SOA connection.

Have you had any experience trying to create a mapping between BPM and SOA? Did you use MDA for this purpose? Are you interested in learning more about this subject? Do you have an alternate point of view? If any of your answers were 'Yes', please post your comments to this blog - we look forward to hearing from you.

Friday, April 07, 2006

David Frankel Weighs In

Check out where David Frankel, another true MDA expert from SAP Labs (SAP America), offers further interesting rebuttal and clever insights on the report from Forrester. Clearly not pleased, David does a masterful job of demonstrating where misinformed [or purposefully misconstrued] information about MDA can lead to incorrect conclusions. David has been at the vanguard of MDA, MOF, CWM and UML so at least he knows what he is talking about!

Sunday, March 26, 2006

Forrester Research Claims MDA is DOA!

Our REBUTTAL to a recently published “MDA is DOA, Partly Thanks to SOA” (March 22, 2006 Trends Report), Carl Zetie, an industry analyst at Forrester Research, claimed that Model Driven Architecture (MDA), a set of internationally adopted software standards and best practices from the Object Management Group (OMG), is fatally flawed. He further claimed that MDA is failing to gain any serious foothold in the market, thanks in large part to the recent surge in popularity of Service Oriented Architecture (SOA).

We don’t agree!

Is MDA really DOA?

In our opinion, which includes a couple of decades each of hands-on practice in the enterprise architecture and modeling industry, MDA is not only very much alive, but it is now frequently being used as a major enabler for SOA. And, far from displacing MDA, SOA is actually helping MDA to gain more attention in the marketplace. We are hardly alone in this view and our rebuttal here is to help inform anyone, including Carl, who may not properly understand how SOA and MDA relate to each other.

From a standards viewpoint, MDA is still the major initiative at the OMG, an open 500+ member international consortium of vendors, consultants and end-users, many of whom are also active in the SOA community. At the OMG, member-run MDA-related activities are growing leaps and bounds, and SOA is as hot a topic there as it is everywhere else in the industry. The top technologists of major organizations actually volunteer their time and talents to work on these MDA-related activities, and many of these are also directly active in SOA activities.

In addition to a whole raft of on-going general standards-setting activities surrounding MDA, there are a number of highly focused OMG task forces and special interest groups (SIGs) focusing on particular industries. Current OMG SIGs include Business Enterprise Integration, C4I (the military), Finance, Healthcare, Life Science Research, Manufacturing Technology and Industrial Systems (ManTIS), Software-Based Communications, Space, and Transportation. Each of these SIGs has MDA as a primary standard with which they are developing their further, more specialized standards.

But that’s not all. The OMG also maintains close relationships with a number of other important industry consortia and standards groups, many of whom either endorse or develop their own MDA-and SOA-related standards. These are all listed on the OMG site ( They include such well-known groups as OASIS, W3C, The Open Group, ISO, DMTF, ISO, HL7, and so on.

Another group of people who would likely dispute Carl’s grim predictions of MDA are the considerable number of software vendors who already have MDA firmly on their product roadmaps, and publicly support it. Many of these are also SOA vendors, including the two mentioned favorably in the report (Borland and Compuware). In all, there are 50+ organizations specifically listed on the OMG website as ‘committed companies’ for MDA, and another 30+ (some overlap) listed as Qualified Service Providers (QSPs) in the OMG’s MDA FastStart program. Remember, these companies volunteer and contribute to be in these publicly available lists. Each QSP even signs a contract with the OMG specifically stating that it will provide professional services to support MDA, and many of these provide SOA-related services as well.

As we speak, all these product and service vendors, who make their living meeting the pressing needs of real end-users, are aggressively integrating MDA standards and best practices into their tools, platforms and professional services offerings. We can also point to any number of case studies, many readily available on the OMG website, demonstrating how MDA is being used successfully to deliver mission-critical solutions in a wide variety of end-user business and IT environments.

Now let’s tackle Carl’s assertion that MDA’s putative demise is somehow ‘partly thanks to SOA’. Carl’s not-so-subtle message is…’SOA good, MDA bad’. But, as we stated above, SOA and MDA are not competitive. In short, MDA is a way of using formal modeling languages, such as BPMN and UML, to concretely improve the entire software lifecycle – from requirements capture through to systems retirement. Meanwhile SOA offers a very effective architectural pattern for designing and implementing loosely-coupled, reusable software services. Properly applied, these two approaches can be highly complementary.

For this very reason, there are already many vendors, consultants and end-users (people who earn their living actually doing this stuff) collaboratively implementing and promoting both MDA and SOA. Carl stated that big-name SOA vendors haven’t been actively participating at OMG forums about MDA, but that’s simply not true. At a recent OMG workshop entitled “SOA, MDA and Web Services”, held in Washington, DC (and dismissively noted in Carl’s report), top technologists from SOA vendors IBM, BEA, Actional (Sonic) and IONA presented, as did representatives from other key industry vendors like Adobe, Borland, and Intel.

Now, this wasn’t a trade show where everyone goes just to get out of the office. It was a three-day highly focused workshop intended to foster continued collaboration between practitioners and standards. Given that nearly 200 of the top technical minds in the SOA and MDA space paid to attend, we believe this is further proof of the budding marriage of MDA & SOA.

We are glad to hear that, while Carl is clearly not an MDA fan, he likes modeling in general, and particularly what he calls ‘pragmatic MDA’. Specifically, he lauds two MDA vendors (Borland and Compuware) for having excellent ‘pragmatic MDA’ products. However, these vendors certainly don’t differentiate themselves in this way, and we aren’t exactly certain what ‘pragmatic MDA’ is since it is not a term of the industry. In any case, these vendors actively market themselves as full supporters of MDA, and contribute heavily to the MDA-related standards activities at the OMG.

In any case, how is the poor consumer supposed to figure out which implementations of MDA are ‘pragmatic’ and which are not? Carl doesn’t really tell us. What he does say is that MDA in general is based on two big ‘myths’:

1. “It’s possible to model systems without regard to implementation details.”
2. “It’s possible to transform systems without human intervention.”

These are indeed ‘myths’, but they are myths about MDA, not myths created or supported by the MDA standard. They are certainly not myths that any responsible members of the MDA community, providers or consumers, actually believes or promotes. However, to help clear up any confusion let’s address these two ‘myths’ directly:

In terms of modeling and implementation details, MDA actually states that the functional aspects of a system—what many people call the business logic—can and should be specified independent of platform-specific details. Once this specification, commonly called the ‘platform independent model’ (PIM), is complete, then a ‘platform specific model’ (PSM) can be developed. The PSM provides the basis for the implementation, whether it is an implementation supporting SOA requirements or any other set of architectural constraints.

During this PIM-to-PSM transformation process, the various elements of the PIM are mapped down (‘transformed’) in a systematic way into appropriate elements of the design target, such as J2EE, .NET, LAMP, or, for that matter, WSDL for SOA. That’s the right time and place to consider implementation details, and few experts would seriously suggest otherwise. There’s neither myth nor magic here—just common sense organized into a formalized framework based on tried-and-true ‘pragmatic’ modeling techniques and supported by industry standards and best practices.

So, what about Carl’s second ‘myth’? Well, unless Carl has never seen a compiler, assembler, interpreter, it’s hard to imagine he really believes his emphatic statement that it is not possible to “transform systems without human intervention”. There are clearly any numbers of transformations in the development of a software system that can [and should] be automated.

To be more charitable, what Carl seems to be trying to say is that it is generally not practical to try to capture every possible specification detail of a system up front in a single formal modeling activity, and then just press a button to ‘automagically’ generate the resulting code. In this respect, we agree with his statements but – sorry again Carl – MDA does not state what he claims it does.

In fact, there is absolutely nothing in MDA that states definitively what part of the process of moving from PIM to PSM, and on to code and deployment, can or should be automated, or which must or should be done by hand. What MDA does say is that if any given transformation rule can be captured in machine-executable format, then that’s a bonus, and it provides a standard way to express and capture that rule. But if that rule can only be practically applied by a skilled human, that’s fine, too.

By the way, although it specifically supports standards like UML and BPMN for modeling, MDA doesn’t even dictate what modeling languages or conventions must be used. MDA just provides a framework to help organize and manage your favorite modeling and model-transformation tools and processes – automated or manual – thereby providing better uniformity and better traceability in either direction.

Based on these two main ‘myths’, Carl goes on to make a number other characterizations of MDA that we simply don’t agree with. For example, despite the fact that there have been a number of recent books and articles devoted to describing various approaches to ‘Agile MDA’, Carl states, without proof or example, that MDA is ‘not agile’. Meanwhile he claims that SOA is ‘often agile’. OK, Carl, if SOA is ‘often agile’, then you can ‘sometimes’ have ‘non-agile’ SOA, right? If so, then SOA is neither agile nor non-agile—it just depends upon how it is applied! That’s true for MDA as well.

Similarly, Carl goes on to assert that MDA is ‘inherently waterfall’ and ‘monolithic’, while SOA is not, etc. etc. In our opinion, such blanket statements are neither true nor particularly useful, in regard to either MDA or SOA. In our experience, neither MDA nor SOA prescribe or proscribe a ‘waterfall’ approach, and neither is inherently ‘agile’ or ‘not agile’, ‘monolithic’ or ‘non-monolithic’. Anything you can say about MDA in these respects, you can pretty much say just as well about SOA. It’s all in how you use the approach.

We were particularly puzzled by Carl’s mention that MDA is bad because ‘does not care about implementation’. In simple terms, MDA says that, when defining business logic (i.e. in a PIM), it just doesn’t make sense to include details about, say, the database schema or network protocol. So, MDA simply advocates adding these details later (i.e. in a PSM), not ignoring them completely, as Carl asserts.

But remember that Carl also claims that MDA is ‘monolithic’. Sorry, but we find it difficult to understand how someone could characterize ANY approach as ‘monolithic’ and, at the same time, claim that the approach ‘does not care about implementation’ – these are obviously contradictory statements. Broad generalities such as these simply serve to obscure any useful point Carl may be trying to make.

To most people in IT, ‘monolithic’ simply means that the functional logic of a system has not been properly decomposed and clearly separated from the associated implementation logic. This typically results in a final deliverable that is both difficult and expensive to maintain, reuse, or integrate with other systems – sometimes described as ‘brittle’. If MDA ‘doesn’t care about implementation’, it certainly can’t be ‘monolithic’ as well. Once again, the real issue is about when and where you need to care about implementation, and when you don’t. In this context, MDA is simply a way of organizing the ‘when’ and ‘where’ into PIMs and PSMs.

Furthermore, as many other industry analysts have already pointed out, it is quite possible to create brittle ‘monolithic’ services deployed using SOA technology, particularly if you don’t properly model those services up front. Such ‘monolithic’ services are just as hard to maintain, reuse, or integrate as conventional ‘monolithic’ applications, even though they have a public interface that has been described in WSDL and available on some kind of SOA-based services bus.

Unfortunately, Carl himself seems to advocate a haphazard, piecemeal approach to implementing SOA. In our experience, the likelihood of this approach succeeding in creating a truly agile infrastructure of reusable services is extremely remote. The more likely result will be ill-defined services reactively bandaged together over time to form yet more brittle, monolithic applications that are based on ‘SOA’ in name only.

Although Carl isn’t the only analyst to advocate doing some kind of ‘incremental’ SOA, he might be the only one who seems to suggest doing so without much upfront planning and, dare we say it, some upfront MDA-style modeling (e.g. implementation independent). True, once such upfront work is done, SOA services can be successfully implemented and deployed incrementally, as driven by the needs of the business.

In MDA terms, once PIMs for a given business domain are reasonably well defined, the associated SOA-based PSMs can be filled in incrementally. Since SOA implementations often leverage legacy systems, MDA-style PIMs can also be leveraged as useful guide for reverse-engineering such systems into SOA-based services, typically one of the first steps undertaken in a successful transition to SOA. Maybe that’s just some more of the ‘pragmatic MDA’ that Carl says he likes. If so, it’s exactly the same kind of MDA that OMG and its supporters have been advocating all along.

In any case, does anyone really believe that someone modeling, say, a banking application (or SOA service) needs to consider whether the software implementation platform is J2EE or .NET at the time at which they are specifying the business rules governing the relationship between two entities such as ‘customer’ and ‘account’? Are the bank’s business analysts wasting their time when, using an MDA approach, they create functional models up front, for example in BPMN, rather than jumping right to SOA implementation details? Should those analysts start by writing WSDL instead?

We don’t think so. In this regard MDA simply reinforces and enhances long-accepted best practices in modeling - that is, the business logic should go first (into a PIM), while the platform details should be added later (into a PSM). By the way, this also helps to ensure that the business logic is actually correct before moving on to detailing the implementation – what a concept! But, once again, this is nothing new, just a way of formalizing what most software development experts have been preaching for more than a decade.

By the way, Carl’s analysis also seems to have forgotten that SOA, which he obviously likes, was created expressly to support the specification of a service and its interface separately from its implementation. That sounds a bit like MDA! In fact, using MDA to help implement SOA makes a lot of sense. For example, you can derive the interface (and WSDL) of a SOA service based on the business logic specifications from an MDA PIM, while deriving the required implementation code (by hand or automagically) from the associated MDA PSM. Both the MDA tools that Carl likes (and many other such tools) actually take this approach.

Why is this good? Because then you are free to make any number of changes to the implementation of a SOA service over time, such as migrating to a federated database, without having to change the public interface used by outside applications and services. You can do this as much as you need too, so long as those changes don’t alter the core business logic originally used to specify that interface. This is truly ‘incremental SOA’ and exactly the approach that allows you construct ‘agile’ services that are not ‘monolithic’. And it’s completely consistent with best practices for both SOA and MDA.

Sadly, we must conclude that it is really most of Carl’s analysis of and advice about both MDA and SOA that is D.O.A., at least in this report. We are sad, because we know that Carl is a respected industry analyst with Forrester, a widely known IT analyst firm with blue chip clientele. And we’re understandably worried, because some of his readers who are not yet really familiar with MDA or SOA might trust his seriously flawed conclusions just enough to decide not to investigate MDA any further at this time, particularly if they are interested in SOA. We believe that would be doing a very serious disservice to the organizations they represent.

We hope that Carl reconsiders his opinions about MDA and paints a more accurate and cogent picture about the relationship between MDA and SOA in his future reports. In the meantime, where can you get the straight skinny on MDA? How about starting with the OMG. Is the OMG biased about MDA? Sure, the OMG invented MDA. However, it’s also a completely open industry consortium, so it doesn’t push only one point of view about MDA. On OMG’s website ( you can find information from a wide variety of sources – vendors, consultants, analysts, real live end-users, and of course the OMG itself. Do these sources all agree about every aspect of MDA? No way! But, in our opinion, they currently provide much better information on this subject than Carl has in his latest report.

-- Mike & Jason

Saturday, March 25, 2006

Introductory Note

Greetings from Michael Guttman and Jason Matthews! We are the principals of The Voyant Group, a professional IT consulting firm focusing on Model Driven Architecture (MDA), Business Process Modeling (BPM) and Service Oriented Architecture (SOA). Our posts will provide a freeflowing discussion of these critical technologies and how they can help organizations achieve the holy grail of the "Agile IT."