LiteBytes from Lustratus Research

Oracle moves to buy BEA: The end of the J2EE era?

Oracle has finally done what so many rumours have pointed to for at least a few years: made an offer to buy BEA.  I am sure that there will be much comment on the challenges of dealing with the total overlap between BEA's core product - the WebLogic applications server - and Oracle's application server (both in the top three by most measures of market size).  The move should also take the wind out of speculation that Oracle will make a spoiler bid for Business Objects. 

The writing off of BEA as a business by some has been totally over-stated.  However, I think if this bid is successful it does marks the end of the J2EE era.  Not that I am suggesting that J2EE application servers are going to go away of course.  Rather, the world has moved on from what created that market in the late nineties.  At one end of the spectrum, the focus has moved back towards technology independent architectures with SOA (just as CORBA attempted to do in the pre-J2EE days - all be it by creating another set of technology).  At the other end, lighter weight approaches such as Spring have superceded the heavier and more complex EJB model (which to be fair has also moved with the times but probably too late).   

It is also worth noting that the disappearance of the large enterprise focused ISVs continues - in one week we appear to be losing another two.  It is beginning to look like that the enterprise software market is heading for a strongly polarised world made up of a big-four (MS, Oracle, IBM and SAP) with a huge gap to the next division. 

Ronan

Trouble with evaluating SOA ROI

I was trying to think how to get another TLA in that title, since I think you get a prize for having three three-letter-acronyms in a row. However, the topic is definitely getting a lot of attention as companies try to decide whether SOA is worth the effort. The problem is, SOA benefits span a wide range, and are often difficult to assess. And yet, as John Soat notes in Information Week, real customers are showing major gains with SOA.

My take is that it is important to sort benefits into a spectrum of tangibleness (if such a word exists). So, reducing redundancy should have an actual dollar value reduction in maintenance costs - a tangible number. Delivering the agility to deal with new regulations more quickly is difficult to estimate in dollar terms, but could even be a survival issue. Seems to me the key is to find a way to include the full range of elements in any justification or evaluation.

Perhaps one way to add a dollar value benefit on some of the intangible benefits is to ask the executive in charge of the area most affected how much they would be prepared to pay to solve the issue. So, it might be interesting to ask the CFO how much he would invest to ensure the company could comply with new regulations within the assigned deadlines. This, then, becomes a tangible number that can be plugged into the case.

Steve

Use a hot-house to get better productivity

I was keynoting at the DeutschePost SOA Days technical seminar held in Bonn, Germany last week, and while I was waiting to present I sat through a very informative presentation from a large telecoms company about its efforts with SOA. However, one point that really caught my attention was the fact that the company has had some success with improving productivity through focusing on 90-day cycles. This extremely challenging timeframe is assisted greatly by the SOA approach used by the company, but a key element is that new projects start off life with a 'hot-house' activity.

The idea is to gather the core team of people involved in the project in one room, for however long is required, to spec out the project and select the implementation approach. This looks at the business and IT requirements and implications. What I love is that this 'hot-house' is carried out in a special room where there is only local networking available (no internet or email), and it even has mobile phone jammers to block calls. Results appear to have been extremely impressive. The hot-house sets the scene for a rapid development effort. Admittedly not every 90-day cycle completes the project - it can determine that another cycle is needed for additional research for example, but project delivery has definitely speeded up.

As for me, I just want one of those mobile phone jammers - I would love to take one on the train, and turn it on intermittently.....

Steve

Reuse still seen as top SOA benefit

I was interested to see, in a recent survey carried out by PMP Research, that the top-scoring benefit expected from SOA adoption was still reuse. The research was documented in the September 2007 issue of Conspectus, and on 1 1-5 scale 'Business process service re-use' scored 4.1 on the benefit list, followed closely by 'provides a more effective integration platform'.

Many SOA advocates talk about SOA being fundamental in terms of building a flexible and adaptable infrastructure while aligning IT and business requirements more closely, resulting in a more agile and effective platform for supporting business needs - so how come reuse still tops the list?

My own view is that reuse is simply the benefit that is the easiest to take in. The others I list above are very powerful, and can promise significantly more overall business value, but they require more than just a technology change. They require changes to working procedures and practices, and much better communications between business and technical communities. So, perhaps, in the eyes of the pragmatic respondents, reuse suggests itself as a reasonably obvious gain which even has a fairly clear and measurable monetary return attached. Reducing redundancy should have a direct correlation on application maintenance costs, for example. And on top of this, it can deliver value from a purely IT point of control (although the value will increase if business departments are involved too).

Steve

SOA, governance and the Trough of Disillusionment™

All shiny new things in IT go through a cycle of boom and bust before becoming an established and useful part of the enterprise IT world.  This cycle is as much about the need for marketing departments and the press to have something new to write about as it is about ‘real’ issues.  The pattern is so well established that Gartner even has graphic to capture where technologies are in what they call the hype cycle – the lowest point before technologies become generally understood and used is called the trough of disillusionment.

Service Oriented Architecture is no exception and Brenda Michelson of Elemental Links even celebrates the possible arrival of SOA at the trough of disillusionment as a good sign!  Moreover, I suspect that with SOA the trough may be worse than with most because it is more fundamental to the enterprise than ‘point’ technologies and attempts to address the fundamental issues of alignment with business strategic and IT agility.

I contrast SOA with point technologies because SOA is actually about changing the way IT solutions are created, deployed and maintained – not providing a single (however big) solution to a single (however big) business problem.  In essence, it is about enterprise architecture (and architects) and how this fits into the whole business.  This means that with SOA there should be no such thing as a stand alone project.  A stand alone project is inherently suboptimal as it removes the opportunity to reuse existing services and to expose further services.   It may take organisations many years to get there but this is the final destination.  This means that the organisational impacts of SOA (both human and technological) are in the long run the most important aspect of SOA. 

What has obfuscated this point to a degree is that most discussions around these issues are gathered into the term governance.  Unfortunately, using the term governance has put off many of the people who should be interested – perhaps governance suggests discussions on how to organise committees!   Furthermore, the term has to a degree been appropriated by vendors with products which assist in the technological aspects of governance.  And finally, there isn’t a single clean solution to governance as it is inherently tied into the way each organisation does its business which makes it hard for the press to get to grips with it.

All of which doesn’t take away the fact that governance is where SOA success will happen or not.  This of course does not mean that we now need a big-bang SOA Governance investment.  To quote Fill Bowen from IBM speaking at a presentation/discussion on governance hosted by the SOA Consortium at its European event back in June:   

“your SOA governance [should be] based on the level of SOA effort. So putting a
Cadillac SOA governance system in place when your SOA effort is targeted at just
trying to walk seems to be a little bit of overkill.”

Ronan

The Year of ESB-ability

Recently, I wrote about the ESB (enterprise service bus) maturing at a basic functionality level, and how the focus was swinging around to the '-abilities' such as scalability, availability, usability, manageability, etc.. There is also a free paper on this subject, available from the Lustratus web store, available here.

I noticed my post generated a number of responses from various people. These were all on a theme which I found quite amusing. The accusation was that ESBs are NOT mature - but the posts then went on to demonstrate exactly what I was saying. Take, for example, the illustrious Jame McGovern's excellent blog on Enterprise Architecture. In his wrap-up of links for 2007-08-05. James discusses my post :

"Another industry analyst gets it twisted by stating: the ESB concept has matured, the functionality checklists are of less value - everyone ticks most of the boxes which holds true if you decide to ignore enterprise security considerations."

David failed to consider more carefully what I said about basic ESB functionality. My point was that now that ESBs in general can support the basic functions of an ESB - pass messages from component to component with mediation services available in-flight for enrichment etc, support key standards such as web services etc - that attention is turning to the functionality surrounding these basic operations. That is, the characteristics and all those functions to actually make the thing usable in a production environment. If David had gone on to read the Lustratus paper I referenced, he would have seen that security was classified as an honourary 'ability' and is indeed one of the types of functions that people are now demanding.

Other posters fell into the same trap. So, I think we are actually all in riotous agreement. I may not agree with the next bit of David's post about the need to support such standards as SPML, XACML and WS-Federation, but that is largely because I can never keep up with the stream of new standards, and as most people know who read this blog regularly, I have a highly cynical view to many standards - particularly to WS ones outside of the obvious and more mature ones, as I outlined in my post on WS-Madness.

Steve

SOA for the scientific community - a practical example

I was intrigued to discover a discussion paper from the University of Southampton in the UK about how SOA is helping the scientific community. The paper, entitled 'A Collaborative Orthopaedic Research Environment', describes how SOA has been used to enable a Virtual Research Environment for orthopaedic researchers to collaborate in the design, analysis and dissemination of experiments.

What I found most interesting were the reasons for using SOA as the base architecture for this environment. The key objective, based on input from the specialists who will use the system, was to provide an easy way to share scientific data and results from collaborative research. However, it was deemed essential that the system could also evolve based on the changing requirements of the user community. One example of change that the paper gives is that of knowing which of a wide range of protocols will be followed for a particular clinical trial. Not only do these vary considerably, but it appears they are also susceptible to changes in regulations.

The solution decided to utilize a coarse level of services, essentially using just four - to manage the trial-related data, analyse it, submit and disseminate related research articles and support discussion forums. However, through the reliance on SOA, these services are flexible and extensible, making it much easier to address the changing needs of this particular scientific discipline. In addition, the services are reusable, and some could therefore be usable for other scientific areas.

It's good to see SOA being used to effectively address clear user needs in this way.

Steve

What is a best of breed ESB (four years on)?

Four years ago my colleague, Steve Craggs, wrote a seminal paper for the Integration Consortium called "Best of Breed ESB" (available from the Integration Consortium web-page and some other places such as here).  It was seminal in that it put an early stake in the ground defining what to look for in an ESB.  At the time, the concept of an ESB was pretty new and the definition was very vague (usually a single sentence from Gartner).  Four years on the ESB category has matured - the major vendors now all have something they call an ESB - but there is stil confusion to dispell and diversity in product offerings which we will be covering in an upcoming new edition of this paper.

Back in 2002, Steve's paper set the ground rules for what to look for in an ESB by setting out and discussing what users should expect to see:

"Fundamental ESB characteristics:
• XML, messaging, transformation, intelligent routing services
• Basic connectivity (Web Services, J2EE Connectors, JMS)
• Service-oriented architecture
• Support for highly distributed deployments
• Manageability
Key, value-add characteristics:
• Robustness
• Scalability and Performance
• Security
• Breadth of connectivity
• Development / Deployment toolset"

Unlike many of the analyst papers that followed it did not simply provide a lengthy ticklist of standards to be adhered to - rather it discussed why capabilities are needed and what alternative approaches were taken by vendors to provide those capabilities.  This was particularly important as not only did it clear confusion and FUD that was already gathering in the space, it also reflected the reality that the diversity of use cases and organisation types considering ESBs meant that there could be no single magic ticklist.  Back in 2003 when the paper appeared, I was CEO of PolarLake, one of the first ESB vendor, and it resonated because it solved a major problem for vendors and buyers alike: It provided a framework for matching requirements against products and comparing products. 

Four years on looking at the vendor offerings, while the basics are now settled there is even more diversity at a value-add feature level as well as new market dynamics (OSS has become a major force in ESBs).  On top of which, great diversity remains in use cases.  For these reasons, now seems like a good time to update and revise Steve's earlier work and I am now in the process of producing a new Best of Breed ESB paper.  While taking a simiar approach and covering some of the same bases, there will be a very different focus reflecting the major issues being faced by users today.  In particular: how do ESBs help with the SOA skills shortage and how do they support service reuse - as well as a whole raft of deployment issues.  The paper should be out in September as I am now in the process of being briefed by vendors.    

Watch this space...

Ronan

Be fair to OSS - lay off the sauce

Open source software has come on by leaps and bounds over the last ten years or so. There are more and more examples of credible OSS (Open Source Software) solutions available today, including well-known brands such as Firefox and LINUX. However, OSS seems to me to face a major enemy - itself.

The problem is that, deep down, the OSS community can sometimes see itself as a bastion of freedom against the evil commercial ISVs - certainly some of the more vocal supporters of OSS are guilty of making more and more outlandish claims about their own projects, really pouring on the sauce. Some of this is to be expected, of course, but there comes a point when fanatical over-statement of the facts can so offend credibility that prospective OSS users are turned away.

I was reminded of this problem by a recent article by Dennis Byron, writing for ebizQ. While I found the article both interesting and informative, the section on enterprise service buses (ESBs) illustrated my point. In the article, Dennis states

"ESBs are especially interesting because they may turn out to be the first category of software code that was OSS from the get go." And later, "The OSS movement in turn blocked those early non-OSS ESB market leaders before they could gain a lot of traction."

The 'early non-OSS ESBs' phrase refers to companies such as Progress, Cape Clear and Fiorano, according to the article. This is the type of hyperbole that in my view causes the OSS movement to weaken its own credibility. The implication is that OSS has ruled the ESB market from the start, apart from some minor incursions by early ESB vendors such as those listed.

Now, I will agree that OSS ESBs are maturing nicely - the Mule OSS ESB, for example, already lists a substantial number of production implementations on its website. I think that as time goes on, these OSS offerings will become more and more competitive to the commercial offerings. However, to state that ESBs were 'OSS from the get go' is just ridiculous. For the first three years of the ESB market's life, the only serious usage was with commercial offerings. While it may be true that some of the early players failed to gain much traction, there are now commercial ESBs available from all the major vendors such as IBM, Oracle, SUN and BEA, and some of these are performing strongly in the market.

I believe the OSS cause is best served by a more realistic assessment, without all the sauce. We are now at a stage where OSS ESBs are a viable choice alongside commercial offerings. There are references for successful usage of OSS ESBs, and the future appears to be bright - for example, OSS solutions with a supportive community could result in pools of less common adapters being available to satisfy a wide range of needs, something less likely to happen in a commercial world. But to state that commercial ESBs never made it, and OSS ESBs ruled the roost from the start, is in my opinion a wild exaggeration.

To be fair to Dennis, later in the article, while discussing message-oriented middleware and the impact of OSS offerings here, he states

"But it will be well into the period 2011-2020 before commodity MOMs/ESBs significantly displace IBM MQSeries and like products."

This comes across as more realistic, and hence makes a far more credible claim. MOM (Message-Oriented Middleware) products like MQSeries are much more extensively deployed that ESBs, with many thousands of users, and therefore even if there were attractive OSS alternatives (and this is definitely arguable today) it would take some years for any significant displacement to happen.

Steve 

The data deluge: new surf-board required?

Another report (this time from Tower Group) is highlighting the likely increase in data volumes that enterprises will need to handle over the next couple of years - a 900% increase in financial market data by 2012.  Of course, increasing amounts of data flowing around the enterprise is hardly new (in the mid 1990s, I remember being told in hushed tones at one of the baby bells that their databases were several terabytes in size).  Each time enterprises have been faced with such an increase, decisions have been made about what needs to be done:  Is the focus on integration, storage or analysis and can we use existing technologies or do we need/want new technologies? 

In the case of The Tower Group  report, the focus is on financial services and in particular the increase in market data handling requirements caused by new regulations (the EU’s MiFiD and US’s Reg NMS – both intended to create fairer markets).   Specifically, there is a regulatory requirement to store the data (in a manner which can be used as evidence if there is ever a compliance issue).  As these regulations relate to best price execution, this in itself will be onerous because every single published quote available when each trade is executed must be stored as well as all other relevant data associated with the trade.

Similar scenarios (where there is a flood of data or a potential one) exist in many industries – from the online games with millions of players generating massive amounts of data to security monitoring devices for government agencies to RFID tags.  However, it is too easy to react by announcing that the data deluge is coming and some new technological surfboard is needed. 

For a start, in many cases the data may have little potential value and storage may not even be needed.  But of course this will not always be the case.  To quote from the Bob’s guide coverage of the press release:

“Regulatory compliance will pave the way for firms to completely automate the trading process,” added Price. “Once all the players have done so, the winner in the hunt for liquidity will be whoever can process the data the fastest.”

To put it another way, in the case of market data there may be business justification for doing more than storage for forensic reasons either in the realm of integration or analysis of the data streams.   In my other examples there will also be opportunities – again in the realm of integration or analysis. 
The second hurdle is whether new technology is needed. 

Simply saying that there will be an increase in data and a need to do something with it is not enough to justify new technology as old technologies may continue to scale.  Going back to my baby bell experience, this was around the time when Object databases were being touted as the relational db killer – as only Object databases could possibly handle that size of database. 

In the case of the “data deluge”, the real-time processing of streams (when the value of the analysis is high enough) is certainly one area where a new approach seems needed – and is an area targeted by the likes of StreamBase and Progress' Apama products.  For analysing the increasing massive data warehouses and for integration, I think the jury is still out – although the increasing complexity within the data may throw a spanner in the works.

Ronan