LiteBytes from Lustratus Research

SAP relies on SOA to meet top customer need

SAP recently delivered a new wave of enhancements to its suite of business solutions, but as CNN Money points out, it is using service-oriented architecture (SOA) to address the top customer need of being able to bring new functionality on stream without impacting current workloads.

The point is that many companies use SAP packages to power their businesses, and anything that threatens to disrupt these key operations represents a major source of risk. So the problem becomes - how can you take advantage of new technology advances without jeopardizing your mission-critical operations? The traditional answer is to rely on extensive testing, both of the new function and, more importantly, regression testing to ensure existing workloads continue to function as expected. But the problem is that with large-scale deployments, such as often found with ERP packages like those provided by SAP, this testing cycle has to be so extensive that migration to a new product release can take years.

SOA offers another answer, as SAP has realised. Using a service-oriented approach, it becomes possible to handle migration activities in a piece-by piece mode. New functionality can be deployed, and new applications can take advantage of it, but at the same time these applications can drive functionality in the existing implementation 'transparently'. Also, as existing applications become ready to exploit the new advances, they can be changed bit by bit rather than lock, stock and barrel. 

I expect SOA to become used more and more by package vendors in particular as they strive to address what is often the top user requirement - don't screw up my current operations.

Steve

Which ESB for me?

The Enterprise Service Bus (ESB) came into parlance around five years ago, and in the intervening years it has matured. As a result, most ESBs offer common functionality, at least at a basic level. So how should people decide which ESB to choose when going through product selection?

When I wrote the first paper on ESBs, 'Best-of-Breed ESBs', all those years ago, the questions to be answered were all around the definition of this new thing called an ESB, and what functionality it supplied. In that paper, I offered checklists of functionality to identify whether a product fitted the ESB definition, and how well it covered the functional requirements. Of course, as the idea became popular the ESB definition got severely mangled by analysts and vendors as each searched for an angle, but the functionality has remained largely agreed at least at the basic level.

But now the ESB concept has matured, the functionality checklists are of less value - everyone ticks most of the boxes. Instead, users seem to be focusing more on the characteristics of different ESBs as they try to make their purchase decisions or validate their original decisions. So now, the differentiators between ESB suppliers tends to be about what I call the '-abilities' - that is areas such as scalability, manageability, availability and usability.  In other words, given that most ESBs 'do what they say on the box' in functional terms, it is now more important to understand how they will serve the production environments for which they are destined.

Expect more focus on the '-abilities' as ESB vendors strive to improve their competitive positoning. For anyone interested, there is a free Lustratus Insight on this topic, 'The Year of ESB-ability', available from the Lustratus webstore. We have also decided that the time is right to produce a new version of the original Best-of-Breed ESBs paper to take the new level of ESB maturity into account, which we hope to make available in the next few weeks.

Steve   

Proving the value of technology/business alignment

The holy grail for IT management has been to prove that IT investment makes the overall business more successful.  To those of us in the business this makes intuitive sense but can be hard to prove.  Unfortunately, those wishing to believe that IT adds nothing but cost have had a strong hand ever since Nicholas Carr famously kicked the IT industry where it hurts in 2003 in the Harvard Business Review by claiming that IT doesn't matter - backing up his claim with an analysis of business success against IT investment.  (To be fair, Carr argues that it is needed to be competitive but won't give one company a strategic advantage over another - which is a subtle point and one easily missed)

Of course one of the central planks of SOA is that of alignment between business and IT produces significant benefits for the business.  This again makes intuitive sense but can be hard to prove.  Therefore, I was delighted to see some research into the benefits of deep IT/business alignment by the BTM institute.  Their recently published report comparing Global 2000 with what BTM call "converged business technology management" and those without. While SOA isn't explicity mentioned, reading the report (available here) it is clear that a company succesful with a SOA strategy will tend to fall within this definition:

"[In such companys] This means that technology not only enables the execution of current business strategy but also anticipates and helps shape future business models and strategies"

In reviewing the study, CIO magazine highlights 4 dimensions:

"governance and organization; strategy and planning; strategic investment management; and strategic enterprise architecture"

Again very much inline with SOA thinking of what is important.  And finally, what is the win for converged business technology management?

“This is not about a project or ROI, but it’s about overall performance of the company,” Hoque [chair of the BTM Institute] says. For example, those companies with converged business technology management had 12 percent average annual revenue growth, compared with 4 percent for their industry groups. They also achieved 36 percent average annual earnings per share growth versus 7 percent for the industry groups.

Pretty impressive.  Of course, this report covers a period when SOA was only emerging and did not focus explicity on SOA.  What will be interesting to see over the next five years is whether firms where SOA is entrenched do better than those who have not taken the SOA route.

Ronan

SOA puts pressure on the network

One of the effects of SOA is that business transactions may move from silo operations to being made up of multiple services spread across the enterprise, all hooked together across the network. However, this can put a lot of extra pressure on the network.

On the one hand, this clearly introduces additional network traffic, as messages linking the business services together fly from one node to another. This is all additional network traffic - in silo mode, everything would be carried out on a single server or platform. As SOA accelerates, and reuse increases, this issue will grow correspondingly. So it is important to make sure that the network can handle all this additional traffic.

However, the other issue for the network is no less serious. Moving from one service to another introduces a whole set of new potential points of failure. If there is a problem with the network between any of the service nodes, the business transaction will be unable to run properly. Therefore, in order for the SOA approach to make sense, the network reliability must be at an acceptable level.

These two problems suggest that anyone looking to adopt and deploy SOA better make sure that they audit both the additional capacity available in the network, and the network availability levels.

Steve

SOA and the ugly bug ball

I was delighted to see Steve stressing the importance of file transfer in enterprise architectures today and for the foreseeable future.  This presents a challenge for any SOA program: to balance between the need to move ahead with 'modern' technologies and the need to take account of the reality that the enterprise world is full of file transfers and excel spreadsheets carrying out critical business calculations.  To be blunt: Any organisation intending to deploy a 'global' SOA, must figure out how these ugly bugs can be made to fit in. 

To look at file transfer first, there is no reason why you should not include file transfer as a transport.  As I put it in an article entitled "Including batch-driven applications in real-time integration projects" published back in 2005:

"When we consider existing transport protocols, and particularly the business processes they support, we are confronted with the reality ... that many of these are still batch based, even to the extent of using FTP or email to distribute non-XML documents such as Excel files. Rather than disrupting the entire organization, SOA implementations must accommodate and integrate - without necessarily enforcing change on the business processes involved."

and

"success [of SOA] will depend on how effectively the SOA can work alongside existing technologies. This means that the SOA must be capable of accessing whatever information, in whatever format, the application can expose"

Moving onto Excel, as we all know it is widely used as a platform for "edge applications" in the area of decision support and reporting - used by business line managers to make critical decisions.  These spreadsheets embed some application logic and a lot of data.  As I said in my ebizq blog a while back:

"Excel is an interesting product in that it often shows the rift between centralized IT and the business units. The reality is that within many if not all organizations Excel is used to make important and complex business calculations and Excel spreadsheets full of reference data and formulas are circulated by email between users and departments."

Therefore, any SOA strategy must also take account of whether it will extend to include these Excel based applications or at least explicitly exclude them.  All of which adds a couple of check boxes to the evaluation of any SOA software suite - and at least some of the vendors already provide good coverage for both file transfer protocols and excel.

Ronan 

IBM, DataMirror ... and Teilhard?

When IBM announced it wanted to acquire DataMirror, they probably didn't even notice that there is an ongoing lawsuit between DataMirror and Teilhard Technologies - after all, this 'battle of the midgets' would hardly register on the giant's radar. For the vast majority of people who don't know what I am talking about, Teilhard is a tiny Canadian company that has a patent to do with heterogeneous data exchange. I previously blogged about this when Oracle announced a settlement with Teilhard, earlier this year. Those people with a life probably do not know that this same company has a lawsuit running with DataMirror (actually, it has been running for three years now).

However, it seems to me that IBM coming into the game could well stir the pot. Would IBM mind if a company it had acquired were to lose an infringement case? Would it decide to get its own legal behemoth involved? If a settlement was agreed, would IBM's presence inflate the figures? In the end, would IBM care at all?

If IBM did decide to take an interest, things could get quite amusing. After all, although Teilhard is suing DataMirror, DataMirror has filed a counter-claim. If IBM chose to notice this tiny irritation and do something about it, perhaps this whole thing could backfire on Teilhard.

Steve 

Does SOA spell the end for file transfer?

OK - I confess it. I am a closet fan of file transfer. Maybe this is why I get worried sometimes that most SOA toolsets and advocates seem to ignore file transfer. However, as far as I am concerned I see file transfer as absolutely still having an important role to play, in no way replaced by SOA concepts.

The focus of SOA is heavily application-centric. It is all about getting application code and components working together - and some people seem to think that this will eventually remove the need for file transfer - after all, for many companies file transfer is a form of integration, and SOA deals with all that, doesn't it?

No, it doesn't. File transfer is still a key component of the integration armoury. The two main reasons are that file transfer offers a clean interface (yes, I know the purists will be killing themselves laughing at this claim, but int he practical sense it is true) between operations, particularly in the B2B (business to business) sense, and it is non-invasive. Could I not achieve the same results with SOA? Well, yes of course, in the B2B case I could implement some services to achieve what I wanted but the partner system would have to play ball. Applications in the partner system would have to be changed to call the services rather than operate on the incoming file as input. When one system receives information from another in the form of a file, there is no requirement placed on the receiving system on how it processes the file - only that it understands the format.

File transfer is simple and effective. In addition, although it is as old as the hills, file transfer tools have actually developed, now offering higher performance and throughput options as well as monitoring facilities and enhanced security and recovery options. And file transfer is ubiquitous - just about every company I ask makes use of file transfer.

I believe file transfer will be with us for a long time yet, so it is important that companies looking to move to SOA remember to factor in file transfer into the overall architectural picture.

Steve 

The delicate art of transitioning your software business model

I was recently discussing with a friend of mine in a VC firm what the biggest challenge facing young enterprise software companies was.  The answer from the VC side was simple but deadly: companies are still struggling to grow fast enough to generate the required return required to keep their investors happy.  Faced with a business model that is not delivering, businesses can fall into two equally fatal traps:  keep using what is in fact the wrong model for too long or jump too fast into another wrong model without planning for the required changes.

Going back 5+ years, the decisions were simpler as the revenue parameters were license fee, annual maintenance fees and consultancy/training with channel strategy providing the other dimension.  Today, there are much more fundamental choices to be made between the traditional enterprise model, Open Source, Software as a Service (SaaS) and even subscription based pricing.  In actual fact the enterprise model was far from easy on its own– and the appearance of the alternatives makes the job of selecting and tuning the right business model even harder.  Additionally, transitioning between these models can be extremely difficult: Coming back from OSS to license-based is nearly impossible, switching to SaaS requires significant business and technology changes.

The first trap to avoid in reviewing your business model is failing to recognise that in the future each of the models will continue to be viable for the right company at the right time.  Sometimes this can be a tough discussion as some investors have been convinced by commentators who seem to believe that the enterprise software license model is dead and only OSS or SaaS is viable.  More sensible commentators, such as Bill in XAware’s blog provide a plausible rationale for the greater diversity within a bigger picture:

The “enterprise software business model” that software vendors adopted during the 1990s and largely continued to use in this decade compounded this barrier to adoption of SOA. These models treat software as capital infrastructure, emphasizing large up front license payments and professional services heavy implementation. Given large up front costs to acquire new more productive technology and the low labor costs, rational buyers have stayed with older technologies and methods.

Bill believes that increasing skill shortages and labour costs will reignite the enterprise software market – all be it with a broader mix of business models.  Even if you disagree with Bill, it is clear that there is a more complex purchasing environment which means that there is no straight forward answer to what business model should be used.  In particular, it is not sufficient to cry the world has changed and we are all doomed– it is about analysing the value of the solution and how best to deliver that value to the customer and get paid for it. 

The trick is to keep the open mind, avoid religious fervour for any particular model and in essence follow good product management practises and focus on the 4 Ps: Product, promotion, price and placement.  Having selected the business model that seems the best bet for the business, the second part (the actual transitioning) is equally short of universal solutions:  It needs to be treated as the business relaunch process it is and planned for across every aspect of the business from dealing with existing partners and customers to ensuring the product works in its new business model. 

Of course this all sounds too easy and the decision and planning process can be a major piece of work on its own.  However, from our experience of participating and guiding such processes we find that even when the outcome is steady as you go, it can reignite faith in the business for management and investors alike.

Ronan

Microsoft moves into SOA as a Service

Microsoft recently made two SOA related announcements:  The next version of Biztalk will have a SOA veneer (commented on by Joe McKendrik ) and more interestingly Microsoft will lauch a  Software as a Service(SaaS) SOA offering. 

I blogged previously about how SaaS itself increases the requirement for integration as multiple SaaS based applications must be integrated together.  This has the potential to derail SaaS initiatives – particularly in smaller organisations without sufficient IT skills and budget to deliver on an integration strategy (SOA based or otherwise). 

Microsoft is taking a different angle on the SOA and SaaS story by focusing on cross-department integration and attempting to solve particularly painful aspects of the problem with SaaS offerings that will be called collectively Biztalk Services.  In essence, each service hits a specific integration issue which would otherwise require a potentially large investment in infrastructure.  The first two are:

• Identity: allowing management of users across departments and organisations and
• Connectivity: providing enterprise style message (pub-sub for instance) across the internet with appropriate security – thus making ‘safe’ exposure of a SOA service across multiple organisations.

Their picks of identity and connectivity for the first two services is smart as both are inevitably part of any cross-departmental SOA project.  The strategy is also smart as it neatly leverages where Microsoft is already strong (at the department level and in SMB) and where the SOA skills shortage is hitting hardest. 

As such I don’t think it is necessary to look at this announcement through Google-tinted glasses and disagree with Ron Schmelzer of Zapthink who is quoted as saying:

"I think Microsoft is really rethinking a lot of their server infrastructure because Google is a competitive threat,"

Microsoft is not doubt thinking hard about Google but it is only fair to point out that this is not a market where Google is relevant as yet and it is not a new departure for Microsoft or even Biztalk:  Biztalk was originally about internet-based integration as this article from 2000 shows.

Ronan

BPEL4People: mad or not mad?

I have a lot of sympathy with Steve’s blog item on the WS-madness:  Vendors will sometimes (inevitably) cynically use standards only for differentiation, standard bodies and the professional standard writers will do what they are paid to do: write standards.  And finally end-users are too disengaged to make sure the standards solve the problems they want solved.  All of which was going through my mind when I was reading the recently published BPEL4People standard:  Is BPEL4People just another piece of WS-madness?  (BPEL4People is an extension of WS-BPEL to handle human involvement in business process modelling and execution.  BPEL is the WS specification for defining business processes.)

To answer my question, I came up with the following simple sanity test (which many WS-standards fail):

1. Does it address problems that users have today or reasonably expect to have tomorrow?

2. Is it the unique standard in addressing the problem?  Anybody following of the web services standards around reliable messaging will remember the complete mess caused by multiple competing standards( WS-reliable messaging and WS-reliability).

3. Is anybody actually going to implement it?  There are too many standards both WS and previous which are never quite implemented.

In fact, BPEL has itself struggled to pass the three tests for sanity:  While business processes sound all pervasive, BPEL addresses only a subset of all business processes:  Those which have a central controller (i.e. not peer-to-peer collaborative processes which are covered by another standard), where the central controller is in the middleware (i.e. not controlled within your SAP application).  It has also been heavily criticised by BPM specialists who prefer more formalism than BPEL attempts to provide.  Therefore, it struggles to pass points 1 and 2.  These limitations have meant that many users are slow to adopt BPEL and are sometimes happy to stick with non-standard approaches with better capabilities.  All of which led to vendors delaying implementing BPEL for a number of years - which is point 3.  (However, I believe that BPEL does now make the sanity benchmark for those who need it and on that basis would have to disagree with Steve’s exclusion of its from his sane list.)

Turning to BPEL4People, the intention of the standard is to create a “BPEL extension to address human interactions in BPEL as a first-class citizen“.  To put it another way, it attempts to tie human-workflow back into the BPEL standard which is mostly used for system to system workflow (or business process flow if you prefer).  This means that BPEL4People contains concepts such as human roles, groups and tasks.

Returning to my sanity tests:  On the first test, BPEL4People certainly addresses a problem I have seen where some human interaction is required in a system-to-system business process.  It is a subset of the set of users of BPEL itself – maybe less than 50% of all BPEL use cases. 

A bigger criticism is at point 2:  You could (and many have) solved the problem BPEL4People addresses with BPEL on its own.  Yes BPEL4People provides standards where people made it up as they went along but it is not clear that the overhead of a new standard justifies its creation.  Finally, is it going to be implemented:  On the face of it, SAP, BEA, Oracle and IBM’s backing makes it a simple answer: “Yes”.  We will have to see.

All of which means that for the moment BPEL4People scrapes a bare pass on the sanity test:  If you are sure that your problem domain needs this type of functionality I recommend watching how implementations develop and planning on transitioning to BPEL4People when they are mature.  Otherwise, you already have standards and solutions capable of muddling through with.

Ronan