LiteBytes from Lustratus Research

Insidious WSDL

I was at the Integration Consortium (IC) European seminar in Brussels yesterday, and saw a fascinating presentation from Wells Fargo Bank that introduced me to the concept of insidious WSDL. Among many interesting points that came out of the presentation was a reference to the problems that WSDL and its use was having at the major bank.

Apparently, Wells Fargo is one of the most advanced SOA users in the world, running many millions of SOA services every month. The architecture has been built up over a number of years, and WSDL has become the standard way to describe services - that is, the interfaces and what they do. However, an interesting phenomenon is starting to make itself felt. The service developers look at the rich WSDL language, and take the view that using it is quite sufficient to make the service is understandable and usable by other users and departments. Where they would previously have communicated to others to explain details of the service and how it works, they now feel that WSDL has replaced this requirement. The result is that communications is falling away, causing problems.

The problem is that just knowing the specifications of a service do not give a full semantic understanding of the nature and characteristics of the service, and special factors to consider in its usage. But with the crutch of WSDL to lean on, service developers think their job is done once the WSDL is created. WSDL is exerting an insidious influence that is slowly impacting internal communications to a greater and greater degree, with potentially damaging effect.

I guess the message is simple - WSDL is fine as a mechanism for specifying services, but do not let its influence replace the ongoing need for clear communications.

Steve

Examples of SOA successes in smaller banks

Two stories that demonstrate success with SOA in smaller financial service organisations show how it can be used to support very different business objectives.  Obviously, SOA is not a certain or easy win and require a degree of organisational and technological change to be successful.  However, it is nice to see some success with SOA emerging in organisations apart from the usual suspects of Wachovia or HSBC.

The first is EBS, a firm with 10% of the Irish home loan market that wanted to automate its relationship with its network of brokers.   EBS recognised that its mortgage products were actually made up of a set of different products (core loan, home insurance, life policy, legals etc), each of which potentially required separate systems to complete processing.  The final SOA system is able to integrate across all of these systems and across to the brokers.  The biggest change/benefit highlighted by David Yeates, the senior manager of IT architecture at EBS, is actually the mythical alignment of business with IT – although he says it a little differently:

“A huge amount of business knowledge exists in the IT department, but that tacit knowledge remained there,” Yeates says of the old way of working. “It [SOA] turns the IT department on its head: suddenly, from being in the back room, you have to be at the front. The issues for IT departments are absolutely enormous. IT isn’t a thing that’s holding people back anymore.”

Yeates also mentions another benefit of their SOA deployment – not only does it reduce time to market for new products, it also allow them to white label other providers’ products.  Which neatly leads onto the second example of SOA success: Webster Bank,  profiled by IBS Publishing.  Webster came to SOA from a very different direction:  It wanted to move away from its own core banking systems – which were under strain – and outsource to use Fidelity’s systems. 

The CTO, John Kershner states that the decision was whether to spend ‘x million dollars’ on point-to-point interfaces that would be difficult to support and maintain or take that money and invest it in new technology, to create an SOA infrastructure with the associated lower maintenance costs. 

Again, the article claims that Webster has had pretty much unlimited success with the project.  Reflecting what is a common experience, Kershner highlights the biggest challenge as managing and coordinating as many as 30 different project teams over 15 to 18 months.

Ronan

Web 2.0 and Enterprise 2.0: What's that?

A recent item on SearchSMB provided an excellent piece of sanity counter-balancing the hype around Web 2.0 and starting to bubble up around Enterprise 2.0.  A speaker at the IDC IT Forum and Expo in Boston asked his audience how many of their employers were using Web2.0 technologies:  The answer zero.  The writer points out that this is in stark contrast with IDC findings that around 45%  blogs, 43% RSS and 35% wikis (the core technology components of Web2.0  by most definitions).  Unfortunately, he goes on to claim that the divergence is because rogue users are experimenting without telling the IT managers – a conclusion which I find a little implausible.  It seems more likely that the 0% rate is probably too low but the other findings are much too high (reflecting the usual over-statement when asking people soft questions like ‘experimenting’ and ‘piloting’ or ‘planning’ as I have previously blogged about).   

Of course, there is real value in Enterprise 2.0 – taking the Web2.0 technologies and philosophy of user engagement and putting them into the work context.  As a starting point, I would mostly ignore blogs, and instead focus on wikis and RSS and of course AJAX-based mash-ups if you regard them as part of the web2.0 palette.  For those of you who have yet to get to grips with Web 2.0 concepts, look at the now famous and still excellent article by Tim O’Reilly here.  On the Enterprise 2.0 side, Don Hinchcliffe has a blog worth tracking.

Ronan

Complex Event Processing (CEP) and SOA: Sitting side by side in finance

A perennial SOA discussion is whether it is a universal architecture or whether there will be complementary or even competing architectures.  This article in Water’s Online focuses on this issue in the context of where SOA is being used and more importantly not being used.  While the tone of the article is probably negative to SOA, it is interesting to notice how established SOA is in the large investment banks with quotes featured from both HSBC and Bear Stearns.

To briefly cover the argument:  SOA has obvious overheads if messages are encoded in XML and used synchronously.  This makes it unusable in some of the trading and market data applications where the volumes of data are huge and speed is everything.  In this space, Complex Event Processing (CEP) tools such as those from Progress Apama and Streambase are clearly more suitable than SOA based on business processes and synchronous processing models.  To quote a well balanced view from a senior IT person at Bear Stearns:

"The more glamorous parts of the trading system typically involve super high-performance networks, data-distribution models, and technologies such as CEP that are 'in line' with information coming from and going to the markets," says Moschetti [chief architecture officer at Bear Stearns]. "But other aspects of the system, such as entitlements, configuration, loading of baskets, lookups on reference data, and so on, can use what the industry conceives as SOA—that is, Web services, XML over message busses, and so on," he adds.

Therefore, it isn’t a matter of one approach dominating the other is some strange Darwinian struggle.  For industries where the processing requirements include real time streams (and in particular where the stream volume and complexity exceeds what ‘static’ approaches based on variations on data warehousing can handle), CEP will have a bright future.  That means at the moment: front office in financial services, some government security applications and Massive Multiplayer Online Gaming (as shown by Streambase’s recent announcement).   Apart from that CEP has an important role to play in control and monitoring within the SOA layer as Steve has previously blogged about.

Ronan

Will SAP buy IDS Scheer?

I notice that the relationship between SAP and IDS Scheer, the purveyor of the popular ARIS process modeling solution, continues to get tighter. This is good news for SAP users - the ARIS tool is powerful and one of the leaders in the market, and aligning it as closely as possible with the SAP application suite will make it easier for SAP users to implement business process-based integration solutions, as I discussed recently in my review of SAP's SOA-based process-integration approach, published at Lustratus.

However, it does raise another more provocative question in my mind. The two companies have such a tight relationship, they are both based in Germany, and process modeling and BPM in general are likely to continue to grow in importance to SAP in the future. IMHO it would make sense for SAP to acquire IDS-Scheer - it looks a good fit, both from a solution perspective and culturally, and would protect what might become a core competency for SAP. One problem might be that ARIS is used by many solutions as a process modeling environment, and it is possible that if it were owned by SAP then this might make it less attractive for an SAP competitor to use it. However this seems a minor drawback. Of course, it wouldn't be cheap, but IDS Scheer continues to perform well....

Anyway, regardless of whether such a thing were to happen, it can only be good for SAP users to have the relationship continuing to strengthen.

Steve

OK, start SOA small if you want - but don't skimp

Suddenly everyone is saying that SOA should start small - work incrementally, get a few small wins under the belt, benefits grow over time. Joe McKendrick summarized it well, with vendors such as BEA, Intel and HP jumping in. In a sense there is nothing new here. Incremental approaches to integration have been the way forward for a number of years now. Admittedly, with all the hype around SOA and its ability to 'transform the business', some companies have got seduced into trying for a full-scale top-down strategy, but generally companies start small because the complexity stops them from doing anything else.

However, there is an important warning here. Be careful the SOA initiative is not strangled at birth. I have written before about the need for an SOA ecosystem or infrastructure, and indeed there is a (free) paper on the subject available from Lustratus. The problem is that if you start a small SOA project in the belief that it is OK to skimp on the ecosystem until more projects are implemented, you are heading for trouble. For example, suppose you decide to start with just web services for the first few projects. Then, the services created will be built without a registry in which to record them - so others can't find them and reuse them, because they don't know they are there. And once you DO decide to get a registry, will all the existing services be recorded in there, together with all the detials and information that will prove valuable in the future?

Start small if you want - that's fine. In fact, it's a good idea. Just make sure the basic SOA ecosystem is in place before you start. To misquote an old proverb, start in haste, repent at leisure.

Steve   

Should BPM or SOA come first?

I enjoyed reading Lorraine Lawson's excellent article on whether BPM or SOA should come first. Her conclusion was 'maybe' - in other words, if you are looking to provide a solution to human-oriented process flow issues, then BPM is best first, while if you want to share business services for new business needs, then SOA is best - but you will probably need both.

I have to add something to this discussion, however. BPM is a term used and misused frequently in the marketplace. Once upon a time, there was Workflow - this was used to refer to a solution area addressing the needs to primarily address human processes, often starting out from paper exchanges between departments / people. So, for example, sending an email with attachment to the next person in a work chain rather than passing on paper is an elementary automation. Actually having some code that knows that when you want approval of a purchase order, copies need to be sent to both the supervisor and audit department takes this a step further. Enabling problem reports to be entered on work queues which are determined based on geographical location might be yet another step.

Workflow products and solutions have been around for years. Then, as part of the EAI (enterprise application integration) shift, the idea came about that once different pieces of code can be easily (or at least with less difficulty is more accurate!) integrated, it becomes possible to dynamically link different IT components representing steps in a business process - ie BPM. In this flavour, BPM referred to short lifespan operations, where linkage was in the second or subsecond range, whereas in workflow solutions the process might extend for days or weeks. Then people started using the term BPM to refer to both forms of process integration.

So, back to the base question. Well, if what you want to do is to take a well-understood human-oriented process and automate it, then workflow / BPM is what you need, and there is no obvious reason why SOA would be a pre-requisite. However, if you are looking for process integration and automation as portrayed by dynamic flowcharts and decision trees that relate process substeps to IT components, then in order to achieve this you have to understand each component or chain of components and how they relate to specific business operations. This is something most companies do not have at their fingertips. On top of this, in order to relate changing a process flow to altering the underlying IT implementation, you have to be able to make the different IT components link together. Working all this out and providing a mechanism to relate back to the BPM modeling tool means that a lot of the work to develop an SOA and its services has to be done anyway.

After all, an SOA service is directly related to a piece of business functionality, and you have to break things up like this to use BPM in its wider sense. Therefore it makes sense to do SOA first. Otherwise your drive towards a BPM strategy will founder on the inability to execute at the IT operations level.

And before I finish, I just have to comment on the idea that 'reuse is a useless financial justification for SOA'. This is a line David keeps trying to push, but the fact is that reuse still remains one of the most common factors in SOA justification. Reducing redundancy provides a clear and measurable way to avoid cost, as companies like Wachovia have shown. I do not disagree there are other factors that can be brought into the business case, but reuse remains one of the clearest to identify in monetary terms.

Steve

Notes from the bleeding edge: Data models, Financial derivatives and the future of SOA.

Golden Source, a vendor of enterprise data management solutions for the financial services industry, has just released a report  highlighting the major problems associated with the management of data connected with some classes of financial instruments (those known as Over The Counter or OTC derivatives).   In particular, it focuses on the strain placed on the existing data models by the need to roll-out new types of financial products ever more quickly.  The relevance to SOA and enterprise integration is two-fold:

• Data models are core to SOA, whether defined implicitly within services or explicitly as part of a separate data modelling exercise.  As the scope of SOA grows within an organisation, the challenge is to balance the need to create global models (and hence reduce the need to translate between models) and the need for different data models reflecting real differences between business units (and hence provide powerful transformation capabilities in the infrastructure).  In the case of business-to-business integration, as is the case with financial trading, there is a need for an agreed data model which is used to exchange messages between the two parties.  FpML is a leading example of such a B2B format which addresses derivatives in particular.

• As the level of integration increases, there are more messages flowing through the network and more complexity in data models and hence more complexity in the transformation between data models.  An objective of SOA is to deliver agility – to allow business to change unhampered by the traditionally slow moving integration capability.  Therefore as SOA grows, the data modelling and transformation challenges move centre stage.

The market in complex financial derivatives (bundled combinations of multiple types of financial instrument designed to have specific risk and return profiles) shows a potential future for SOA in terms of data complexity and rate of change.  However, it should be noted that it is unusual in a number of ways: as with all financial instruments, they are entirely virtual; with a fast moving and growing market, advantage is firmly with the organisation that can create and roll-out a new type of derivative and they are highly complex in their structure.  Nevertheless, the challenges faced in this problem domain should be indicative of the challenges that will be faced the most advanced SOA adopters.  The key findings of the Golden Source commissioned report were:

• Most of the manual data management effort goes into fixing the 1%-2% problem cases.   In the SOA context, this statistic should focus the mind and justify the business case for prioritising data modelling in any wide-scale SOA programme.
• Flexibility is essential in creating data models capable of changing as business changes and markets become more complex.  Data modelling is not about making a perfect representation of the business frozen in time – it is about making something sufficient for today which is built to change and extend as the business changes.
• While there is a clear need for industry wide collaboration in defining base models, the report notes scepticism that it will actually happen due to lack of incentives for each institution to share information.  While there are huge economic drivers towards standardisation within financial services, the natural suspicion and lack of trust between competitors has meant that the significant progress towards standards has taken a long time and has been painful process.  Therefore, it is reasonable to assume that most other industries will take a lot longer.  In the SOA context, this means that organisations should not wait for standards to emerge but rather focus back onto addressing their own needs and build data models in such a way to ensure easy integration to other organisation’s models. 

Ronan

The true nature of SOA

In a recent article in Redmond Developer News, William F Zachmann, claiming to be an SOA cynic, attempts to brush SOA off as no big deal. He writes

"Bottom line: SOA simply means designing application systems that create and access local or network services to meet the objective business needs of the enterprise."

In fact, surprisingly (!) he goes on to state that all you really need is .NET and Web Services....what was the name of the magazine again?

There are some points in the article on which I would agree. I am a great believer that there is much obfuscation going on in the marketplace over and around SOA, but I hate to say that this article is just another example. It goes on to say

"SOA is no paradigm shift. The underlying concepts and architectures are the same as those of minicomputer-based distributed systems in the '70s, PC-based distributed systems in the '80s, CORBA and COM/DCOM in the '90s, and JavaBeans and .NET today. The basic architectural ideas date back to the '60s. So what's the big deal with SOA today?"

I am all for keeping things simple, as anyone reading this blog or any of the papers provided at Lustratus would, I hope, realize. But Mr. Zachmann has missed the fundamental point about SOA that makes it a bit more than just a smart type of structured programming. As I have written about before, SOA services are designed with boundaries that satisfy a business rather than a technical context. In other words, the service executes a discrete piece of business functionality as opposed to a set of technical operations. It is this business context that delivers the improved business visibility of operational systems and improves the alignment of IT with business.

Steve

Is SOA just a flash in the pan?

I was just reading a thoughtful piece by Clive Longbottom entitled 'SOA- Strategic or arbitrary', in which Clive was discussing views of some who are predicting the death of SOA. My own view is that, although the phrase 'service-oriented architecture', and the acronym 'SOA' might disappear from common usage, the shift in thinking ushered in by SOA is permanent.

I see it as a similar situation to OO, Object-Oriented. Back in the 80s (wow, that DOES seem a long time ago!) software engineers were raging about OO, and the related 'objects'. Eventually, as the world turned and the market moved on, OO became rather a dirty word, and fell from usage. But its impact remains.

The reason is that at its underlying core, OO was about sensible programming discipline. What is was really all about, when you cut through all the obfuscation the IT market frequently generates, was about separating code into self-contained chunks, with a clean input and output. In fact, ancient COBOL programmers would have told you at the time that it is really just what used to be called 'structured programming'. The driving force is that by building objects where code and what it does is self-contained, you get an improvement in quality as well as the opportunity to share or reuse pieces more easily.

Now switch to SOA. Despite all the talk about ESBs, Adapters, BPM, Registries, Repositories, Brokers and so on, what is SOA REALLY about? It is about extending the OO thinking to relate these self-contained pieces of code with pieces of business functionality - the SOA service - and providing a 'simple' (or should I say simpler!) discovery and invocation mechanism, in a similar way to OO with its objects, classes and methods. The result is sharable/reusable services with boundaries that fit in a business as opposed to an IT context.

This gives real benefits, if you can achieve it. But even if the market moves on to something else, and drops the SOA word, the basic concept of shared services with business as opposed to technical visibility has so much to offer that it will not be thrown away. Like OO, it will continue to play a role in whatever acronyms and buzzwords come next.

Steve