LiteBytes from Lustratus Research

Breaking the SOA logjam

One of the 2008 forecasts in the annual Lustratus look ahead is that SOA decision-marking has become fractured in most companies, with clashes between architects / IT who 'get it', and business-oriented budget holders who don't. In fact, this problem is turning out to be so severe that it is causing SOA adoption to stall at many companies - although enterprise-wide decisions may have been taken to adopt SOA, projects steadfastly refuse to enact this because of the extra costs, at least initially.

The roots of the difficulty here are twofold: understandable cynicism and the need to reach critical mass of SOA deployment before benefits start to show. However, there may be a light on the horizon, as discussed in more detail in the Lustratus Whitepaper, 'Justifying SOA to the Business', available for free from the Lustratus webstore. For business audiences, it may be that the enhanced business visibility offered by SOA could be a compelling benefit to justify the extra investment, and this visibility becomes apparent immediately the project is complete - there is no need to wait for critical mass to be achieved before seeing the benefit.

Hopefully, this angle of attack may succeed in breaking down the SOA adoption log-jam, enabling companies to flow smoothly to widespread SOA adoption.

Steve

Will mashups mash up your infrastucture?

One of the forecasts in the Lustratus predictions for 2008 Insight, available free of charge from the Lustratus web store, deals with the emergence and adoption of mashups. At this moment it is unlcear how fast mashups will be adopted, but Lustratus thinks that any serious adoption will place massive strain on enterprise infrastructures, causing the unwary to buckle and collapse.

Mashups seem great. The user is suddenly in a position to create his or her own page layout with all the business applications needed to carry out this user's activities. A great productivity boost, perhaps, but what are the impacts on the enterprise? Basically, as Lustratus points out, every desktop becomes an application. Instead of an IT department having to worry about 10 or 20 applications, all of a sudden there are 100s or even 1000s. Worse still, while traditional IT-controlled applications are usually controlled fairly rigorously with procedures, policies and management practices, the world of mashups could well be more akin to anarchy.

Fundamental to a productive mashup will be the need to drive the different business services required by the particular user, and therefore services will suddenly become tools used by hundreds in many different ways. All of this activity could create huge traffic increase as well as a generally uncoordinated style of operations, causing major difficulties for the infrastructure software trying to hold everything together.

Well, OK, maybe this is a little negative - but the point is, enterprise architects and management should start considering these issues now. Trying to sort this out when the genie is out of the bottle will be a lot more difficult.....

      Steve 

SOA Communism vs Corporate Capitalism

During a nice break for the Holidays, I got to thinking about SOA and its similarities to communism, and on my return I have been chatting to a number of SOA users and have confirmed my suspicions. SOA is just a communist plot!

Seriously though, there is a real issue here that is starting to affect companies as SOA penetration increases. Once upon a time, IT investment was funded primarily through a central IT budget with all business units having to share the burden, but over the years this became unacceptable to many who wanted a more capitalist view where funds came from the people generating the business and were justified based on returns. So nowadays IT budgets typically fall into two pieces; a central component for the IT 'infrastructure' that everyone shares, and project-based funding attached to specific business initiatives.

Then along comes SOA. In its simplest terms, SOA is about two things - packaging functionality in business services, and encouraging the sharing of these services across different business needs. But who owns these services / is responsible for funding them? Are they infrastructure? Well, not really because they are business-driven. So are they funded by a project? But in that case the project is now having to fund something being done 'for the good of the whole'.

This problem can be seen more clearly if we look forward. Imagine a company where SOA has become a way of life, and all applications are now made up of shared SOA services linked in different ways. Does that mean everything is now infrastructure and should be centrally funded? Then that takes us back to the centralized funding days, losing the link between business need/return and targeted investment. In reality, this introduces the principles of communism, where everyone owns everything, and the problem for companies is that this model stifles business performance and progress. Perhaps one way around the problem is for monitoring and management software to keep pace with the changes, so a clear picture can be built of which business operations drive which services. At least this would provide some more granular level of investment/return linkage.

However, my personal view is the problem will sort itself out - once the initial funding hurdle of SOA has been overcome, project funding to achieve a particular business end will be happy to build the required functionality in 'SOA mode' because it will be just as cheap as not doing this, and this will have the convenient by-product of helping other projects. In other words, the kick-back against SOA by projects today is based on being forced to increase investment 'for the good of others'. If SOA works out right, the future should see projects tomorrow investing less but also seeing others benefit, a much more palatable situation.

Steve   

Can SOA be bad for your health?

Recently I featured in a podcast and wrote an article on the 5 SOA Security traps, and one particularly sticks in my mind. The issue is about flexibility - a good thing, most people agree, but in security / governance terms it can be a two-edged sword, and so it proves to be in the case of SOA.

The problem comes down to security domains. IT implementations can be thought of as a group of structures with varying levels of security - all the way from a community village where anyone can wander in anywhere, up to castles with moats, drawbridges and even boiling oil! Imagine for example a company with a particular silo application which is highly sensitive and must be absolutely secure. This could be implemented on a high-availability cluster with hardware encryption, and even have physical access controlled by putting it in a room with locks on the door and a guard! Well, OK, this might a little over the top, but the point is the company can take whatever measures it sees fit to implement a high level security domain - think castle.

Now along comes SOA, with its philosophy of flexibility and shared, reusable services. Instead of running silos, applications become a linked set of services and logic, and the wonderful flexibility of SOA means these services could be running anywhere across the enterprise, on any platform and in any technology environment. So supposing there is a shared 'create customer' service, and the high-security application switches to using this service instead of its own redundant create customer code. Now, since the security is only as good as the weakest link, the security domain is broken. Someone just drilled a hole in the castle wall.

Of course, companies can take measures to ensure this disaster does not befall their critical apps. Procedures can be put in place to protect the integrity of the security domains, restricting changes to these applications and blocking them from SOA-based distribution. But many people are unaware of the exposure, and sometimes programmers, with the best intentions, might accidentally end up compromising operations. In the end, it is up to management to put in place any education programs, working practices and policies and then to enforce them. But at least forewarned is forearmed.

Steve

So what exactly is SOA governance?

If SOA is all the rage at the moment, then SOA Governance is the most frequently used term related to it as far as I can see. All the vendors, and many analysts, are talking about SOA Governance as the big issue, and tools are rapidly appearing to help with this governance issue.

I am wondering, however, if there may be an ulterior motive here. In business parlance, governance is an important word - it is definitely seen as more business than technical. So, is the word being hi-jacked by vendors keen to try to find yet another way to hook into the business world in order to secure the budget commitment to make product purchases? What is really meant by governance?

Governance seems to be all about defining expectations, controlling and granting powers, and measuring results. Funny that it is the root of the word Government, and a cynic might argue that these are three things that most Governments do NOT do well, but there you go. Anyway, in SOA tools terms this follows on to a number of different areas - project management tools / SLAs, security and management. Notice that these are implications at the tools level; some of the biggest aspects of governance are actually at the management process/procedure policy level.

If you now look at vendors claiming to play in the governance space, offerings typically fall into one or more of the following categories:

• Technical management tools - administration, systems monitoring, statistics
• Business management tools - SLAs, BAM, KPI management, dashboards
• Security tools - user authentication, authorization
• Project management tools - requirements management, policy enforcement
• Planning/education/training services to build the relevant knowledgebase and skills

In my view, the outcome of all of this is that it does not help to keep preaching about SOA governance, unless you offer answers to all these areas. I would find it much more helpful for vendors to say that they sell management tools, or business monitoring, or professional services, or project management assistance. The problem today is vendors are so desperate to hook into the business community that everything is lumped together as governance.

Is it any wonder users are confused about what they are being offered, and how it fits to what they want?

Steve

SOA Governance: "The beatings will continue until compliance is achieved"

Dan Foody of Actional gives an excellent real-world analogy for how not to apply SOA governance in his blog: The government of his home province of Quebec kept upping the taxes on cigarettes (in order to discourage smoking of course) but eventually:

"Then a funny thing happened [when the taxes were increased yet again] ... without warning, smokers started rebelling and they started buying cigarettes that had been smuggled over the border through indian reservations because they were much cheaper.

Within weeks it went from a few die hard smokers buying these smuggled cigarettes to double digit percentages of all smokers in the entire province doing it.  The taxes hit a flash point without warning,  and no amount of additional police work was getting the smuggling problem under control."

The analogy that he draws is the following:  If your SOA governance policies are working, this does not mean that more policies and controls will make the situation better - it may cause a revolt among the developers.  I strongly agree:  SOA Governance must be as light as possible - which typically means that you should start too light and build up carefully rather than attempt to design a complete set of rules at the beginning and then prune back when the backlash occurs.

Furthermore, governance procedures must engage and involve the participants and not be simply handed down from central IT.  As Dan also points out, this means that governance should include carrots as well as sticks.  If this isn't done and governance is exclusively a set of police enforced control mechanisms, at the first opportunity a project will be declared too important and urgent to follow 'all those stupid policies' and the whole of governance will fade away like so much smoke from a smuggled cigarette.

Ronan
 

Open Source and risk

The focus of debate on Open Source is too often focused on "its free" and sometimes overstated claims about software quality.  As everybody knows, the cost and risk associated with bringing anything into an enterprise go far beyond the license costs.  For OSS, a big problem is that by its nature it can bypass the controls imposed by procurement and the legal departments.  This can lead to a range of potential risks from IP infringement to plain old version control.  Of almost equal importance to the actual risk is the fact that the risk associated with OSS can be invisible  (as the OSS use will often not be tracked as licensed software would be) and therefore undermine the whole of IT risk management.

This article covers one approach to dealing with issue:  specialist software to analyse the Open Source software.  There are of course more straight forward alternatives:  Any vendor supplying OSS as part of a licensed product should be held to account to provide support and 'handle' the risk issues.  For 'pure' OSS, there are plenty of commercial organisations who will provide a degree of quality assurance and service guarantees around projects.  It may take away from the "Its free and I won't need to talk to legal and prodcurement" but do we really want staff bringing software straight from the web into deployment?

Ronan

SOA and its effects on Business Risk

SOA is a Big Thing - it transforms the business, it is a key strategic initiative, it aligns IT more closely with business goals, etc.  But this brings up an important issue for executives. How does SOA affect the business risk picture? Does it drive additional risks? Does it provide any mitigation?

Lustratus has just published a new paper, "The Impact of SOA on Business Risk", that looks at this subject in more detail. The paper does not try to come up with a definitive answer, but instead considers the strategic, compliance, financial and operational areas of business risk and comes up with a grid of effects generated by SOA adoption, providing a framework against which companies can carry out their own risk assessments.

I believe this is an important area for companies to be aware of, with little guidance available. Bearing this in mind, Lustratus has decided to make the paper available at no charge. But for those people who cannot take the time to read the whole paper, the broad conclusion is that although there are areas where SOA drives risk, on balance it mitigates considerably more risk than it drives, and on top of this the new exposures are largely manageable.

Steve 

IBM's Information on Demand streamroller gains speed with the Princeton Softech acquisition

IBM announced the completion of its acquisition of Princeton Softech - a company which focused on data archiving, classification and discovery software.  All of which sounds quite specialist until it is put into the context of IBM's Information on Demand (IoD) strategy.  Back in March, Ambuj Hoyal, who heads us IBM's Information Management division (with responsibility for the Information on Demand strategy) explained:

"... an inflection point occurred in 1996 when there were many techniques to create Web sites or do Web-based business... We are at a similar inflection point in 2006. We have myriads of techniques – metadata management, ETL (extraction, transformation, and loading) tools, data creation tools, Federation tools, cleansing tools, profiling tools. People use these tools to solve the information challenge."

To translate, IBM see a huge opportunity and are putting serious money into it - this acquisition is the latest of 21 which are part of this strategy (to see the list go here).  The opportunity is to build an information management platform which allows organisations to create, maintain and (most importantly) extract value from the myriad of data sources which flow around the enterprise.  Data cleansing, data distribution, data integration and master data management (among other areas) are each expensive activities but often have clear budget and value associated with them - this even before getting to semi-structured information which is also with the Information on Demand remit.  While there are best of breed solutions to different parts of the puzzle, there aren't single integrated solutions - which is what IBM hopes to offer.  Interestingly, IBM has yet to move on Business Intelligence vendors - it appears to have correctly realised that the major task is not creating dashboards; it is ensuring that what goes into the dashboards is correct and timely. 

Any familar with the area of enterprise data management will realise that the challenges inherent in building and deploying such a platform are formidible.  At a recent briefing IBM gave Lustratus, the whole area of data governance in particular was highlighted:  how do you organise structures and responsibilities to ensure that coherent and consistent data definitions can be used and reused through the enterprise (this should sound very familiar to anybody involved in SOA - just switch the word service for data!).  To figure out how to do this right IBM set up the Data Governance Council back in 2005 with many leading financial services and telecoms companies (among others).

Yet again getting into detail is beyond the scope of a normal blog - but I would recommend anybody with a passing interest in BI (or indeed enterprise architectures) to take a look at IBM's web-site on Information on Demand. Of course the strategy is not without obvious challenges:  The technology is from many different sources (even if it now all belongs to IBM) and there is a significant amount of complexity associated with solving such a complex problem.  Also, when there isn't a significant regulatory stick (Basel II for instance), I imagine it could be very hard to sell at a strategic level.  This is because while there are clearly valuable uses of Information on Demand, but there seems to be no common theme around which business momentum can be built.  And finally, its association with the term business intelligence may well go against it - already some analysts are wondering where IBM's query tools will stack up against Business Objects et al (not a relevant question as BO and others will sit on top of IoD) and in many cases the proposition is operational efficiency or regulatory compliance, not (to my mind at least) classic BI.

Ronan

SOA and Conway's law

Jim Webber's blog reminded me of the existence of Conway's Law which seems particularly relevant to the challenges SOA governance attempts to address.  For those of you who haven't come across this law, two forms of it are...

"Any piece of software reflects the organizational structure that produced it."

or a more techie version:

"If you have four groups working on a compiler, you'll get a 4-pass compiler."

The first is a good encapsulation of the reality of working in enterprise IT:  And SOA governance attempts to optimise the way in which software reflects the organisational structure through building expertise, capturing and formulating successful patterns of use, promoting 'good behaviour' and so on.

The second statement of Conway's law, while being more jokey, also brings in implications of human nature: If you set up teams with separate responsibilities, expect them to collaborate sufficiently to complete the job but also expect them to carve out their areas of control to the potential detriment of the overall solution.  This is true with all professions - however mixing in the stronger tendency to 'not invented here' with IT and the problem becomes even bigger.  The SOA equivalent of the 4-pass compiler is poor rates of reuse and multiple versions of what is essentially the same service.  Overcoming this again requires good governance - promoting communication between teams and incenting reuse - and good technology to support it.

Ronan