Aberdeen report highlights need for proper testing of SOA: Surprise

I generally don't get too upset about surveys stating the obvious - what is obvious to me may not be to somebody else and vice versa.  However I think Aberdeen's report on SOA testing as reported on by Joe McKendrick in his ebizq blog needs some comment.  It should be said in the report's defence that it does make a valid if incredibly obvious point.  To quote Joe:

"New research from Aberdeen Group shows that to effectively test and provide QA to SOA, companies are testing not only vertically (using unit and functional testing as their benchmark for quality) but are also testing horizontally across an entire business process. In addition, in successful testing/QA environments, there is close involvement of the business user in more phases of the development lifecycle"

And from the report itself:

"It isn't enough to just deploy automation, and it isn't enough to simply rely on functional tests.  QA for composite applications needs a horizontal, process-oriented view, not the vertical unit-test methods used in the past."

The insight is what any professional software developer has known to be good practice for years: when you deploy a new system you need to do "integration testing" and "system testing" - testing all the pieces of the final system work together and checking that it works against spec (which should involve business users).  If the system is distributed the testing will need to cover that added complexity (the horizontal, process-oriented view as referred to).  That this is promoted as something new amazes me. 

Don't get me wrong on this: In SOA, testing is an even more important than usual aspect of your software development process because a fault introduced in one service may result in down stream problems across multiple systems which use that service.  As many organisations adopt SOA, they will need to scale-up the sophistication of how they test software and detect problems in live software.  These are areas which certainly justify survey based analysis.  However, Aberdeen's conclusions as reported by Joe seem to stay in the realm of the obvious.

Ronan