AMQP - Great idea, but it will never work

The recent revelation of the secret work on AMQP (Advanced Message Queuing Protocol) has got lots of people talking in the integration industry. Basically, the idea is that a bunch of folks such as JPMorgan, IONA and Cisco have got together to work on an open source solution for guaranteed messaging. Currently, the market for multi-platform guaranteed once-only messaging that underpins all of SOA and business integration is owned by IBM, with IBM's WebSphereMQ owning in excess of 80% of the market according to IDC. The theory is that AMQP will commoditize the market and supplant MQ.

Now, I need to declare my conflict of interest up front - I ran IBM's MQ business for its first three years int he market, and therefore my independence may be called into question. However, I believe I can take a balanced view, and the fact is that there are some significant problems with this AMQP idea, and I think these are serious enough to be given close consideration by anyone thinking of going this route.

The first, possibly minor, issue is that I personally was confused about the integration model being proposed, and I fear that it may cause companies to have to unpick their architectures. I am no expert. but from what I can read, AMQP is not just a messaging wire as MQ is. It also does other things like content-based routing. This is part of the functionality typically provided by ESBs and the like, along with other mediation functions like transformation. But AMQP doesn't do transformation, so it seems to suggest a redrafting of architectural diagrams.

The second is that for this to be valuable, it needs to be multi platform. One reason for MQ's success is that it can connect many different platforms, since it runs on them all. AMQP needs to do the same, and I am sure that technically it may be able to - BUT!

The problem with multi platform support is not how to do it, it is the testing! Someone has to invest in testing the product across all the platforms it supports, and the combinations that can be made up out of the base set. AMQP is open source - so who can afford this sort of investment? Is someone like Iona or Cisco committing this sort of test expenditure on hardware, resources and time? Or is the assumption that the user community will do this 'on the job'?

Thirdly, guaranteed messaging is almost by definition the backbone of a company's mission-critical operations. If they weren't critical, then guaranteed delivery wouldn't be so important. So, where is the AMQP 'throat to choke'? Who do you run to when it fails, knocking out your key online systems? And going back to the testing question, who will invest in the huge expenses involved in stress testing and performance tuning to ensure it can meet the demands that will be placed on it?

AMQP is a great idea - but in my view it is doomed before it has got started.

Steve

Comments

[Peter Rhys Jenkins]

Steve is right. We used to joke that Java was "write once", "test everywhere" - and the same premise now holds true for Web Pages - IE, Safari, Opera, Firefox - just to name a few - why should this scenario be any different for complex messaging ?

The other key component that needs to be a part of the puzzle is systems management - you won't know that your messaging infrastructure is in trouble unless someone - or some thing - is watching it.

Hopefully, the only people peering over systems management monitors are air traffic controllers - the rest of us trust software to monitor our systems, to keep us informed - and to automatically fix simple problems - lets see some open source for that before we get too excited about open source assured message delivery.

Oh - and another caveat - Steve used to be my boss - caveat lector.

[John O'Hara]

Declaration: I Chair the AMQP Working Group, and came up with the idea for it.

AMQP is a *protocol* (not a product) which tries to plug a gaping hole in the standards landscape. There are open standards for networking, email, file systems, operating systems, instant messaging, hypertext, media.... but there is no cross-platform standard for MOM.

The absence of a mulit-platform standard with tight semantics (JMS is no use to VB) means no easy switching between products for end users. Therefore no real competition in cost, and little urgency for vendors to improve product quality.

As an open protocol AMQP gives enables academics to study it (this has already begun), innovators to enhance it, and anyone to implement it.

The AMQP protocol is now implemented in many products: Apache Qpid, iMatix OpenAMQ, LShift RabbitMQ and Iona Celtix AM with more arriving shortly.

Platform support is pretty broad too: brokers are available in Java, C++, C, and Erlang. Clients are available in Java, C/C++, C#, Python, Perl, Ruby, and Erlang. They run across Linux, Solaris and Windows (so far). The technology has been used reliably in mission critical systems for >6 months.

Those software products have appeared in the 8 short months since AMQP was announced. It's still quite early days, after all it took protocols like SCTP 6 years to make it into operating systems. Qualification will also be tricky, but the desire to make this work is strong and the momentum is there.

I think MQ-Series is a good product and it would be terrific if IBM implemented an AMQP protocol driver for it. There is absolutely nothing to prevent IBM from doing this if it so wished.

For those curious about AMQP it's worthwhile reading the FAQ http://www.amqp.org/tikiwiki/tiki-view_faq.php?faqId=1 .

Best regards
John

[Ganesh Prasad]

Steve,

I can see at least 3 inaccuracies in your post:

1. [AMQP] also does other things like content-based routing. This is part of the functionality typically provided by ESBs and the like, along with other mediation functions like transformation. But AMQP doesn't do transformation, so it seems to suggest a redrafting of architectural diagrams.

No, just because AMQP proposes to perform *some* of the functions performed by ESBs (i.e., content-based routing) does not mean it has to perform *all* of them (e.g., message transformation). No redrafting of architectural diagrams is required, thank you.

2. AMQP is open source - so who can afford this sort of investment [in testing]?

As was already pointed out, AMQP is a protocol, not a product. It currently has some Open Source implementations, but there could be proprietary implementations as well. Your former employer could choose to implement AMQP support in WebSphere MQ, if they choose to :-).

3. There is no AMQP throat to choke.

Well, is there an HTTP throat to choke? I understand that hasn't blocked its popularity.

Peter,

Any number of jokes aside, Java *is* "Write Once, Run Anywhere". That's why it's so hugely popular. It's nowhere near as fragmented as you imply. Yes, there will be some compatibility/interoperability testing required for AMQP, but this is hardly insurmountable.

Cheer up, MQ could do with some competition :-).

Ganesh

[Thorlin Naysmin]

John, while it is true there is no cross platform message oriented middleware, we already have the DDS (Data Distribution Service for Real-Time Systems) open standard specification from OMG. OMG is the Object Management Group, the standards body that is responsible for UML. DDS is platform neutral, language neutral, and location independent (e.g. shared memory or UDP transport plug-ins). Unlike AMQP DDS does not require a central broker which could represent a single point of failure. So I am just wondering why AMQP folks never seem to mention DDS, it's almost like re-inventing the wheel, maybe the founders were not aware of DDS when AMQP was conceived? Or wanted something to be message oriented rather than data oriented?