AMQP – Great idea, but it will never work
The recent revelation of the secret work…
…on AMQP (Advanced Message Queuing Protocol) has got lots of people talking in the integration industry. Basically, the idea is that a bunch of folks such as JPMorgan, IONA and Cisco have got together to work on an open source solution for guaranteed messaging. Currently, the market for multi-platform guaranteed once-only messaging that underpins all of SOA and business integration is owned by IBM, with IBM’s WebSphereMQ owning in excess of 80% of the market according to IDC. The theory is that AMQP will commoditize the market and supplant MQ.
Now, I need to declare my conflict of interest up front – I ran IBM’s MQ business for its first three years int he market, and therefore my independence may be called into question. However, I believe I can take a balanced view, and the fact is that there are some significant problems with this AMQP idea, and I think these are serious enough to be given close consideration by anyone thinking of going this route.
The first, possibly minor, issue is that I personally was confused about the integration model being proposed, and I fear that it may cause companies to have to unpick their architectures. I am no expert. but from what I can read, AMQP is not just a messaging wire as MQ is. It also does other things like content-based routing. This is part of the functionality typically provided by ESBs and the like, along with other mediation functions like transformation. But AMQP doesn’t do transformation, so it seems to suggest a redrafting of architectural diagrams.
The second is that for this to be valuable, it needs to be multi platform. One reason for MQ’s success is that it can connect many different platforms, since it runs on them all. AMQP needs to do the same, and I am sure that technically it may be able to – BUT!
The problem with multi platform support is not how to do it, it is the testing! Someone has to invest in testing the product across all the platforms it supports, and the combinations that can be made up out of the base set. AMQP is open source – so who can afford this sort of investment? Is someone like Iona or Cisco committing this sort of test expenditure on hardware, resources and time? Or is the assumption that the user community will do this ‘on the job’?
Thirdly, guaranteed messaging is almost by definition the backbone of a company’s mission-critical operations. If they weren’t critical, then guaranteed delivery wouldn’t be so important. So, where is the AMQP ‘throat to choke’? Who do you run to when it fails, knocking out your key online systems? And going back to the testing question, who will invest in the huge expenses involved in stress testing and performance tuning to ensure it can meet the demands that will be placed on it?
AMQP is a great idea – but in my view it is doomed before it has got started.
Steve
Related Posts
Posted in 6 Responses to “AMQP – Great idea, but it will never work”
Leave a Reply
Lustratus LiteBytes Blog Feed
- PIPESCOM – Because the world needs yet another acronym…
- New Report From Lustratus Research: A Competitive Review of SOA Appliances
- Four Words
- Positioning, depositioning and the UK General Election
- Lustratus 2010 Predictions for the Infrastructure Software Market
- A Comment on 10 Defining Points for Cloud Computing
- Cloud Computing Explained – Without the Hype
- A Lustratus REPAMA Guide to the Positioning Statement
Steve is right. We used to joke that Java was “write once”, “test everywhere” – and the same premise now holds true for Web Pages – IE, Safari, Opera, Firefox – just to name a few – why should this scenario be any different for complex messaging ?
The other key component that needs to be a part of the puzzle is systems management – you won’t know that your messaging infrastructure is in trouble unless someone – or some thing – is watching it.
Hopefully, the only people peering over systems management monitors are air traffic controllers – the rest of us trust software to monitor our systems, to keep us informed – and to automatically fix simple problems – lets see some open source for that before we get too excited about open source assured message delivery.
Oh – and another caveat – Steve used to be my boss – caveat lector.
Declaration: I Chair the AMQP Working Group, and came up with the idea for it.
AMQP is a *protocol* (not a product) which tries to plug a gaping hole in the standards landscape. There are open standards for networking, email, file systems, operating systems, instant messaging, hypertext, media…. but there is no cross-platform standard for MOM.
The absence of a mulit-platform standard with tight semantics (JMS is no use to VB) means no easy switching between products for end users. Therefore no real competition in cost, and little urgency for vendors to improve product quality.
As an open protocol AMQP gives enables academics to study it (this has already begun), innovators to enhance it, and anyone to implement it.
The AMQP protocol is now implemented in many products: Apache Qpid, iMatix OpenAMQ, LShift RabbitMQ and Iona Celtix AM with more arriving shortly.
Platform support is pretty broad too: brokers are available in Java, C++, C, and Erlang. Clients are available in Java, C/C++, C#, Python, Perl, Ruby, and Erlang. They run across Linux, Solaris and Windows (so far). The technology has been used reliably in mission critical systems for >6 months.
Those software products have appeared in the 8 short months since AMQP was announced. It’s still quite early days, after all it took protocols like SCTP 6 years to make it into operating systems. Qualification will also be tricky, but the desire to make this work is strong and the momentum is there.
I think MQ-Series is a good product and it would be terrific if IBM implemented an AMQP protocol driver for it. There is absolutely nothing to prevent IBM from doing this if it so wished.
For those curious about AMQP it’s worthwhile reading the FAQ http://www.amqp.org/tikiwiki/tiki-view_faq.php?faqId=1 .
Best regards
John
Steve,
.
.
I can see at least 3 inaccuracies in your post:
1. [AMQP] also does other things like content-based routing. This is part of the functionality typically provided by ESBs and the like, along with other mediation functions like transformation. But AMQP doesn’t do transformation, so it seems to suggest a redrafting of architectural diagrams.
No, just because AMQP proposes to perform *some* of the functions performed by ESBs (i.e., content-based routing) does not mean it has to perform *all* of them (e.g., message transformation). No redrafting of architectural diagrams is required, thank you.
2. AMQP is open source – so who can afford this sort of investment [in testing]?
As was already pointed out, AMQP is a protocol, not a product. It currently has some Open Source implementations, but there could be proprietary implementations as well. Your former employer could choose to implement AMQP support in WebSphere MQ, if they choose to
3. There is no AMQP throat to choke.
Well, is there an HTTP throat to choke? I understand that hasn’t blocked its popularity.
Peter,
Any number of jokes aside, Java *is* “Write Once, Run Anywhere”. That’s why it’s so hugely popular. It’s nowhere near as fragmented as you imply. Yes, there will be some compatibility/interoperability testing required for AMQP, but this is hardly insurmountable.
Cheer up, MQ could do with some competition
Ganesh
John, while it is true there is no cross platform message oriented middleware, we already have the DDS (Data Distribution Service for Real-Time Systems) open standard specification from OMG. OMG is the Object Management Group, the standards body that is responsible for UML. DDS is platform neutral, language neutral, and location independent (e.g. shared memory or UDP transport plug-ins). Unlike AMQP DDS does not require a central broker which could represent a single point of failure. So I am just wondering why AMQP folks never seem to mention DDS, it’s almost like re-inventing the wheel, maybe the founders were not aware of DDS when AMQP was conceived? Or wanted something to be message oriented rather than data oriented?
Now, this is a late reply!
@Thorlin. I looked at DDS before embarking on AMQP (I also looked at a ton of other protocol; I really didn’t want to do this *again*!). I also looked at XMPP, and many others.
Two things:
1) DDS appeared, at the time, to be a mysteriously encumbered. There was one commercial implementation, no open ones, and dark hints at patented technologies around the DDS spec which the OMG licensing permits. I see just this year PrismTech has done something open source; but 4 years after I started looking….. and LGPL may or may not say anything about patents.
2) DDS appears to be more about event propagation and less about store-and-forward. The bias in AMQP is the other way around – it’s pretty state heavy.
I like DDS, if the protocol had been genuinely unencumbered it was a contender. But this was not the business model of its sponsors at the time.
Have you noticed in AMQP that every company contributing has already granted all necessary patent rights they hold to other users and implementors? Even IPR-heavy firms like Cisco, Novell and Microsoft.
Best regards
John
As someone who has worked on DDS from an implementation perspective as well as an OMG standards perspective, I hope I can offer some clarification to the issues John has raised.
Let me go in reverse order:
2. In regards to the communications paradigm: This probably isn’t the right forum for a prolonged architectural discussion, but in general you’re correct, John. DDS is about peer-to-peer data-centric distribution, not about a store-and-forward message passig. I think this gets back to the distinction Thorlin originally made between message-oriented vs. data-oriented communication, which does have implications for where your state gets maintained.
1. In regards to patents: The OMG requires any implementers participating in its process (including RTI, PrismTech, and others) to provide fair and and non-discriminatory terms to all users as well as to all other implementers for any relevant patents they may have. Speaking from pure self interest: creating and implementing standards is expensive and time consuming. The vendors who work on OMG specifications do so to increase the size of the ecosystem in which our products play. We *want* people to implement those specifications! It would be completely self-sabotaging if after, say, creating the specification for DDS interoperability we then acted to prevent anyone from interoperating with us.
1. In regards to DDS implementations and their business models: I can’t say what you found four or five years ago, John, but I’m happy to say that today there are a number of DDS implementations available under a variety of free and commercial licenses. See http://www.omgwiki.org/dds/category/web-links/vendors for an incomplete list. The open-source implementations differ in the sizes and cultures of their respective communities, and the closed-source implementations differ in whether and how they provide source and the level of support they offer. Each DDS user can decide what makes sense for them.
Regards,
- Rick