We're well underway developing the new search.active.com, and I thought I'd take a moment to discuss a realt-time, publish/subscribe messaging layer we're inserting into the Search architecture. In the beginning of my career I worked building distributed OSS/BSS applications integrating various line of business systems with products from a middleware vendor called TIBCO. During this time I was introduced to the concept of "[publish/subscribe|http://en.wikipedia.org/wiki/Publish/subscribe]", whereby an application sends one message that may be consumed by many disparate applications. For example, when a customer calls into a call center to order broadband internet, the customer service rep enters order details into a CRM system. When the order details have been captured, the CRM system submits the order for processing. Various applications might be interested in that order, including the workflow system which manages the provisioning process and the Customer Data Store application which warehouses all customer records. Though the publish/subscribe pattern, the CRM system simply publishes the "new order" message knowing nothing about the applications that might be interested in it. Interested systems then consume it and do work with the data.
Decoupling in this way yields a powerful layer of extensibility whereby "n" number of applications can be integrated to consume messages with no modification required to those that produce them.
TIBCO provided a messaging layer called Rendezvous that I worked with on several projects back in the early days. Rendezvous was the medium, or "Enterprise Service Bus" (ESB), through which messages were exchanged. It also cost about $100K. Cheaper (i.e. free) alternatives have come about since then, including an implementation of the XMPP protocol called ejabberd. While XMPP is widely thought to be an instant messaging protocol, it's publish/subscribe plugin makes it a good ESB candidate for simple messaging solutions. There are XMPP clients written in .NET, Java, Ruby, etc., so interoperability is maintained.
The new Search solution is trialing XMPP to synchronize ratings data with the search index. Functionally we've proven XMPP does the job as our prototype is using it today, and we're hoping the platform will be robust enough that we can broaden its use to other AMP initiatives which might include real-time registration notification and social message streaming.
I'm excited about the options publish/subscribe gives us architecturally speaking. Hopefully this is the first of many posts heralding this new design paradigm and highlighting its uses.