We're well underway developing the new search.active.com, and I thought I'd take a moment to discuss a realt-time, publish/subscribe messaging layer we're inserting into the Search architecture. In the beginning of my career I worked building distributed OSS/BSS applications integrating various line of business systems with products from a middleware vendor called TIBCO. During this time I was introduced to the concept of "[publish/subscribe|http://en.wikipedia.org/wiki/Publish/subscribe]", whereby an application sends one message that may be consumed by many disparate applications. For example, when a customer calls into a call center to order broadband internet, the customer service rep enters order details into a CRM system. When the order details have been captured, the CRM system submits the order for processing. Various applications might be interested in that order, including the workflow system which manages the provisioning process and the Customer Data Store application which warehouses all customer records. Though the publish/subscribe pattern, the CRM system simply publishes the "new order" message knowing nothing about the applications that might be interested in it. Interested systems then consume it and do work with the data.
Decoupling in this way yields a powerful layer of extensibility whereby "n" number of applications can be integrated to consume messages with no modification required to those that produce them.
TIBCO provided a messaging layer called Rendezvous that I worked with on several projects back in the early days. Rendezvous was the medium, or "Enterprise Service Bus" (ESB), through which messages were exchanged. It also cost about $100K. Cheaper (i.e. free) alternatives have come about since then, including an implementation of the XMPP protocol called ejabberd. While XMPP is widely thought to be an instant messaging protocol, it's publish/subscribe plugin makes it a good ESB candidate for simple messaging solutions. There are XMPP clients written in .NET, Java, Ruby, etc., so interoperability is maintained.
The new Search solution is trialing XMPP to synchronize ratings data with the search index. Functionally we've proven XMPP does the job as our prototype is using it today, and we're hoping the platform will be robust enough that we can broaden its use to other AMP initiatives which might include real-time registration notification and social message streaming.
I'm excited about the options publish/subscribe gives us architecturally speaking. Hopefully this is the first of many posts heralding this new design paradigm and highlighting its uses.
The update to active.com last week brought with it a significant improvement in load time and page size. We met last month to determine how we could give the site a bit more zip and came up with a solid list of about 8 items. One of those items, removal of VIEWSTATE, was included in our latest update. VIEWSTATE is a variable used in ASP.NET to make HTTP, a stateless protocol, act like a stateful one. Most of the pages on active.com, however, dont require state, making VIEWSTATE extraneous. And on average VIEWSTATE added an additional 50 to 150 KB to page sizes to the site.
In our latest release we turned VIEWSTATE off. Data shows us that the site is about 23% faster and, with an average page size reduction of 27%, well lower consumed bandwidth out of our production data center by about 500 to 800 GB/month.