Blog Posts From Active Product Development Tagged With cloud_computing

Active Realtime

BrianActive — Thu, 11 Mar 2010 00:27:02 GMT

Active Realtime is the newest addition to our product incubator labs.active.com. Every day thousands of people are searching and registering for things to do at Active. The idea behind Active Realtime is to provide a window in to actual user activity, giving a sense of whats happening right now at Active. We also aggregate popular activities on a local level, which reveals favored events and activities in a region. We also have the opportunity to look at trending topics over time, which will perhaps yield new insight into what is keeping us Active.

Presently data for Active Realtime is provided by our newly launched search engine . In the future we hope to include realtime data from other parts of Active, such as event registration openings or community posts. Keeping this in mind, we designed a technical infrastructure for Active Realtime that would allow us to add additional sources of data in the future, as well as allow other applications at Active tap into the information.

To accomplish this, we used XMPP Publish Subscribe, an addition to the venerable IM protocol. Using XMPP PubSub, we are able to keep Active.com Search and Active Realtime loosely coupled, so each service can be independently maintained and scaled. Additionally, we developed a Search Logger, which acts independently of the Search site. The Search Logger acts as the Publisher, and pushes search data over the XMPP bus. Any interested part of the Active infrastructure may subscribe to receive these updates, and the Realtime application is just that. The Realtime architecture includes a Ruby daemon that acts as Subscriber, registering an interest for search activity and receiving data accordingly. This data is recorded into the Realtime database, which then can be displayed on the Realtime site. A diagram showing the architecture is included to the side.

The payoff for investing the time to create this architecture, rather than just querying the search infrastructure directly, is twofold. First, it provides a simple and scalable way for future datasource additions to be made to the Realtime application. Secondly, it de-couples realtime with each of the data publishers, allowing all systems to function and scale independently, without having an impact on one another.

I hope you enjoy this window into Active. The data on Active Realtime is generated by Active users, so perhaps another like minded person will help you to find a great local activity to participate in.

Active Widgets

Cannikin — Tue, 02 Mar 2010 00:11:07 GMT

Hi, I'm Rob and I'm a Software Architect on the Light Engineering Group here at Active. LEG, as we call it, was modeled after the LED group at LinkedIn. LinkedIn was looking for a way to rapidly prototype new technologies and architectures with Ruby on Rails. Their first product was Bumper Sticker which at one point was the mostly highly trafficed Facebook application in existence! You can find many of the fruits of LEG's labor at http://labs.active.com

This morning we officially launched our new Active Widgets site at http://widgets.active.com This was a quick two week project to create an embeddable window into Active.com's thousands of registerable events. The idea being that the blog you write about running/cycling/baseball can show a list of upcoming marathons/track events/Little League games in your area. The widget is customizable as far as what activities it displays, where it displays them, whether or not the user can change those settings, as well as the size and color of the widget itself. Once your done just click a button and you're given a couple of simple Javascript tags to drop onto your site wherever you want the widget to appear. And since the widget is written out onto your page like any other HTML, you can further customize it using your own CSS.

The Technical Stuff

This customization site (known as The Configurator) is a Sinatra application written in Ruby. There is no database—the data store is a single YAML file which defines a widget and what about it is customizable. The widget itself is written in Javascript and after minification with the YUI Compressor, the whole thing is 14K (for comparison, the Google homepage is 47K).

Since we planned on having the widget be embeddable on any website we couldn't use Ajax to pull in the event listings. Ajax is subject to strict cross-domain security policies set by your browser—you can only access servers in the same domain that the page itself was served from. This is quite a problem since search results are coming from http://search.active.com The workaround for this issue is to make server calls with <script> tags and then wrap the resulting code in a function call known as a callback. When the <script> tag is added to the page the browser immediately executes it as valid Javascript. Since this Javascript contains a function call (the callback) it looks a function you've already defined with the same name and calls it, passing in the returned data.

The server itself is an Amazon EC2 instance running in their cloud infrastructure. This allows us to quickly react to any scaling needs by simply spinning up a new instance of the same server and deploying the code to it. Nearly half our division's products are now running on EC2 and we're moving more there every day. We've found it be an fast, secure and cost effective way to get our applications live.

Stay tuned for more exciting projects from the LEG team!

Experimenting with the Cloud

JeremyGThomas — Mon, 19 Jan 2009 21:17:26 GMT

We recently rolled out a private Beta for our new Search solution. Not only is it a good functional test for us, we're also experimenting with Amazon EC2 and S3 for hosting and data storage. And from what I've seen so far I'm impressed with Amazon and the approach they've taken to cloud computing.

Amazon has made it dead easy to provision new servers. They've created a collection of webservices to integrate to for starting and stopping instances. It seemed odd to me, at first, that I'd have to manage my infrastructure through SOAP calls. But client-side tools like Elasticfox and now Amazon's own AWS Console make it easy to manage. There's also a good selection of Windows and various Linux flavors to choose from when setting up servers (we're running mostly Ubuntu 8.1.0 in Labs).

We wanted to make our Labs infrastructure extensible so we can new pilot applications quickly, regardless of platform. To do this we're setting an Apache server on an EC2 Ubuntu host that routes http://labs.active.com to the appropriate app. Requests to the base URL will be sent to a CMS/wiki that describes the various things we're working on. Search, which currently resides in the root, will soon be available at http://labs.active.com/search. Any future pilot we rollout will be available, then, at http://labs.active.com/{pilot-name}.

We're also setting up "global" memcached and MySQL servers, so that any application deployed to labs might benefit from these services (we love memcached by the way).

Each pilot application will be responsible for load balancing its requests. The Search application uses HA proxy running on a dedicated Ubuntu server to route requests to a collection of Apache instances running mod_rails distributed across Ubuntu "workers" (we can scale horizontally here, pending load, by adding more workers).

So far (and it's been about 1.5 months) we haven't had any Amazon-caused downtime. Personally, I think this is a game changer.