Active Product Development : Tags : architecture

Organizer Technology Solutions:

Post an Activity | Advertise your event | Account Access

Active.com

San Diego, CA

Active Product Development

2 Posts tagged with the architecture tag

Checklist for Deploying New Apps on Amazon

Posted by JeremyGThomas Aug 19, 2010

Within M+M we've deployed myriad products to Amazon Web Services, including photos.active.com, advantage.active.com, ihoops.com, mobileapps.active.com, widgets.active.com, realtime.active.com, search.eteamz.com and labs.active.com. We've found that, with each product, we have to run through the same checklist of items to ensure continual up-time. Some of these items are related to application architecture, while others are related to infrastructure:

Cloud Checklist

Caching - Serving pre-compiled content is always the best way to reduce I/O and have a fast website. Memcached and Apache's mod_cache are frameworks we commonly use.
Load Balance Across Availability Zones -- AWS has a 99.95% uptime SLA, but only if applications are deployed across multiple availability zones. As such, nodes in a pool should run in two or more availability zones. And, as if it needs to be said, applications should be load balanced.
Data Backup - The cloud doesn't magically solve data retention issues (but it does help). Amazon's RDS offering is a great option when not wanting to deal with setting up backup policies. Otherwise, these must be put in place just like they would in any given data center.
VIP Monitoring - Like most companies with products in the cloud, we run some of our products in a proprietary data center, and others on Amazon. But our monitoring system acts as a central repository reporting status on all systems. As such, monitoring must be setup on VIPs (i.e. www.ihoops.com) and nodes, with alerting configured should anything act up. Node monitoring is enabled by allowing traffic through from the source ip address of the monitoring system through Amazon's security group configuration.
CPU and Memory Monitoring - CPU and memory utlization should also be monitored. UDP port 161 should be opened to the monitoring system to record SNMP traps.
Human Support Process - When something goes wrong, somebody's gotta get paged.

1,297 Views 0 Comments Permalink Tags: architecture, cloud_computing, amazon, infrastructure

Active Realtime

Posted by BrianActive Mar 10, 2010

Active Realtime is the newest addition to our product incubator labs.active.com. Every day thousands of people are searching and registering for things to do at Active. The idea behind Active Realtime is to provide a window in to actual user activity, giving a sense of whats happening right now at Active. We also aggregate popular activities on a local level, which reveals favored events and activities in a region. We also have the opportunity to look at trending topics over time, which will perhaps yield new insight into what is keeping us Active.

Presently data for Active Realtime is provided by our newly launched search engine . In the future we hope to include realtime data from other parts of Active, such as event registration openings or community posts. Keeping this in mind, we designed a technical infrastructure for Active Realtime that would allow us to add additional sources of data in the future, as well as allow other applications at Active tap into the information.

To accomplish this, we used XMPP Publish Subscribe, an addition to the venerable IM protocol. Using XMPP PubSub, we are able to keep Active.com Search and Active Realtime loosely coupled, so each service can be independently maintained and scaled. Additionally, we developed a Search Logger, which acts independently of the Search site. The Search Logger acts as the Publisher, and pushes search data over the XMPP bus. Any interested part of the Active infrastructure may subscribe to receive these updates, and the Realtime application is just that. The Realtime architecture includes a Ruby daemon that acts as Subscriber, registering an interest for search activity and receiving data accordingly. This data is recorded into the Realtime database, which then can be displayed on the Realtime site. A diagram showing the architecture is included to the side.

The payoff for investing the time to create this architecture, rather than just querying the search infrastructure directly, is twofold. First, it provides a simple and scalable way for future datasource additions to be made to the Realtime application. Secondly, it de-couples realtime with each of the data publishers, allowing all systems to function and scale independently, without having an impact on one another.

I hope you enjoy this window into Active. The data on Active Realtime is generated by Active users, so perhaps another like minded person will help you to find a great local activity to participate in.