OpenX Ad Serving Platform at Mail.com
February 21, 2014
OpenX Ad Serving Platform at Mail.com
Sr. Systems Engineer
Tango MeMedia Production, 51-200 employees
Score 9 out of 10
- OpenX Source
At Mail.com Media Corporation (currently Penske Media Corporation http://www.pmc.com ) OpenX was used to power the Mail.com portal page which served millions of unique users per day and more than half a billion ads per month. The Mail.com portal page delivered unique content and was the focal point in which users would pass as they logged into their webmail each day. OpenX was chosen to supplement then reduce the reliance of third party ad networks which were already serving this strategic space in the Mail.com network. OpenX allowed Mail.com to reduce the dependency and heavy costs associated with third party ad networks. In addition to being less dependent on third party ad networks and reducing costs, OpenX helped empower the ad sales and ad operations teams by giving them more control of their own network inventory. OpenX also served ads network wide in each brand and line of business in the Mail.com portfolio.
- OpenX is a system that was designed to scale as is evident in some key design decisions found throughout the platform. The multi-server setup that was chosen at Mail.com allowed for a distributed server architecture which separated the front end web delivery nodes from the backend MySQL Database master which replicated data back to each front end delivery node mysql slave. This eased the ability to horizontally scale as needed due to the ingenious separation of the ad impressions data tables on the delivery nodes which were being collected locally on each ad impression and then processed periodically back to to the master database which replicated the aggregated statistics back to the delivery nodes. The ability to load balance across the front end web delivery nodes, add caching at many different layers, utilize a CDN for the static ad images, implement PHP accelerators, and hit memcached instead of the MySQL master made the OpenX platform a service that was very resilient to failures.
- The ability to optimize the platform is also something that OpenX does very well. This is evident in many of the config key parameters available. In addition to OpenX specific tunable's, optimizations can be made at many different levels in which the system sits. These include hardware and operating system level optimizations, tcp and networking stack optimizations, web server/php-cgi configurations, and database (MySQL InnoDB) tunable's. There are many different optimization knobs that can be tuned to help scale the application for the best performance possible
- The documentation was simple, to the point, and well written which led to an easy initial implementation and roll out of a multi-server setup. The installation and upgrade procedures were straight forward even for the complexities of the multi-server setup. I also found the OpenX team to be accessible and even got a chance to meet some of the team at their Up Close and Personal events at Cal Tech in Pasadena. Core developers were on hand to answer technical questions and also made themselves available to us via email and irq channels.
- The open nature of the OpenX platform allows for flexibility in the choice of web serving platform to use (Apache, nginx, Lighttpd) and also the database management systems (MySQL, PostGreSQL)
- Although I do believe that OpenX was designed to scale, many of the platform's target users are running single server implementations which probably led to the silly choice of MyISAM as the default database storage engine for the MySQL tables. In multi-server setups or any installation with a significant amount of load these should be altered and optimized for InnoDB. The default storage may have changed since my last roll out of this platform but I do remember the tables needing altering before putting any significant amount of load on them.
- In a multi-server setup sometimes problems can arise from the statistics collector and processor on the front end delivery nodes that process the ad impression data and aggregate the resulting statistics back to the master database resulting in several scenarios in which the master database can be out of sync with information collected on the delivery nodes and manual intervention is needed. However, most of these problems can be monitored and engineered for.
- The focus of OpenX on enterprise customers serving hundreds of millions of ads as opposed to the long tail of smaller publishers who simply can't afford to utilize the enterprise hosted solution and must utilize the OnRamp solution which has suffered from performance problems and might not be a viable option for many medium sized publishers
- Reduced dependency on third party ad networks reducing costs
- Increased ad sales and ad operations team efficiency
- Ability to tailor ads for specific brands using many advanced features such as demographics and geo targeting.
The ad sales and ad operations team had been utilizing DoubleClick/DART previous to the migration to OpenX. One of the driving factors for the migration to OpenX was to lower costs associated with utilizing a third party such as the DoubleClick/DART system. The ease of use for the ad ops team, advanced targeting features, and seamless integration into the many brands of the Mail.com portfolio were also other factors involved in why OpenX was selected.
I would renew use of OpenX as it is a solid ad serving platform that is easy to use, install, and maintain with some great advanced features that give more control to publishers ad ops teams large or small while reducing the costs associated with managing, serving, and delivering ads from third party vendors.