How to Get a Job 35 Million Times a Day Using RabbitMQ
This talk was held on Wednesday, March 27, 2013 at 7:00pm
The goal of Indeed’s aggregation engine is to find and retrieve every job in the world, as quickly and accurately as possible. As we described in our previous tech talk, we strive to build products that are simple, fast, comprehensive, and relevant. The world’s most comprehensive job search site is fueled by the more than 35 million job postings we process every day, which we deliver to jobseekers within minutes of discovery.
Our original aggregation architecture was implemented using standard patterns. Our growth required levels of scalability, performance, and resilience this architecture simply could not handle. In a case study of scaling for the web, we discussed how we tackled this problem. We covered the issues we saw with our original architecture, how we analyzed our options to guide a solution, how we used RabbitMQ as a key component in the new architecture, and benchmarks to evaluate how successful we were.
Speaker
Cameron Davison is a software engineer and a graduate of UT Austin.