We are excited to highlight the open-source availability of util-mmap, a memory mapping library for Java. It provides an efficient mechanism for accessing large files. Our analytics platform Imhotep (released last year) uses it for managing data access. Why use memory mapping? Our backend services handle large data sets, like LSM trees and Lucene indexes. […]
All posts categorized in: Big Data
Serving over 1 billion documents per day with Docstore v2
[Editor’s note: This post is the second installment of a two-part piece accompanying our first @IndeedEng talk.] The number of job searches on Indeed grew at an extremely rapid rate during our first 6 years. We made multiple improvements to our document serving architecture to keep pace with that growing load. A core focus at […]
From 1 to 1 Billion: Evolution of a Document Serving System
[Editor’s note: This post is part 1 of a two-part companion piece to our first @IndeedEng talk. Slides and video are available.] Indeed.com launched in November of 2004 in the US. Today, Indeed is in more than 50 countries and has 100 million unique visitors performing over 3 billion job searches each month. We have […]