Efficient Query String Parsing with util-urlparsing

We’re excited to announce the open source release of util-urlparsing, a Java library we created to parse URL query strings without unnecessary intermediate object creation. It also includes number parsing methods in ParseUtils that are faster than Java’s equivalent methods like Integer.parseInt and Float.parseFloat.

Java versions 1.6 and lower have a significant flaw that leads to inefficient memory usage when using the String.substring method. When processing data from our log repository, we need to extract small substrings from much larger strings containing event data key/value pairs. The primary class in util-urlparsing, QueryStringParser, was written to efficiently parse this data without generating any intermediate string objects. It does this via a callback mechanism that lets you only parse the keys you are interested in from the larger query string.

Our query parsing benchmark shows nearly 4X speedup over a naive Java implementation using String.split under significant heap space constraints. It can parse a million key-value pairs in under 3 seconds given a max heap of only 64MB. Our number parsing benchmark shows over 2X speedup compared to equivalent methods like Integer.parseInt and Float.parseFloat.

util-urlparsing is available for download on GitHub (github.com/indeedeng/util/tree/master/urlparsing) or maven.org (search.maven.org/#browse%7C-525259937), and we have documentation which includes usage examples to help you get started. If you have any questions, check out the Q&A forum for our open-source Java utilities.

If you’re interested in learning more about Indeed’s log repository, check out the video and slides of our January 26th talk “Logrepo: Enabling Data-Driven Decisions.”

5 Tips For a Best. Hackathon. Ever.

It’s been a week since the 5th Semi-Annual Indeed Music, Food and Art Festival Hackathon and I’ve fully caught up on my sleep. How did it go?

Screenshot of Indeed's internal social media feed in which an Indeedian states "Best. Hackathon. Ever." with 30 likes and one comment.

Screenshot from IndeedFeed: “Best. Hackathon. Ever.”

Here are some of the lessons we’ve learned for putting on hackathons.

Tip 1: Pick a theme, but have room for interpretation.

Although it may seem liberating to have a no-theme hackathon, a few boundaries actually help people to come up with ideas – creativity loves constraints. Our approach is to have a theme, but allow people to interpret it as they like – or even ignore it if they have some idea they’re passionate about. Past Indeed hackathon themes have included the usual suspects for a consumer-focused Internet company: mobile, social, data, etc. For this hackathon, we did something a little different. At Indeed, our mission is to help people get jobs. Our theme this time was “Help People Who Help People Get Jobs,” which encouraged the hackers at our engineering offices around the world to think of their colleagues in other departments (sales, marketing, client services, etc) and how they could be helped.


To: All Sites
From: Indeed Hackathon Committee
Subject: Hackathon Theme

Who helps people get jobs? WE DO! All of us – from Ops to Dev to HR to Marketing to International to Sales and Client Support. We’re pretty good at it. Jobseekers love Indeed, and we love them. But for the first Hackathon of 2014 lets give some attention to another special group of people: Indeeders.

That’s right. It’s time to help people who help people get jobs!

  • Does your buddy in International look stressed out? Write some code to generate his powerpoints!
  • Think our software should sell itself?  Make it so with Twilio!
  • Know a CS rep who needlessly suffers because he or she can’t automate away some daily task?  Prevent hardship with Python.
  • Feeling particularly subversive? Build a web app to replace your (or someone else’s) manager.

To solicit ideas from the whole company we had an idea wall on IndeedFeed (our in-house social network) where everyone could vote. Then we had an idea happy hour and pitch-fest where folks from all parts of the business could practice their elevator pitch and entice hackers to work on their ideas.

Tip 2: Provide plentiful good food.

Great hackers deserve great food (better than pizza and soda). And healthy food! That’s why we kicked off mid-day Thursday with a chocolate fountain at lunch. Oops:

  s/healthy/delicious/g

For dinner, Korean-Mexican fusion food truck Chi’Lantro set up a taco buffet. Foodies in Austin love Chi’Lantro and so did the participants. But it wouldn’t be Austin without some high quality barbecue! FYI: in Texas, barbecue (“BBQ”) does not mean meat cooked over an open flame on a grill, it means smoked for many hours at low temperature in a smoker. Fortunately for us, we have a couple of amateur pitmasters on staff. They brought a smoker and treated us to a whole side of beef brisket at midnight and another mid-day Friday. Delicious!

People filling plates with various snacks, including chocolate from a chocolate fountain

An open smoker grill with a slab of brisket cooking on it

Photos: The chocolate fountain and brisket (not at the same time). Yum!

Tip 3: Get quality T-Shirts.

Make t-shirts that people want to wear. Besides great artwork, this means comfortable good-looking t-shirts in a range of women’s and men’s sizes. We’ve tried a number of suppliers, and our current favorite is American Apparel tri-blend.

T-Shirt Design with cartoon people grouped together to form the word "Hack"

Many thanks to Hans Krebs, one of our visual designers, who made the artwork for this hackathon’s t-shirt. The design depicts Indeed employees, wearing our trademark blue “I Help People Get Jobs” t-shirts, arranged to spell hack. But it’s also a game: find the 10 unique representations of people (hint: one has a mustache, one is wearing a skirt, etc).

Tip 4: Involve everyone, not just coders.

Pieces of artwork Indeedians created, including one that says "I love Indeed"Not everyone can code, but everyone is creative and can make stuff. Our hackathons are more than just code-a-thons. They’re a 24-hour creative-expression-fest. For example, at previous hackathons we’ve painted artwork, including a giant system diagram, on the walls. At this hackathon we had non-coders learning to code (thanks codecademy!), we had a product manager direct a movie, and folks made some great oil-on-canvas paintings. Next time we’re thinking of expanding the art-side of the hackathon even further.

A cartoon character DJ with turntablesAs for music, if you were saddened by the shutdown of Turntable.fm rejoice! PlugDJ is a great replacement – a real-time shared music-playing experience. Set up a PlugDJ community room for your event and send your hackathon participants the link. For our hackathon everyone moves into one big workspace and we set up loudspeakers hooked up to our PlugDJ room. This way everyone gets to take turns choosing the music, and people can vote it up or down.

Tip 5: Finish strong.

After a hard 24 hours hacking, everyone deserves the opportunity to show off their work. Teams need to be encouraged to practice their presentation ahead of time. We limited our presentations to 2 minutes, which is very short when you’re trying to demo something and explain it. To keep the 30+ presentations moving, we set up two presentation stations, each with its own laptop and projector. While one team was presenting, the next team was setting up their demo. This made for fast transitions, which everyone appreciated after being up all night.

The best part of this hackathon? 15 projects are going into production to help people who help people get jobs! These include a new internal Question & Answer platform, a tool for managing translations, several new reports for various departments, improvements to IndeedFeed, and much more.

If you have other suggestions for a great hackathon, share your thoughts with us, we’re @IndeedEng.

@IndeedEng 2013 Year In Review

Indeed engineers help people get jobs. We take that mission very seriously, and we love that we get to work on interesting problems in an exciting, dynamic environment that encourages end-to-end ownership, engineering velocity, and data-driven decision making.

Every member of our engineering organization contributes to this unique culture. We’re not complacent about it, either. We treat our culture like our products, as an evolving set of values and processes that require constant iterative improvement.

In the past, we didn’t share very much about what we’d done at Indeed with the software development community at large. We were focused on our mission and constantly improving how we help people get jobs.

We began to emerge from that cocoon last year, when we launched this blog. Then, in February 2013, we began presenting @IndeedEng, a series of monthly talks in our Austin headquarters. The slides and video for these talks have been posted on our blog and are now available on our new site, engineering.indeedblog.com/talks.

We presented nine @IndeedEng talks in 2013. We poured lots of effort into these talks, making sure that they clearly presented interesting technology ideas and best practices for developing great software. In our early October talk, we announced the open source release of Proctor, our A/B testing framework.

The response to these talks has been encouraging and rewarding, with over 600 people attending the series in 2013. We have enjoyed testing the capacity of our “auditorium” (it’s actually our cafeteria).

We’d like to take this opportunity to thank every great software developer out there who has inspired us to give back to the community, to thank those at Indeed who have helped make these efforts a reality, and finally to thank all of you who have read our blog, watched an @IndeedEng talk, or used our open source projects.

We look forward to continuing our conversation with you in 2014!