Perspectives from Indeed’s Data Scientists
I still remember the moment I told my advisor that I was considering leaving academia. The stress. The fear. Saying the words, “I don’t think I want this for myself” out loud. And afterwards, the relief.
At the time, I was juggling work on my dissertation proposal, multiple publications, teaching responsibilities, and a research assistantship. Meanwhile, I had begun researching data science, a field where I could use the skills I had learned in my doctoral program, but in a setting that better suited how I wanted to live and work. So, I made the decision to leave academia, pivot my skillset, and look for data science work in the private sector.
Of course, my experience was not unique. Tenure is often touted as the ultimate form of job security, but the number of tenure track jobs available has declined while the number of graduating Ph.D.’s has increased. Furthermore, pay has not risen with inflation. Many of the data scientists I interviewed who had left academia said that “the road to professorship was long and uncertain” and often full of “soul-crushing” levels of anxiety. By contrast, as companies look to leverage their vast stores of data, job posts on Indeed for data scientist positions have exploded since 2013.
Data science jobs on Indeed.com have quadrupled over the last four years.
We looked at a 7-day rolling mean of all Indeed job posts that featured “data science” or “data scientist” in the title across the world from January 1, 2014 to November 16, 2017. In that time frame, those posts went from slightly more than .02% of all job posts to over .08%. The data was pulled using Indeed’s open source analytics platform Imhotep.
Due to the sequestered nature of the academy, those who might want to leave academia often don’t know what things are like in “the real world.” Since I left the academy two years ago, some of the more common questions I receive are: “Why did you leave academia?” “What should I do to make myself more hireable?” And the most existential question of all:
“Will everything be OK if I leave?”
To answer these questions, I enlisted the help of eleven other Data Scientists and Product Scientists here at Indeed who left academia. For simplicity, I refer to them as “data scientists” throughout this post.
The data scientists I interviewed come from a variety of backgrounds, some from Liberal Arts and others from STEM. For some, Indeed was their first job after leaving academia, while others have been in industry for nearly 10 years. They left at various points in their academic careers, some while still ABD (“all but dissertation”), others while serving as assistant professors. Some went on the academic job market first, others “didn’t even bother,” still others never intended to. Some of our data scientists discussed that a major life event, like getting married or having a child, spurred them to rethink staying in academia.
It’s my hope that this blog post offers some guidance and reassurance for academics considering making the leap to industry.
Why move to industry?
Simply put, moving to industry allows for more freedom for how you spend your time, energy, and money; where you’re able to live; and what kinds of statistical methods you can use.
Suddenly, you have more autonomy to decide where to live or work. For one data scientist, moving to industry after a series of post-doctoral fellowships allowed for “a more predictable career path where there were plenty of job opportunities.”
One of the biggest adjustments former academics made when moving to industry was how much more free time they had and how much more relaxed they felt. Our data scientists described “getting weekends back” and being able to “go home at 5pm and not needing to do any more work.” Some of the former academics discovered new hobbies with their newfound time and money, like cooking, cycling, photography, and sports analytics.
Working in the private sector also provides another form of security: a better salary. One data scientist commented that her “pay doubled and the amount of work required halved” and another about “how good it feels to not be struggling financially!” The average data scientist salary on Indeed is well above the United States median income and the median salary for academics in postdocs or professor positions. I vividly remember the first time I bought a new pair of shoes on my new salary and realized that I did not have to worry about the effects on my budget.
Apart from the positive effects on their personal lives, data scientists who left academia also noted benefits for their work. Some noted that they were “afforded more methodological freedom outside of the academy,” due to the amount of data readily available and fewer concerns about whether their work would be publishable. Nearly all of the data scientists I spoke with mentioned how much they enjoyed “how much impact you can have, and how quickly.”
Speed and collaboration in the private sector
Non-academic jobs do have some notable differences that can surprise newcomers, particularly around speed and collaboration.
Nearly all of those I interviewed commented on the shorter project cycles and faster pace in the private sector, adding that “there is a constant focus on moving fast and iterating.” The tendency for academics to “spend forever on a project until it is exactly right is not a habit that will transfer well.” Another data scientist noted that “in academia you stay with one topic or research area for many years. In industry you need to be OK with working on something new every quarter or so.”
Perhaps the biggest difference is that you no longer work in isolation. “Academia was very isolating and highly values individual contributions you could claim as entirely your own,” discussed one data scientist. “However, in industry the most successful folks are the ones that collaborate well.”
The opportunity to collaborate can be especially meaningful for academics transitioning to industry. “If you’re struggling on something,” one data scientist mentioned, “you don’t have to do it alone.” Teams also expect you to check in with them periodically and not “disappear to work on a project for weeks or months at a time.” While all of the data scientists I spoke with had experience coding for their academic research, it was usually done “in an environment where the only person I would affect was myself. Having to share a codebase with 5 other people and not trip all over each other was a skill I had to learn quickly.”
Transferable skills between research and industry
It’s sometimes hard to remember in the hustle of research, but academics have a multitude of transferable skills beyond expertise in their field. In a way, “academics are trained to be startups with a single employee. They need to gather information, find funding and support, allocate scarce resources, and acquire the technical and non-technical skills needed to get off the ground,” one data scientist noted.
For some, these transferable skills include coding, statistics, and model building. For others, skills can include a variety of non-technical abilities that are so familiar to academics that they overlook their value to the private sector. As one example, because of all their independent work, academics usually pick up valuable time and project management skills. The importance of being able to do independent research and teach oneself cannot be overstated. “Lots of data science questions are complicated and no one has all the required skills,” said one data scientist. “It’s therefore super important to teach yourself.”
Academics also learn valuable communication skills. One data scientist said, “I had so much practice giving talks to an audience of experts trying to pick holes in my argument, that presenting results to clients was a breeze.” Context switching between teaching an intro class and presenting to fellow experts also “provides plenty of practice in pitching your presentation to the appropriate audience.” After writing theses, dissertations, books, and publications, it’s a lot less daunting to write documentation, project updates, and blog posts.
Finally, the value of critical thinking is immense. Academics are trained to be skeptical and to take on “big, complex, and messy problems.” They’re well-equipped to “build a project in an iron-clad way and understand all of the caveats of [their] work.”
How to get hired in industry
The data scientists I spoke with took several different approaches to redefining themselves and their skillset. These included reading books, attending onsite and online courses, working on internships over the summer, and meeting with an on-campus career counselor. Many universities offer these resources for a small fee or even for free.
Several former academics suggested starting a personal blog to write about side projects while learning data science skills. Organizations such as DataKind and Data for Democracy offer volunteer opportunities where budding data scientists can hone their skills. A couple of them “asked around for data projects to do pro bono so I could practice my skills and have some projects to share in my portfolio on my website.” At least two of the eleven data scientists I talked to attributed getting their first industry job in part to these blogs.
Some data scientists who went into industry “deliberately switched” from languages like Matlab or Stata to languages like Python and R for their research projects while still in academia. Others tried to “guide [their] academic work toward slightly more practical technological aims.” One of the most common data skills not usually learned in academia is SQL. Luckily, SQL is relatively easy to learn with abundant resources in print and online. Our data scientists recommend SQL in 10 Minutes by Ben Forta and Learning SQL by Alan Beaulieu.
Finding other academics interested in leaving academia can also be helpful. Some data scientists I spoke with formed study groups, and even practiced mock whiteboarding interview questions together. Others attended local meetups or conducted informational interviews to learn more about the field of data science.
Indeed is looking for data scientists!
Indeed’s Data and Product Science teams look for people who know statistics and how to program. Our Data Science team looks for “full-stack” engineers with strong machine learning backgrounds. Our Product Science team emphasizes prioritizing and solving business problems. Equally important to both teams are soft skills: curiosity; a desire to grow, learn, and improve; being humble and self-aware of one’s limitations; and “a true passion for helping people get jobs.”
Interested in applying? Indeed is hiring Data Scientists and Product Scientists at all levels of seniority and experience, and we love former academics! Find all of our open job listings at this link.
Methodology
This was a qualitative and self-reported study with an admittedly small sample size. We’d love to hear if this was representative (or not) of your experiences. Did you leave academia to pursue a data science role? How has your experience been similar or different? Leave us a comment below!
Special thanks to Christo du Plessis, John Jardel, Evan Koh, Eric Lawrence, Chris Lindner, Donal McMahon, Elias Ponvert, Ke Sang, Annette Taberner-Miller and Wenzhao Xu for taking the time to share their thoughts about transitioning from academia to industry. Trey Causey gave fabulous feedback on an early draft of this post, and James Beach and Leah Pasch helped edit the final draft. Thanks to all academic advisors out there who are understanding and supportive when their students leave academia (thank you, Pam!).
Transitioning from Academia to Industry cross-posted on Medium.