Data engineers design and build systems that store and analyze data. They are crucial to managing the large quantities of data that aid in building algorithms, machine learning, and much more. A fundamental reality of the data engineer job market is that demand far exceeds supply. There are too many companies looking for too few data engineers. Here we’ll look at how to hire a data engineer with the skills and qualifications that fit your company's needs so you onboard the best talent. 

How to Attract Talented Data Engineers 

With such a limited talent pool, it’s vital to streamline your hiring process and show potential candidates your value as an employer. A recent survey of data engineers by technology recruiting firm Stott and May found that job seekers want the following: 

• Clearly defined roles. Close to three-quarters of those surveyed said that a clearly defined job description was the most critical factor in whether they applied for a job. LinkedIn recommends that a data engineer job description outline the role’s responsibilities, qualifications, and objectives. Additionally, it should include a description of what it is like to work for your company and how the candidate can impact the culture.

• The right technology stack. Around half of the respondents said that the company’s technology stack was the most important factor in accepting a job. If your technology stack does not match the skill set of the candidates, they are going to move on. 

• Competitive salary and benefits. Forty-two percent of respondents said a below-market salary and benefits would prompt them to accept a job at another company. So you must survey the market and offer a competitive package. According to Glassdoor, the average salary for a data engineer in the United States is $100,497 per year, with additional pay totaling $14,366 per year. So, make sure your offer is competitive.

• Streamlined interviewing process. The survey found that having a streamlined interview process that assesses the right skills for the job is essential for both the company and the candidate. “Interviewing should be about making the candidate feel at ease and creating an environment where they can show themselves at their best,” observes Dani Solà Lagares, senior vice president of data and analytics and former director of data at Simply Business.

Update Your Technical Interview Process

Unfortunately, the standard interviewing process for technical positions like data engineering does not make the candidate “feel at ease” or create an “environment where they can show themselves at their best.”

According to a North Carolina State University and Microsoft study, the current interviewing process for technical positions is significantly flawed. Researchers conducted interviews with 48 students serving as job candidates. Half were given a conventional technical interview, with an interviewer observing. The rest were directed to solve their problem on a whiteboard in a private room.

The researchers demonstrated that the public interviews did not select the most skilled candidates but rather those who perform well under pressure. The public process also tended to penalize women. All the female candidates failed the public interview, but all passed the private interview.   

"Our study suggests that many well-qualified job candidates are being eliminated because they're not used to working on a whiteboard in front of an audience,” says Chris Parnin, an assistant professor of computer science at NC State. One of the best ways to update your hiring process is to use a hiring platform. The right platform will greatly expand your talent pool, vet candidates, and reduce long-term costs.

How to Hire a Data Engineer: Use the Right Platform

The key to an interviewing process that puts the candidate at ease and finds the best person for the job is using the right platform. 

Filtered’s AI-based platform can test candidates for a wide range of skills using a variety of formats, such as technical assessments, take-home testing, and live interviews. 

For data engineers, we create a live, standardized Jupyter environment for each candidate, enabling them to work on actual data projects. 

We can test for such vital skills as model selection, parameter tuning, data visualization, metric selection, data cleaning, feature engineering, and knowledge of Python, C, R, and other programming languages.

Our results speak for themselves. We slash the cost of a new hire from $40,000 down to $400 and cut the time to fill a vacant data scientist position from 35 days to 5 days. We increased the interview to hire from 4% to 60% and cut the interview time per candidate from 2 hours to 27 minutes. We also weed out fraudulent applicants, slashing undetected fraud from 29% to 0.001%. 

Filtered also allows you to see what your candidates search for on Google during the test, track which tabs they open, record why they copy-paste and see which packages they install. Our customer support teams are staffed by actual engineers and data scientists who have done their share of interviewing for jobs. We can answer your questions and those of candidates during their testing. 

Filtered is a leader in skills-based, data-driven recruiting technology. Our end-to-end technical hiring platform enables you to spend time reviewing only the most qualified candidates, putting skills and aptitude at the forefront of your decisions. We’ll help you automate hiring while also applying objective, data-driven techniques to consistently and confidently select the right candidates. To get started, contact our team today or register for a FREE demo. 

‍