In the talent war, available data scientists are a scarce resource. There is a growing need for specialists who can explore a large amount of data and extract insights. Evaluating someone in such an interdisciplinary field can be very difficult. After thousands of man-hours, we successfully developed a one-of-a-kind data challenge experience and scoring rubric in JupyterHub. Data-driven companies like Lyft are using Filtered to test and hire their data scientists.
There are many aspects to evaluating a data scientist. It may include data exploration, checking rows, data visualization, using various modules, identifying relevant metrics, drawing conclusions, and more. Filtered runs test cases against a candidate’s modules which score on the accuracy, measuring MAE (Mean Absolute Error) and RMSE (Root Mean Squared Error) of the output versus the correct answer. Candidates are required to submit explanations of their solutions to help you understand their thinking process. You can take the error number, scores, and explanation into consideration when evaluating each candidate.