Sign in Subscribe
Tech Career

Quantitative Tech Interview Preparation Guide

A short list of interview preparation resources for Data Scientists, Machine Learning Engineers, Machine Learning Scientists, Quant Developers and Quant Researchers.

Ran Ding

• 5 min read
Quantitative Tech Interview Preparation Guide
See more posts here about tech careers
Tech Career - Ran Ding
Ran Ding

This is a short list of interview preparation resources for Data Scientists, Machine Learning Engineers, Machine Learning Scientists, Quant Developers and Quant Researchers.

At a high-level we can divide things into 3 main areas:

  1. Machine Learning
  2. Coding
  3. Math (calculus, linear algebra, probability, etc)

Depending on the type of roles, the emphasis can be quite different. For example, AI/ML interviews might go deeper into the latest deep learning models, while quant interviews might cast a wide net on various kinds of math puzzles. Interviews for research-oriented roles might be lighter on coding problems or at least emphasize on algorithms instead of software designs or tooling.

List of resources

A minimalist list of the best/most practical resources:

Machine Learning:

  • Course on classic ML: Andrew Ng's CS229 (there are several different versions, the Cousera one is easily accessible. There is also an older version recorded at Stanford)
  • Book on classic ML: Alpaydin's Intro to ML link
  • Course with a deep learning focus: CS231 from Stanford, lectures available on Youtube.
If you are just breaking into the field I think the above are enough, stop there and move on to other areas of preparation. Here are a few very optional items, mostly on deep learning, in case you have more time:

Coding:

Math:

  • Calculus and Linear Algebra: undergrad class would be the best, refresher notes from CS229 link
  • Probability: Harvard Stats110 link; book from the same professor.
  • Statistics: Shaum's Outline link.
  • [Optional] Numerical Methods and Optimization: these are two different topics really, college courses are probably the best bet. I have yet to find good online courses for them. But don't worry, most interviews won't really touch on them.

List of topics

Here is a list of topics from which interview questions are often derived. The depth and trickiness of the questions certainly depend on the role and the company.

Under topic I try to add a few bullet points of the key things you should know.

Machine learning

  • Models (roughly in decreasing order of frequency)
  • Linear regression
  • e.g. assumptions, multicollinearity, derive from scratch in linear algebra form
  • Logistic regression
  • be able to write out everything from scratch: from defining a classification problem to the gradient updates
  • Decision trees/forest
  • e.g. how does a tree/forest grow, on a pseudocode level
  • Clustering algorithms
  • e.g. K-means, agglomerative clustering
  • SVM
  • e.g. margin-based loss objectives, how do we use support vectors, prime-dual problem
  • Generative vs discriminative models
  • e.g. Gaussian mixture, Naive Bayes
  • Anomaly/outlier detection algorithms (DBSCAN, LOF etc)
  • Matrix factorization based models
  • Training methods
  • Gradient descent, SGD and other popular variants
  • Understand momentum, how they work, and what are the differences between the popular ones (RMSProp, Adgrad, Adadelta, Adam etc)
  • Bonus point: when to not use momentum?
  • EM algorithm
  • Andrew's lecture notes are great, also see this
  • Gradient boosting
  • Learning theory / best practice (see Andrew's advice slides)
  • Bias vs variance, regularization
  • Feature selection
  • Model validation
  • Model metrics
  • Ensemble method, boosting, bagging, bootstrapping
  • Generic topics on deep learning
  • Feedforward networks
  • Back-propagation and computation graph
  • I really liked the miniflow project Udacity developed
  • In addition, be absolutely familiar with doing derivatives with matrix and vectors, see Vector, Matrix, and Tensor Derivatives by Erik Learned-Miller and Backpropagation for a Linear Layer by Justin Johnson
  • CNN, RNN/LSTM/GRU
  • Regularization in NN, dropout, batch normalization

Coding essentials

There are a lot of resources online on how to prepare for this, some are already listed in the resources section. I think the key thing is to pick a language and know it really well. For example, for Python, if you claim you know it well and have used in some non-trivial code base. I'll assume you know why abstract base class exists, how decorator works in general and what @property means along with a few language-specific data structures (like OrderedDict, deque, defaultdict and etc).

The bare minimum of coding concepts you need to know well.

  • Data structures:
  • array, dict, link list, tree, heap, graph, ways of representing sparse matrices
  • Sorting algorithms:
  • see this from brilliant.org
  • I think in real life you'll most likely never implement a sorting algorithm, but I think quick sort is very cool and quick select / partition is used in many other places, so take a look.
  • Tree/Graph related algorithms
  • Traversal (BFS, DFS)
  • Shortest path (two sided BFS, dijkstra)
  • Trees related
  • TBA
  • Heap related
  • TBA
  • Recursion and dynamic programming
  • TBA

Calculus

Just to spell things out

  • Derivatives
  • Product rule, chain rule, power rule, L'Hospital's rule,
  • Partial and total derivative
  • Things worth remembering
  • common function's derivatives
  • limits and approximations
  • Applications of derivatives: e.g. this
  • Integration
  • Power rule, integration by sub, integration by part
  • Change of coordinates
  • Taylor expansion
  • Single and multiple variables
  • Taylor/McLauren series for common functions
  • Derive Newton-Raphson
  • ODEs, PDEs (common ways to solve them analytically)

Linear algebra

  • Vector and matrix multiplication
  • Matrix operations (transpose, determinant, inverse etc)
  • Types of matrices (symmetric, Hermition, orthogonal etc) and their properties
  • Eigenvalue and eigenvectors
  • Matrix calculus (gradients, hessian etc)
  • Useful theorems
  • Matrix decomposition
  • Concrete applications in ML and optimization

Probability

Solving probability interview questions is really all about pattern recognition. To do well, do plenty of exercise from this and this. This topic is particularly heavy in quant interviews and usually quite light in ML/AI/DS interviews.

  • Basic concepts
  • Event, outcome, random variable, probability and probability distributions
  • Combinatorics
  • Permutation
  • Combinations
  • Inclusion-exclusion
  • Conditional probability
  • Bayes rule
  • Law of total probability
  • Probability Distributions
  • Expectation and variance equations
  • Discrete probability and stories
  • Continuous probability: uniform, gaussian, poisson
  • Expectations, variance, and covariance
  • Linearity of expectation
  • solving problems with this theorem and symmetry
  • Law of total expectation
  • Covariance and correlation
  • Independence implies zero correlation
  • Hash collision probability
  • Universality of Uniform distribution
  • Proof
  • Circle problem
  • Order statistics
  • Expectation of min and max and random variable
  • Graph-based solutions involving multiple random variables
  • e.g. breaking sticks, meeting at the train station, frog jump (simplex)
  • Approximation method: Central Limit Theorem
  • Definition, examples (unfair coins, Monte Carlo integration)
  • Example question
  • Approximation method: Poisson Paradigm
  • Definition, examples (duplicated draw, near birthday problem)
  • Poisson count/time duality
  • Poisson from poissons
  • Markov chain tricks
  • Various games, introduction of martingale

Statistics

  • Z-score, p-value
  • t-test, F-test, Chi2 test (know when to use which)
  • Sampling methods
  • AIC, BIC

[Optional] Numerical methods and optimization

  • Computer errors (e.g. float)
  • Basic root finding (newton method, bisection, secant etc)
  • Interpolating
  • Numerical integration and difference
  • Numerical linear algebra
  • Solving linear equations, direct methods (understand complexities here) and iterative methods (e.g. conjugate gradient), maybe BFGS
  • Matrix decompositions/transformations (e.g. QR, Givens, LU, SVD etc)
  • Eigenvalue solvers (e.g. power iteration, Arnoldi/Lanczos etc)
  • ODE solvers (explicit, implicit)
  • Finite-difference method, finite-element method
  • Optimization topics: linear programming (and convex opt in general), calculus of variations