How to Ace a Technical Interview

Transcript

1. How to Ace 

   a Technical Interview
   Jacob Kaplan-Moss
   [email protected]
   Photo by Leone Venter on Unsplash
   

   View Slide

2. What we’ll cover
   1. Orientation and overview: how most companies think about hiring, and
   where the technical interviews fit into the process.
   2. General advice: tips, tools, and techniques that’ll help with all forms of
   technical interviews.
   3. Technical interview techniques: 11 ways that companies test for technical
   skill, why companies use them, and how to tackle them effectively.
   

   View Slide

3. Who the heck is this guy?
   • Co-owner, REVSYS
   • Director of Technical Services,
   HackerOne
   • Formerly: 

   Engineering Supervisor, 18F; 

   Director of Security, Heroku.
   • I’ve interviewed hundreds of
   candidates, hired dozens, and helped
   multiple organizations build or
   improve their hiring processes.
   

   View Slide

4. Disclaimers
   1. This talk won’t help you land a job if you’re not qualified.
   2. Companies mostly suck at hiring. This means you can be
   qualified, do everything right, and still not get an offer.
   (I know, right?)
   ⚠
   

   View Slide

5. Part 1: 

   Orientation & Overview
   Photo by Han Chenxu on Unsplash
   

   View Slide

6. The hiring funnel
   Source
   Screen
   Interview
   Select
   Technical Skills
   Professional Skills
   {
   

   View Slide

7. The problem:
   • We’re trying to predict performance at Task A (the job) by asking people to
   complete Task B (an interview).
   • How do we know that performance at Task B correlates with Task A?
   • Spoiler alert: usually we don’t!
   • The best interview technique would be to actually ask them to perform the job.
   Unfortunately, that’s usually unrealistic. So, we find other techniques.
   

   View Slide

8. People you’ll encounter
   • Recruiter: in-house or external, responsible for hiring company-wide, usually
   partner with the Hiring Manager to fill positions. Sometimes technical, often
   not.
   • Hiring Manager: the person responsible for hiring; usually your future boss.
   • Director: often the organization’s executive (Director/VP) is involved; e.g. you
   might interview with a Director/VP of Engineering for a job reporting to
   someone further down the chain.
   • Note: your interactions with these people are part of the interview!
   

   View Slide

9. Part 2: 

   General Advice
   Photo by Joshua Davis on Unsplash
   

   View Slide

10. If you pay attention 

    to only one thing, 

    make it this:
    Prepare
    Ask questions
    Take notes
    

    View Slide

11. Preparation
    • Ask about the format, what questions you’ll be asked.
    • Take notes, and bring them to the interview.
    • Practice: mock interviews are very helpful.
    

    View Slide

12. How to find out what you’ll be asked
    • Ask the recruiter, hiring manager, etc
    • At each stage, ask what's next
    • Ask for questions or topics ahead of each interview
    • Consult Glassdoor, etc
    • Read the job description carefully
    

    View Slide

13. Answering questions, in the moment
    • Listen carefully, take notes
    • Ask questions, clarify that you understand the question
    • "Let me repeat what I just heard, to make sure I understand..."
    • It's OK to mess up! Acknowledge you made a mistake, ask to go back / start over.
    • Do your best to project confidence. (Great interviewers shouldn’t care, but most
    interviewers aren’t great.)
    • This should go without saying, but: be honest.
    

    View Slide

14. After an interview
    • Reflect on what you were asked (you took notes, right?), and how you think 

    you did.
    • If you can take it, ask for feedback (but understand why you might not get it.)
    • Iterate and improve on your preparation for the next one.
    

    View Slide

15. Part 3:
    Technical Interview Techniques
    Photo by Scott Webb on Unsplash
    

    View Slide

16. The techniques
    1. Behavioral
    2. Hypothetical
    3. Trivia

    4. Live-coding
    5. Take-home
    exercises
    6. Pairing
    7. Online tools
    8. Whiteboarding

    9. Code Review
    10. Lab exercise
    11. “Starter Project”
    RED FLAG
    RED FLAG
    RED FLAG
    YELLOW FLAG
    ⭐
    ⭐ ⭐
    Question/Answer Coding challenges Exercises
    

    View Slide

17. How we’ll examine a technique
    • Theory: why a company might use this technique.
    • Example: an example question using this technique.
    • What use of this technique tells you about the organization. You can infer
    some things about your potential new employer and coworkers. Poor hiring
    practices are often the tip of the iceberg on a toxic working environment.
    • Preparation tips: how best to prepare for this kind of interview.
    • How to give a strong answer: how interviewers evaluate answers; what strong
    answers look like
    

    View Slide

18. Question & Answer
    interview techniques
    1. Behavioral
    2. Hypothetical
    3. Trivia RED FLAG
    ⭐
    

    View Slide

19. 1. Behavioral interviews
    • “Tell me about a time…” questions — questions about your past behavior.
    • Theory: past performance predicts future behavior.
    • Example: “Tell me about a difficult technical problem that you helped solve:
    what was the problem, and how was it solved?”
    • Behavioral interviews are a strong positive signal about the organization’s
    culture. They have the most evidence backing their effectiveness. They take
    effort to use effectively. So you can infer that the organization has its act
    together: they’re being evidence-driven, and are making a serious attempt to
    hire effectively.
    ⭐
    

    View Slide

20. Preparing for behavioral interviews
    • The best organizations will share questions or general topics ahead of time (a
    strength of this format is that it rewards preparation and doesn’t provide any
    good ways to “cheat”)
    • Look at the job description; you can probably guess.
    • Common themes: problem solving, testing/refactoring, debugging,
    performance, deadlines/execution, conflict, feedback.
    • The guide I wrote at 18F has lots of examples: 

    https://eng-hiring.18f.gov
    

    View Slide

21. Answering Behavioral Interview Questions
    • A good format: situation, actions, outcome.
    • The interviewer is looking for a story - a specific instance, or time, you did
    something. Be specific, not general.
    • Make sure it’s relevant - summarize quickly, ask if it’s a good example.
    • Include details: names, dates, specific tools you used, etc.
    • Focus on behavior: what you did, said, etc.
    

    View Slide

22. 2. Hypothetical questions
    • Like behavioral questions, but forward-looking. Theory: candidates need not
    have experience with a specific thing to be good at it, so we can ask
    hypothetically.
    • Example: “imagine you’ve inherited a Django app, and it’s running very slowly —
    the p95 request time is 5 seconds. How would you diagnose the problem?”
    • These are very common, so you can’t learn much. There’s no evidence of
    effectiveness; in fact, the data shows that people say different things when
    asked about hypotheticals versus past behavior. Orgs using hypotheticals may
    not have thought carefully about good interviewing practices.
    

    View Slide

23. Preparing for hypotheticals
    • Pretty much the same as for behavioral.
    

    View Slide

24. Answering hypothetical interview questions
    • Strong answers look very similar to behavioral; see that too.
    • In fact, if you can turn a hypothetical question into a specific example: you
    should!
    • If not: make sure you properly scope out and understand the hypothetical; since
    it’s imaginary, you need to ask questions to learn what the interviewer has in
    mind.
    

    View Slide

25. 3. Trivia
    • Questions about specific technical details.
    • Theory: check that candidates know… things… I guess?
    • Example: “what algorithm does list.sort() use?”
    • This is a terrible technique. Success at trivia doesn’t correlate with job
    performance. Companies that rely on this practice are likely to have hired
    poorly. Proceed with caution.
    RED
    FLAG
    

    View Slide

26. Tips for trivia
    • Prep: Unless you really, really need the job: don’t waste your time.
    • If you must: spaced repetition is the most effective technique for
    memorization of trivia. Several apps exist to help.
    • Answering: try not to stress out. Either you know the answer or you don’t.
    • You can try explaining how you’d look up the answer — even ask to use a
    phone/computer to try — but it’s unlikely that your interviewer will care;
    they’re likely evaluating pass/fail.
    RED
    FLAG
    

    View Slide

27. Coding Challenges
    4. Live-coding
    5. Take-home exercises
    6. Pairing
    7. Online tools
    8. Whiteboarding
    RED FLAG
    RED FLAG
    ⭐
    

    View Slide

28. 4. Live coding
    • The interviewer will ask you to write some code while they watch (physically or remotely)
    • Theory: the best way to predict if someone can code is… to ask them to code.
    • Example: “Here’s a CSV file containing healthcare plan costs. It’s got fields for zip code, plan
    level (bronze, silver, gold), and cost. Write a command-line script that, given a zipcode, prints
    the second-lowest bronze-level healthcare plan in that zipcode.” 

    (Source: https://homework.adhoc.team/slcsp/)
    • The theory is sound, though little evidence of effectiveness exists. If the problem’s well-
    designed and appropriate for the role, that’s probably a strong positive signal.
    • Be wary of companies forcing weird environments — spyware, coding in Google Docs, etc.
    These are RED FLAGS
    

    View Slide

29. Preparing for live coding
    • Ask to use your own machine, if possible. If not, make sure you know what the
    testing machine will be.
    • If you’re using your machine: make sure everything’s all set up to write/run
    code and tests. Turn off distractions.
    • Practice with the tools you’ll have available. Fumbling with your editor or
    terminal is a bad look!
    • Practice writing code under time pressure. For exercises, try Project Euler,
    Advent of Code, etc., and set a clock.
    

    View Slide

30. Doing live-coding exercises
    • Talk through the problem with the interviewer; make sure you understand it. Ask questions
    until you do. Check your assumptions.
    • You’re being graded on your process as well as the results. So talk out loud as you do
    everything. Explain what and why you’re about to take each step. If you need to stop and
    think, say so. If you get stuck, ask for help.
    • Start outside-in: write the function signature / CLI script / etc., make sure the inputs and
    outputs are what the interviewer expects. (Sometimes this is done for you.)
    • Write tests next, if appropriate. Validate that the tests match the interviewer’s assumptions.
    • Finish the code. Don’t forgot to keep talking!
    

    View Slide

31. 5. Take-home exercises
    • Like live-coding, except you’re given the challenge and some time to work on it at your own
    speed. There’s a deadline, typically around 1-3 days.
    • Theory: We want people to write code, but coding while someone is watching is unrealistic
    and stressful. We’ll see more realistic results if we give people more realistic conditions.
    • Examples: https://homework.adhoc.team/
    • Like live-coding, take-homes are on solid theoretical ground but have little evidence of
    effectiveness. Anecdotally, they seem more effective than live coding.
    • Good-home exercises shouldn’t take tons of time. Asking you to spend more than half a
    day is a — a sign that the company doesn’t respect its candidate’s time. Proceed
    with caution.
    ⭐
    RED FLAG
    

    View Slide

32. Preparing for take-home exercises
    • Much the same as live-coding, except asynchronous. Be sure you really
    understand the problem before diving in.
    • Some good practice exercises:
    • https://homework.adhoc.team/
    • https://adventofcode.com/
    • https://projecteuler.net/
    

    View Slide

33. Doing take-home challenges
    • Mostly: just write code normally.
    • If you get stuck: reach out to the recruiter/hiring manager immediately.
    • Pay close attention to better practices: tests, documentation, comments,
    clean code, tests.
    • Did I mention tests? Submit them even if you’re not asked to.
    • Double-check that your code runs and works before submitting. Document
    how to run the code even if you think it’s obvious.
    

    View Slide

34. 6. Pairing
    • Like live coding, but you’ll be paired with a company engineer and write the
    code together, pair-programming style.
    • Theory: we practice pair programming in real life, so let’s do it in the interview!
    • Pairing tells you that the organization probably practices pairing! As an
    interview technique it’s unusual, though it’s on the same theoretical ground as
    the previous two techniques.
    

    View Slide

35. Tips for pairing sessions
    • Prep is mostly the same as live coding.
    • Read up on, and practice pair programming if you haven’t done it before. It can
    be uncomfortable if you’ve never done it.
    • Talking is even more important. Talk about every step you take.
    • If it’s really pairing, you should expect your pair to write some of the code too.
    So if you get stuck you can really get help, this is great.
    

    View Slide

36. 7. Online Tools
    • Like the previous coding exercises, except you have to use an online 

    platform to write, submit, and grade your code.
    • Theory: I want to ask candidates to write code, but I’m too lazy to grade it
    myself.
    • This is a poor technique. Good coding interviews judge communication and
    code quality as much as they do success, but online platforms can only measure
    success. This indicates that the company doesn’t prioritize hiring effectively.
    Proceed carefully.
    RED
    FLAG
    

    View Slide

37. Tips for platform-based challenges
    • Prep: mostly, I wouldn’t bother with specific prep for platforms.
    • Exception, some of these platforms (HackerRank, TripleByte) can be 

    good practice for you, for technical interviewing in general. Just be wary 

    of companies forcing the issue.
    • If you must use one: try to get on it ahead of time, explore the environment.
    Some can be very awkward.
    • Examine your output very carefully: these platforms can be unforgiving.
    Sometimes even just whitespace can cause a fail, and often you have limited
    retries.
    RED
    FLAG
    

    View Slide

38. 8. Whiteboarding
    • Like live coding, but they force you to do it on a whiteboard for some reason.
    • Theory: Microsoft used to do this, so… we should too?
    • This is a terrible, horrible, very bad practice. It has no predictive power
    whatsoever. Companies that force you to whiteboard are wasting your time.
    Proceed with caution. If you have the power, consider walking out.
    RED
    FLAG
    

    View Slide

39. Tips for whiteboarding
    • Try to get them to let you use a computer.
    • Explain you’ll probably make mistakes in syntax without a real editor.
    • Most tips as for live-coding apply: outside-in, talk a lot, etc.
    • Take advantage of it being a board: draw diagrams, etc.
    RED
    FLAG
    

    View Slide

40. Exercises
    9. Code Review
    10. Lab Exercise
    11. “Starter Project”
    ⭐
    YELLOW FLAG
    

    View Slide

41. 9. Code Review
    • You’re given some example code (or a pull request) and asked to review it.
    • Theory: you can demonstrate coding competence through reading and
    understanding other code. And, code review is an important part of an
    engineer’s job, so it makes a good technical screen.
    • Like other coding exercises, this is on solid theoretical ground but lacks good
    data. It indicates that the organization puts a premium on code review, which is
    generally a good sign.
    

    View Slide

42. Preparing for code review
    • Study good code review technique. Some resources:

    https://github.com/ryanmcdermott/code-review-tips

    https://github.com/joho/awesome-code-review
    • Practice: ask a friend to review their code, or find a project with patches
    needing review (Django usually has plenty!)
    

    View Slide

43. Reviewing code in an interview
    • Talk through the code’s intent with the interviewer, make sure you understand
    it. Remember to ask questions if things are unclear.
    • Start with the basics: typos, errors, readability, comments.
    • Then look for bigger stuff: edge cases, security, performance.
    • Make concrete suggestions, don’t just flag errors.
    

    View Slide

44. 10. Lab Exercises
    • Any sort of hands-on exercise with a simulated environment — a running web application, a
    virtual machine, etc. You’re given tasks to accomplish in this simulated environment.
    Sometimes this is live, sometimes take-home.
    • Theory: direct simulation — if Task X is part of the job, create a repeatable environment for
    candidates to do Task X in an interview setting.
    • Example: “there’s a running web app at https://company.example.com/, which has several
    security vulnerabilities. Find as many as you can!”
    • Good sign. Technical lab exercises are on solid theoretical ground. Technical exercises haven’t
    been studied directly, but exercises more generally have (e.g., journalists submitting writing
    samples), and are highly effective. Lab exercises take serious effort to set up, and indicate that
    the organization is serious about hiring well.
    ⭐
    

    View Slide

45. Lab exercise tips
    • You’ll typically get some preparation steps up-front, make sure you follow and
    understand them. Ask questions if anything’s unclear.
    • Besides that… there’s little you can prep for — one of the reasons exercises are
    good techniques.
    • If you’ll be working the lab live: talk out loud as you do everything. Explain what
    and why you’re about to take each step. If you need to stop and think, say so. If
    you get stuck, ask for help.
    

    View Slide

46. 11. “Starter Projects”
    • Work directly with your potential team to complete a simple but realistic 

    project (build a real feature, fix a real bug, etc.). See also:

    http://www.craigkerstiens.com/2011/12/02/how-heroku-works-hiring/
    • Theory: actually trial working together as a team, in a real way.
    • Example: when I interviewed for the Director of Security position at Heroku, I
    managed a real security incident (a PostgreSQL vulnerability).
    YELLOW
    FLAG
    

    View Slide

47. 11. “Starter Projects”
    • Starter Projects are complicated. They’re highly predictive, not just of
    technical ability but also ability to work as a team. They provide a huge 

    amount of confidence. But they require a major time commitment from the
    candidate. Companies that use them are serious about de-risking hiring, and
    are willing to lose good candidates over it.
    • Projects shouldn’t last more than 2-3 days. The company should be very flexible
    to your schedule. Your work should not actually be used by the company.
    That’s not a hiring exercise, that’s unpaid labor.
    YELLOW
    FLAG
    

    View Slide

48. Starter project tips
    • Like lab exercises (but moreso): specific preparation is hard.
    • Remember that you’re being judged by your interactions with the team, as well
    as your technical ability. Be friendly, pleasant, and professional. Treat everyone
    like they’re your coworkers — they might be!
    YELLOW
    FLAG
    

    View Slide

49. Techniques: Recap
    Question/Answer
    1. Behavioral
    2. Hypothetical
    3. Trivia

    Coding challenges
    4. Live-coding
    5. Take-home
    exercises
    6. Pairing
    7. Online tools
    8. Whiteboarding

    Exercises
    9. Code Review
    10. Lab exercise
    11. “Starter Project”
    RED FLAG
    RED FLAG
    RED FLAG
    YELLOW FLAG
    ⭐
    ⭐ ⭐
    

    View Slide

50. Final thoughts
    • Interviews are weird and stressful. It’s normal to feel nervous. Practice really
    helps: the more you do it, the easier it gets.
    • Remember to prepare, ask questions, and take notes!
    • Most companies are bad at hiring. You might do everything right and still not get
    an offer. Try not to let it get you down!
    

    View Slide

51. Good luck!
    Jacob Kaplan-Moss
    [email protected]
    Photo by Jeremy Bishop on Unsplash
    Deck background by Joanna Kosinska on Unsplash
    

    View Slide