Interviewing with your career ladder

by David Morgantini

The problem

Picture this: You’ve got a job interview. It’s your dream job, you’ve prepared your heart out and you’re now sitting with the hiring manager. The manager starts with the standard introduction and describes the role to you. You’re sitting there thinking to yourself:yes; I’ve got this in the bag. And then the questions start:

• Tell me about yourself.
  Right… they are testing my ability to sell myself - I wonder if I need to do this in the job…
  “Well, I’ve been writing code since I was 12…”
• Discuss one of your previous projects and explain how you completed it successfully
  Um, it was a mixed success; how do I explain this in a reasonable amount of time…
  “Well, we learned a bunch of things and tried to make sure we delivered early and often into production.”
• Did you run into any obstacles with your project and how did you handle the issue?
  There were obstacles all over the place… which one do you want to hear about?
  “There weren’t any serious obstacles, we had to deal with some shifting user requirements but managed that by discussing with the product owners and shifting our architecture.”
• Do you follow the Agile software development process?
  Of course, I do! I hope my definition of agile meets with your definition of agile...
  “We used Scrum and had some challenges.”
• What do you know about multi-threading?
  Um, some…
  “Well, I’ve used plenty of threads. Mostly relied on the” damn-it what was that “thread pool and tried to” what did Maggie fix last week? “avoid deadlocks.”
• What is MVC?
  Easy, I know this buzzword, how deep into design patterns should I go?
  “Model view controller.”

And so on. Generic questions that cover a broad set of engineering skills. The interview ends. You wait a few days and get the email “Thank you for your time interviewing with ACME Co. We have decided to proceed with someone else. Good luck with your job search.” You’re shocked, you had spent time preparing for the interview, you’re a good engineer, but you’ve failed to pass the first stage of the interview process.

Familiar? The problem is that the interviewer didn’t interview you. The interviewer asked you a set of canned questions that they ask every candidate. In some senses, this makes sense - if I ask everybody the same questions then I’ll be able to judge them against each other. But there’s a problem that is illustrated below:

Clearly, Candidate 1 is a better fit for the role overall. Like most of us, Candidate 1 isn’t a perfect fit, but their skills in other areas would make it easier to grow. Unfortunately, they’ve decided to give Candidate 2 the job. Why? A one size fits all approach to interview questions means you will optimize to hire people who fit the mold of your interview questions. It’s impossible to cover all the skills required for the complexity of modern software delivery in a 1-hour interview. It’s impossible to cover those skills in an 8-hour interview. So, we create approximations and fail to hire the best candidates.

How can we fix this?

The easiest and most obvious answer to how we can fix this is personalized interviews. The question, then, moves to how do we go about creating a sensible, personalized interview that still maintains the aspect of fairness that we get by asking every candidate the same questions.

CVs are one possible answer. They allow us to create personalized questions based on a limited set of generic information. A CV only shows the things that a candidate worked on, but it does not actually cover any skills or behaviors that we expect. So, by necessity, our questions would be driven based on perceived experience rather than the role itself. We could use their CV to modify our standard questions for the role and this is used by some to create limited personalized interviews. However, a CV is the briefest summary of a skill set designed to convince a hiring manager to give an interview. It does not provide the depth required to actually design a meaningful personalized interview.

One thing that we spend a massive amount of time building, working on, and using are our career frameworks. A career framework is used to, in essence, evaluate our existing employees against the required skills and behaviors for the role. Given the amount of time and thinking and refinement that we do with our career frameworks and that they provide coverage across all the required skills, utilizing them to personalize our interview process for each candidate would allow us to avoid the issues highlighted above.

How can this work?

The first step to effectively using your career framework as a personalization tool for interviews is to ensure there is no pressure on the candidate to answer ‘yes’ to a specific skill only because they relate to the grade they are applying for. To do this, you’ll want to make your career framework generic. The simplest option would be to rename your existing levels, level 1 to level N. You could instead transpose the skills into a graduated growth path so as to remove any implicit bias around what each level might mean. This would mean creating a growth framework as described in our other blog post here. Since you’ve already built a career framework, this isn’t a difficult proposition. The other thing you’ll need to do is to make sure that your career framework also contains skills that cover the entire spectrum required by both your candidates and your employees. This means making sure that you cover both the technical skills but also people and leadership skills.

When your career framework is ready, you then ask the candidates to self-evaluate against it. SkillsMap provides this feature, though you could do this via other tools such as a google form or a shared spreadsheet. The benefits you would get via SkillsMap are the ability for the candidate to add optional notes, a well-structured evaluation process for your candidates and finally the ability to automatically generate standardized interview questions based on your candidate's responses.

Once the candidate has completed their self-evaluation you now have a huge amount of information to help build a personalized interview that covers the entirety of the required skills for the role. The key at this point is to review their evaluation and look for skills/behaviors they need and don’t have (do they have the foundations to allow them to grow in this area?) or have and that we should verify (do they actually have this skill, or did they over evaluate themselves?). The notes that your candidate adds, can give you a significant step up in available information allowing you to move quicker and really hit the critical areas you’re looking to explore.

During the interview, your task is to ask the questions that you prepared in order to dig deeper into the candidate’s self-evaluation. This means digging into their responses until you have a clear understanding of whether or not you’d actually consider them to have the skill/behavior that they claimed to have. The interesting aspect of this is that this is exactly the same sort of questions you would be asking your employees to evaluate them against your career framework. This means that you’re using the skills you build while working to help your existing employees to grow. This makes the process streamlined and easier for each hiring manager to maintain consistency.

Following this interview process means that you have a consistent, yet personalized interview process for each individual. This is still fair, because every candidate who claims to have a given skill or lacks a given skill, will be asked the same questions for that particular skill. So while you won’t ask every candidate an identical set of questions, you will end up with comparable information to make a strong bias-free evaluation of any candidate based directly on the needs of the role.

Let’s imagine again

This time, you’ve had the opportunity to evaluate yourself against the companiey’s career framework. You’ve now gone through every skill in the entirety of their career framework and answered ‘YES/NO’ for each skill and possibly added additional details. They have a clear idea of your overall suitability for the role regardless of the level you applied for. The questions start:

• Writes highly maintainable code and coaches their teammates to build maintainable systems. (YES)
  Can you please explain what maintainable code means to you? How would you go about teaching this to your colleagues?
  “Yes. Maintainable code is code covered by automated tests, it is readable and follows design patterns when required…”
• Leads the design of highly secure systems by following modern security best practices (NO)
  Can you explain some of the basic security vulnerabilities that you look out for? Would you feel comfortable running a security assessment of an existing tool?
  “Oh, well I know about XSS attacks and SQL injection attacks but most of the security work was taken care of by another team...”
• Follows the practices of evolutionary architecture when designing new software systems (YES)
  Can you explain evolutionary architecture and how your most recent architecture followed the principles of evolutionary architecture?
  “Evolutionary architecture is a set of architectural patterns that allow your system to grow and flex as you learn more about your domain...”
• Drives the agile practices of their team (YES)
  Can you explain how you would go about teaching your team lean and agile principles?
  “Teaching agile practices is mostly focused on shifting mindsets to accept a…”

This time, the questions are focused on your capabilities that are relevant to the job and where you’ve expressed personal strength. In the image above, you’re candidate #1, who with this style of interview, has your strengths highlighted. Yes, you might be ‘overqualified’, but you applied for the job in spite of being ‘overqualified’. Hiring managers have a responsibility to put the best possible people in positions to succeed. Given that the interview process highlighted that your skill set is above and beyond the requirements for the role, your hiring manager now has options they didn’t have before. They can offer you the job, they can offer you a different level than you applied for, or they can recommend you to another hiring manager. In the end, the outcome is the same. A fairer interview process helped you highlight your strengths and led to a better outcome for everybody involved.

Does it work?

This interview process is currently in use. SkillsMap provides a clean way to manage the process and allows our customers to quickly get an understanding of the person they are interviewing. It’s very difficult to evaluate an interview process before 1 year is up and it’s impossible to know how good the person you’re saying no to actually is. Early returns are super positive as our clients are able to provide clear, justifiable feedback to each candidate to ensure they know where they lack skills and the candidates who pass the process feel it is a good process that allowed them to highlight their skills.