Vetting specialized developer experience

Let’s say your job today is to find people to work on a project that uses a crazy-cool-man-I-can’t-wait-to-show-my-old-high-school-girlfriend technology. Since everyone wants to try something new, improve their resume architecture, and dominate buzzword bingo you find that every single person you talk to about said technology appears to have “experience” with it.

(Sidebar, the technology is: Objective-MongoNodeGitDBBoneOnClojureCoffeeQueryTalkScript5 but we’ll call it HipsterScript)

The problem is that you aren’t (yet) the guru of said technology – you only know it from the knowledge gained by your research, protoyping, and decision to use it on your project. You are looking for someone to come in who is farther along to help you kickstart the project in the right way. You want to avoid the pitfalls, do things in the right way, and can’t afford to hire someone who is only a month ahead of you with said tool.

How do you separate those with real experience from the ones that have just read “Top 21 Pitfalls of HipsterScript, “How to scale HipsterScript to web scale”, and “Is HipsterScript right for your project?” merely a month ago?

Here are some things you can do:

#1 Ask directly:

We are looking for a combination of people who are either a) skilled and will learn quickly or b) people that have actual deep experience with this technology; we think you are skilled because we are talking to you, but are trying to get a feel for the other aspect, can you tell us more about your experience with HipsterScript

Listen to their answer. Then if it still isn’t obvious ask them to categorize their previous work into the following buckets:

Academic: reading articles, watching videos, playing around in the IDE / SDKs.
Laboratory: creating a system for their personal use with said technology; making said technology print your name, etc.
Maintenance Project: They were on a project that used said technology but they might not completely understand how/why it was used.
Project: They made heavy contributions using HipsterScript and understand why it was used rather than HipsterClojure.

#2 Assess Fanboy status:

Tell me what you hate the most about HipsterScript

Tell me what you shouldn’t do with HipsterScript

Inexperience means that you haven’t screwed up enough yet.  And with technology inexperience means that you still love the technology because you haven’t seen its limits – I loved ColdFusion until I tried to support a system with heavy customization with it, I loved Active Reports until I had to modify one to do a pixel-small change, I loved serializing everything as JSON until it got big and I didn’t.  Experience means exploring to the edges of the box.  If they have only had success with a certain tool that means they haven’t had a chance to really push it.

#3 Assess ability to learn

All this said you can’t only hire gurus in certain technologies – you want skilled people who can adjust.  After all – what if you find that HipsterScript doesn’t scale once you get spikey periods of high write to read ratios in your database layer?

Assessing if someone can learn is hard because most developers are really good at learning.  So I’ve found that judging moxie is more effective – anybody will tell you that they can take on a PHP project without actually ever hacking at PHP but you want to find people that routinely do this – only success with jumping in with new technologies indicates that a person can do it again.

Ask them about an experience in which they didn’t know a technology at the beginning of the project and did at the end.  Did they need a training class, a book, a few days alone with the Internet, did they use peers/their network to speed up learning?

Also, search the edges of their toolset – ask them how they learned about tools that most developers simply pickup as they work – regular expressions, algorithms, database scaling, reporting technologies, logging tools, exception-handling.  Did they just do the minimum or do they truly understand them?  How well they know these will be a good indicator of how intentional they are with mastering things and just making it work.

Good luck with your HipsterScript project.