Category Archives: consulting

A system for analyzing systems [Part 2]

This is the second part of an (awesomely epic) series on how to analyze a software system quickly, click here to view Part 1.

So now you (having the right attitude and documenting everything) have viewed the main nouns (data) and verbs (use cases, workflows).  What do you do next? Profit?

#3 Find integration points and environmental concerns

It is important to understand the system within its software ecosystem – find the system edges:

  • Make a list of the technologies used (again – avoid judgment)
  • Make a list of the integration points (other systems, off the shelf products, web service apis, manual steps in overall process)
  • Look at dll references, build files, web references, configuration files, database connection strings, etc.
  • Draw a simple sketch of the application within its context with others

#4 Read some code

This is normally where developers start – reading semi-colons.  But maintain some discipline:

  • Make a simple outline of the project structure and main namespaces and entry points. Input, processing, output.
  • Try to not go down every rabbit hole too quickly – add to your WTH list liberally and then circle back.
  • Guided by what you know already dive in based on what makes sense – read the billing module, the calculation piece, the ranking algorithm, or the integration with Facebook.
  • Document system conventions, code conventions, and architecture. The build order can be useful and painfully revealing here – on larger projects I break out ndepend to see the overall structure.

#5 Read some documentation

If the project has some artifacts now is a good time to read them. Most people will read them earlier in the process but since documentation is a record of what was the truth during a time in which the truth was still being figured out when people had time to do such things I’ve grown untrusting of it until I know what is what in the application. Project documents are fascinating in that they provide historical clues to design decisions, project status, pressure, and onetime events like data conversions and client meetings that affect the system in the future. Read up and take notes. Be aware that some pieces of documentation are position papers and do not represent the current reality.

#6 Talk to some people

Armed with nouns, verbs, and a basic understanding of the code go get your questions answered by someone on the project (if they will return your calls). A few tips for this process if a developer is handing off to you:

Realize that when a developer hands off they will do two things:

  • Blow off steam (I really didn’t want to do this but that freakshow client forced us to)
  • Play defense / apologize (I was under pressure so just hacked this up)

Let this happen, avoid judgment and be friendly and helpful.

#8 Identify the Win Condition

By this stage in the game you should know the “win condition” of the system – when it works how it provides value to a customer.  Examples would be:

  • “Allows users to find out about flight delays before other airlines, saving them time and allowing them an advantage against other travelers in a cancellation scenario” (FlightCaster)
  • “Provide an easy to use way to update online ads across multiple platforms based on your inventory” (Tentail)
  • “Allows users to find out about current events much faster, trade cat pictures, and waste half of their life” (Twitter)

#8 Wrapping up

So now you have your document with just Facts and Questions having hopefully whittled down your WTH list. Polish up your facts and add a summary statement and the application “win condition” to the top and you are done:

EasyTrack is a web-based accounting system used by small businesses that is built on ASP.NET Web forms with a SQL Server 2005 database. It integrates with Quickbooks, exports transactions as csv, and has a web service api for 3rd party integration. It syncs nightly with a client customization of an older version that uses the same database but has a non-web frontend. It works in Internet Explorer. Its niche is being an easy to use tracking system for a discrete set of common accounting tasks that are harder to do with larger systems.

A system for analyzing systems [Part 1]

We seek definition to understand the system so that we can discern
the rules so that we know what to do next so that we win.

— Michael Lopp, Being Geek: The Software Developer’s Career Handbook

How do you approach learning a new system that is dumped on your lap like a spilled plate of nachos? Let’s say somebody walks up to you and says:

Hey nerd- we need you to look at the EasyTrack system tomorrow – we have a buglist that we need you to churn through before the end of the week. Ralph was working on it but after his nervous breakdown he won’t return our calls.

As developers we are asked to jump into complex software systems all the time and a big part of our value is simply understanding systems (not just changing them or making new ones). This is a skill that we need to be intentional about. We need a system for learning systems. Here is mine.

First steps: Attitude

Avoid judgement of the people who have worked on the system, the client’s goals, the choice of typeface on the one page of system documentation, the developer’s skills and experience, the organization of the code, the choice of technology, the choice of coffee used during the project, the organization of the altar built to the stored procedure that does everything, et al.

You have no idea the in-the-womb conditions on the project and don’t know the real trade-offs that were in play. Like many negative emotions judgement will close you down and cloud your thought so keep a clear mind when reviewing a system. Starting with a good attitude will also help you in talking to people on the project later on.

First steps: Document

As you work through the steps document what you are finding. When I do this my goals are:

  • To make me feel like I’m doing something as just staring at a new system can be brutal.
  • To make a deliverable that I can publish at the end of the system analysis if needed.
  • To allow me to have a place to store questions that I can’t find out from just reading code.

On that last point – I normally have Facts, Questions, and WTH sections in the document that I use as I work through a system. Facts are things that I have learned definitively about the system, Questions are open questions that I have researched and can’t find an answer to without asking someone or gaining access to something else, and WTH are bogeys – items that I see that I don’t understand that I’m deferring until later.

An example – as I’m reviewing the database I see an oddly-named stored procedure: spSyncOrderNoTransactionWithRollback? Hmm… So we add this to the WTH list until we can go back and read through it, where it is called, etc.  So we might end up with:

WTH

  • In spSyncChangeOrderNoTransactionWithRollback - how can you have a rollback without a transaction – is this nested?

Which leads later to the following:
Facts

  • Some of the stored procedures exist in the database but aren’t used by the application and are part of an export / sync process.  They start with spSync*
  • Some, but not all, of the stored procedures use nested transactions with checkpoint.

Questions

  • What is the export/sync process called by job execSync4_Bak that runs every night and calls the spSync* sprocs? [No access across linked server to the tables it is writing to]

As I work through the system I’m simply turning WTHs into Facts or Questions. The temptation once you have a basic understanding of the system is to stop typing – don’t do this.

If you maintain the facts list and polish it up you’ll have a nice system overview document. The easiest time to document a system is when it doesn’t feel like you are having to write-up up a bunch of things you already know.  So by writing while learning the system you can trick yourself into producing nice documentation that is much easier to keep updated later.

#1 Start with the data, don’t start anywhere else

I didn’t say start with the project plan, system documentation, email threads, interviews, test plans, or insulting notes about the system on the bathroom wall. The live data is the only thing that you can be sure is actually real in the beginning.

Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious.

— Fred Brooks, The Mythical Man-Month: Essays on Software Engineering

I’ve found this quote to be true – your code follows your data and you will get further faster if you understand the basic nouns in the application first. Seeing an Order table with a StatusId field gets you along the road faster than trying to read a 2,000 line method:

1
void placeOrder(int? productId, int priceStructureId, decimal price, int priceSigDigits, int userKey, string userNameInternational, bool isA, object? requiredBlob, out long intRetVal1, out long intRetVal3, out bool multi)

So go ahead:

  • Create database diagrams to identify relationships
  • Begin drinking if there are no enforced relationships
  • Look at table sizes to identify transactional vs. configuration tables
  • Look at meta and programmable database constructs – sprocs, functions, jobs, indexes, linked servers, etc.
  • Begin thinking through the possible workflows – an Order table with OrderDetail which points to Product and PriceStructure implies that in the application someone is purchasing something and that prices vary.

#2 Use the application

If at all possible get a login to the application to view it as a customer does. You can’t always do this safely but you should try. Developers are always saying that the UI is the tip of the iceberg and that all the logic is in the back-end, but knowing the data model and having poked around the application will teach you the vocabulary of the system very quickly.

  • What are the main uses cases [verbs] in the application?
  • What nouns/entities that we saw in the data model are actually shown to the user?
  • Is there data shown to the user that wasn’t in the data model? [hint for the next step]
  • What doesn’t appear to be working or isn’t intuitive? Enjoy this perspective as once you learn the rest of the system you’ll lose it forever.

Tune in for the exciting – no electrifying – conclusion to how to analyze a system for fun and profit.

The Three Laws of Robotics (for tech recruiters)

The tech market is so hot right now that if you are a developer and can tell the difference between O(n) and your own butthole you might be getting calls from aggressive recruiters. Recently I’ve had two recruiters that I’ve never met contact friends saying that I had recommended them for a position and I’ve heard stories of continued home phone calls, at-work phone calls, and all sorts of odd LinkedIn abuse.

I’ve heard many people say that recruiters don’t have much value, but I strongly disagree – we just need some agreed-upon moral boundaries. Let’s use Asimov’s Three Laws of Robotics [If you haven’t heard of these I’ll just hold on to your geek card while you go read that…]:

Law 1: A recruiter may not injure a human being or, through inaction, allow a human being to come to harm.

Harm means:

  • wasting my time
  • affecting my reputation
  • affecting my friend’s reputation

A recruiter shouldn’t hurt me, they should only help me and themselves in that order. Long-term it can work out for both of us; don’t affect my reputation negatively and I won’t affect yours either.  Some more specific ways they can harm me:

  • Submitting my resume to anybody without talking to me
  • Saying that you know me when you don’t
  • Talking to a 3rd party about me without us knowing each other
  • Being dishonest about the positions that you have in order to have a conversation with me

Law 2: A recruiter must obey the orders given to it by human beings, except where such orders would conflict with the First Law.

If I say I am not available at this time, don’t want you to do anything on my behalf, or would like you to stop calling me and crying into the phone – I mean all these things [But if I’m honest the crying does help me sleep].

Law 3: A recruiter must protect its own existence as long as such protection does not conflict with the First or Second Laws.

Of course there is a place in this world for recruiters and they should be allowed to make a living.  There is a lot of tech-recruiter-bashing but they are valuable for a hiring manager in that:

  • they can maintain a healthy pipeline of competent people [still up to you to find the people out of this pipeline]
  • they can filter people by personality [still up to you to detect people you don’t like]
  • they at times have a very good sense of the competency of tech managers (as they see loyalty from movement)
  • they can tell you who *won’t* work [they will filter out some incompetent people and some jerks]

They are also valuable for people looking to get hired:

  • they can tell you a good deal about the internals of particular companies
  • they can provide serendipitous events like connecting you with people that you end up working with later
  • they can help you get back out there after staying with one company for an extended time by giving you a feel for the market

We can even apply the later-added-4th law to recruiters:

Law 4: A recruiter must establish its identity as a recruiter in all cases.

Recruiters go to user meetings, tech conferences, social meetups, etc. I’m not sure if recruiters have gang signs (throw up a P for placement) but you should identify yourself as a recruiter to everyone other than the tollbooth operator that you talk to while you are working.

Following these laws will go far in re-establishing the reputation of recruiters within the technology industry.  Also: free pie mailed to my house.  Contact me for more details.

How to work from home without going insane (purple monkey dishwasher)

I’m writing a book about successfully working from home; click here if you want to know when it is complete.

So you’ve decided to work from home. As a rookie-veteran of working without a traditional office for one year I’m here to tell you that it is the greatest and worst thing that can happen to your work life – much like being transferred to a glitter-packing facility. What follows is a quick two minute guide of what you need to know before you take your pants off and turn up those old Hootie and the Blowfish albums.

Interruptions

Working from home completely changes your interrupt cycle. At a typical desk job you are interrupted constantly due to meetings, cake feasts, fire drills, people coming over to tell you about what the lake was like on Saturday, daydreaming about tripping said individuals, etc.

When you work at a location of your choice you can control what distracts you. If you want to work for 4 hours and not use the bathroom you can do it; if you want to work with 2 lbs of nachos taped to your face like a beard while wearing a sombrero filled with nacho cheese for snacking you can do this. Most people think they will be far more productive due to being able to control large blocks of time, but I found that the experience was quite jarring.

What you will realize is that outside of your normal distractions your body has learned to not focus for very long on anything. In my case after years of working in an environment where I was constantly interrupted I couldn’t focus on anything for more than 20-30 minutes (about the longest free period I had on average). I would internally interrupt myself constantly: twitter, facebook, doodling, trying to clean nachos that fell onto the keyboard from my nacho beard, calling the manufacturer of said keyboard after the ‘s’ key stopped working, writing a strongly-worded letter calling the manufacturer a bunch of “lazy jackae”, picketing outside their headquarters, etc. Typical stuff.

When you work in an office you don’t allow yourself to constantly be interrupted internally – you simply can’t watch every video that looks funny, you can’t read each article on The Morning News everyday, you can’t keep up with Hacker News like you think karma points are worth money.

The way to get over this is pretty simple – practice. I for one spent about a month making love to The Pomodoro Technique (meta: learning to make love using this technique is experimental and not recommended by most doctors) but it worked quite well for me in training my brain to recognize interruption events and stop them. Now I can go over an hour without eating a nacho.

Pressure

When you have a ‘bad’ day at a normal job you still feel a sense of accomplishment – you drove to work, you drank some coffee, you attended a few meetings, you chatted with some co-workers about how little progress you were making – you did some concrete tasks.

Working from home alone you have days that you feel like you get *nothing* done. As soon as you get going you are interrupted, you spend a few hours working on something that you later scrap and start over, you can’t figure something out. These days happen in an office but when you are alone with them they create extreme emotions. The thing you need to remember is that one day like this can be followed by a sugar-rush Work-From-Home monster day: a day in which you get as much done in a day as you remember getting done in a week at BigCorp. An entire feature imagined, mocked up, coded, then just plain mocked, debugged, re-tooled, polished, stabilized, and then shipped.

Crippling Depression – ride it like a wave

If you work at home for long enough away from other real people you will be surprised how much you will miss the interaction. The annoying “did you see the game?” water cooler talk and “OMG Its Friday!!” chit-chat that used to make you want to hide in a conference room is actually a pretty effective social convention to avoid The Question: “Why am I working right now when I could be doing XYZ?”.

While working you will have many moments when you will think things like:

  • I work from home now, I could go take a walk – right now!
  • I work from home now, I could go eat some yogurt – right now!
  • I work from home now, I could ride a bike – right now!
  • I work from home now, I could watch The Wire – right now!

In the spring these thoughts are tough and you will stare out the window and cry a single slowly descending tear before turning back to your semi-colon delimited job and push through. In a normal office you don’t think about the difference between working and playing hooky because the threat of getting fired for playing Grand Theft Auto III at your desk is very real.

In addition you will need to take steps to keep in touch with others in your field less you become a Work-From-Home monster like many that I have met. Signs you are becoming cripplingly depressed without realizing it from working from home:

  • You see a former co-worker and talk for 20 more minutes than would be normal. Even after they have gotten up and left the stall you continue.
  • You are way too active on twitter, facebook, g+, etc. and constantly send videos to your friends like you work at Tosh.0.
  • You watch TV instead of listening to music and you talk back to the characters.  You think you hear Ferb talk back to you.
  • You drink Diet Pepsi just to let the pain make sure you can still feel something.

If any of these things happen put some pants on and go to a coffeeshop.

The greatest thing about working from home is your kids. They are also the worst thing.

This might be a personal problem as:

  • I work on the same floor of the house as my kids typically play.
  • My wife currently stays at home with the kids.
  • Before I worked from home I never listened to music while working (due to a deep-seated and irrational fear of being caught singing out loud a Jon Secada song)

I was very stressed with the noise of my kids and the general stress that kids cause when you do concentration-based work. I felt like a bad Dad because I would have to tell them 8 times that I wasn’t “done yet” but was just coming out to go to the bathroom, refill nachos, etc.

Once I realized that working at home meant more time with my kids I just mentally substituted the time spent in the car with time on the floor with them, and my stress melted away awkwardly like reheated queso.

Less informal communication means more organization

I’m an organic and an improviser. I’m not naturally organized. But working from home in the absence of informal methods of communication means that you need to have prepared the following:

  • a to-do list that you can share (workflowy, RTM, etc.) in less than one minute
  • a general daily plan that you can recite at beginning, middle, end of day.
  • a weekly plan of what you want to get done and what you did last week
  • a plan for when you will meet with the client, boss, parole officer next

Enjoy it

I know that at some point my work from home life will be put on hold or go away. And I decided when I started that I wanted to do a good job and to enjoy it enough that when I looked back I wouldn’t say that I regretted much. To remind myself of this I wrote “San Diego means a whale’s vagina” on my whiteboard (since this kinda isn’t the sort of thing you can do in a normal office).

So go – enjoy it. And please just go take a shower.

I’m writing a book about successfully working from home; click here if you want to know when it is complete.

Great discussion, as always, on Hacker News.

Republished on Lifehacker.

Fixing problems Part 1: Introduction and Attitude Adjustment

As DBAs, software developers, Homo Sapiens, and lovers we have to solve problems.  There is a common misconception that support is for the more junior folks on a team and thus being good at it is a sign of “being a little baby”.  While support is a great way to learn a software ecosystem and organization and thus “grow” a junior person into a senior one a lot of problem-solving falls to the rock star programmers or wizard DBAs in most organizations.

Yet some of these people aren’t any good at it.  In fact, they are awful.  I’ve seen people that are very good at doing very hard technical things – creating something from nothing, thinking of all the things that could go wrong, refactoring and integrating a large subsystem, etc. – fail at simply fixing a problem with an existing system. You can take your rock star and send them off to fix a support issue and they will return with a confused look, eight hours of wasted effort, a missing finger, and an STD.  Working with some truly gifted problem solvers I think that there are some differences in Attitude, Practices, and Skills that separate the junior and senior problem solvers.  Let’s start with Attitude.

First, there IS a problem

Never deny that there is a problem.  If someone is at your desk, on the phone, flooding your email with red exclamation points, or outside your house knocking on the window there is clearly a problem.  The problem might be that they don’t understand something and the problem might not be your fault, but the issue should be treated with respect as a real problem if they took the time to contact you.  Don’t deny it or argue.  Why would you deny it anyway, because you see support as negative.

Don’t see support as a negative shameful thing or a junior task

As long as things keep changing, there will always be problems.  (This next part is hard to put down in writing) If you aren’t writing bugs you aren’t writing software.  If you aren’t changing systems you aren’t working.  A support issue is not an insult, a bug is not a breakup.  Yes, you should make sure that you don’t write infinite loops, and yes you should make sure that you test the latest SQL Server upgrade before you install it in production at 10 am on the last day of the month.  But in most organizations there is a constant to and fro of creating new things and then fixing issues that crop up with them, so don’t pretend as if a problem is an anomaly.  True root cause and issue prevention are topics for another post, but don’t act surprised that software systems don’t always work as expected.

Confidence

In my home office I have a fortune cookie taped to one of my monitors that says: “You have the ability to analyze and solve any problem”, and over time I have started to believe it (by looking at it 27 times a minute).  The fact is that most people that are good at support are good because they believe that given enough time they could fix any issue.  ANY issue.  Given 20 years and enough coffee they could learn C/C++, reverse engineer SQL Server, learn about cross-platform multi-threading, and fix that bug.

The ability to not freak out and lose it

Something is broken but don’t be scared (its just moving electrons for the highest bidder).  The fact that someone is at your desk and not someone else’s is a good thing, don’t panic and freak out and yell things that you’ll regret later (Aside: yelling is always regretted – only thing I’ve not regretted yelling “OMG ITs MILEY”). Don’t blame people or come off as condescending; assume that they are your desk because they know you can fix it, not because they think you caused it.  Figuring out root cause and a long-term solution are separate things; as are fixing and blaming.  You are in charge of fixing for now, so just focus on that.  Besides, over time a calm person is going to be relied upon more while those freakout will only end up on reality shows.

Next post – Practices that improve your ability to fix complex technical issues.

Key considerations for your next development job

As a developer, your basic job is to create things. Since the world needs software in every industry you might think that one is the same as the next, and you’d be shamefully wrong. Outside of the obvious questions you should ask yourself when looking for your next gig – how sharp are the coworkers, how good is the tech, how hard are the problems, how good are the tools – you should also ask: “Who are we working for?” It turns out that who you produce for can more directly impact how much you like your job than you might think.

Internal* vs. External client

Internal clients don’t matter as much as external ones. Yeah I said it, wanna fight about it? Working for a bank on their internal accounting software is not the same as working on banking software that is sold to banks. Period, close bracket, EOF. The reason for this is that one is a profit center with real financial pressure, and the other is a cost center, with pressure to simply exist cheaply. A profit center has direct competitors that you sometimes have to react to, but a cost center rarely implements new product due to hearing that an internal customer at another company is happy.  The internal vs. external switch plays itself out in multiple subtle ways:

Rate of Change

A profit center tries multiple things, watches competitors to match features, explores new lines of business, etc. A cost center does not take much risk, and thus is more setup for small improvement or the support of company growth.

Rate of spending

Another way this plays out is in lack of budget flexibility – since a cost center is under pressure to lower costs their budgets are smaller and less innovative. The type of managers that run these organizations are the special type of demon that is good at finding ways to save money – like on hardware or crappy coffee.

Rate of Respect

In a profit center the business leaders interact with the technical leaders and producers enough that they begin to understand their importance. Over time a mutual respect grows and is a healthy team behavior. In a cost center at times the cost center is in a servant position and the IT functions are not held in the same level of respect.

Producer vs. Maintainer

There is another subtle difference in “software developers” at times that can play out in affecting you position. Some people build tools and processes and some people build deliverable product. In the software realm the tools can include continuous integration modules, deployment tools, operational helper tools, code generators, etc. Product includes things that directly sell outside your organization. If you are a developer working on the toolset *primarily* you are secondary to those working on the end product – you are in effect serving an internal customer.

Deadline driven vs Shame-driven

When we interview someone we always ask how the end game of projects work – “Do you work off of deadlines?”, “Who sets these deadlines?”. Many internal customers have false deadlines because there is no competition – are they going to go use another internal accounting department? The same fire does not always exist in those working in internally-facing companies. Just because your coworkers are smart this doesn’t mean they have any hustle.

Industry and Economy

The industry that you are building solutions for can matter because the level of tolerated innovation differs across industries and product categories. My first full-time programming job was working on an audio-dispatching call center product that was sold to air traffic controllers and 911 call centers. The sales cycle for this product was very different than other industries – if a client saw a demo and a single thing went wrong they would typically say: “We will reevaluate changing our product in 5 years, get back to us then”. This is obviously different than the change cycle for a web-based project management tool that might receive changes every two weeks.

Customer distance

How “close” you are to the customer matters as well whether that customer is internal or external. While only some developers are interested in directly speaking to customers, the dev team’s distance to the customer can affect whether or not what they are building matters. As a producer you should care very deeply about whether you are building the right thing.

* Internal clients means full-time – if you are working on a project for an internal client the effects are more minor. Most of what is mentioned above is when you wake up everyday to a world of internal client demands only.