Working at a startup is like riding a bike (off a cliff)

My life the last six months has been consumed by two activities: training for a long bike ride and trying to help get a startup profitable. I’ve been surprised to learn that I can take lessons learned from each and apply them to the other.  So I present (with no commercial interruptions) three lessons:

Know your limits going in and don’t change them

When you get on your bike for a training ride you have to decide how much stuff you want to take with you – the more stuff you take the harder the ride is:

  • 1 or 2 water bottles? Will there be more water down the road?
  • 2 spare tubes, CO2 cartridges? wrench? phone? credit card? spare bike? cab fare?
  • food: gu, Honey Stinger, chicken biscuit, rack of ribs, hot pocket? (or all of the above)

Once you get on the bike to do a 30 mile ride you can’t easily decide to do a 130 mile ride as you will run out of food, water, and daylight.

When you decide to take on a startup as a founder or an early employee you have to decide how much risk to take, and it’s not a good idea to suddenly change that risk equation during the life of the startup (as you most likely will be tempted to do so):

  • Pay: how much equity vs. raw salary? How long can you survive on a lower salary?  (Are you willing to switch to cheap toilet paper?)
  • Time: What are you giving up: sleep, family vacations, taking the family to the zoo on Saturdays, hobbies (what you used to do before you had kids), time for friends, time for extended family, time to read a newspaper or cereal box, time for TV (How many seasons of Jersey Shore are you willing to miss?)
  • Opportunity: how long are you willing to go and say ‘No’ to every additional job/opportunity/sudden trip that comes along during your tenure at the startup?

You can’t take 70% of your salary with massive options vested in 4 years and then run out of money for skittles 2 months in. And when the company tells you that they can now only pay you 60% of market you shouldn’t try to make it work as you will, just as in cycling, run out of food.

A better way to say this that works for both: you and your family should know the terms/conditions going in:

  • I’m going on a 120 mile training ride and will call you at mile 60; I’ll be home and showered in time for dinner.
  • I’m working for a startup for 6 months at 80% of market rate, I will work normal hours and from 10:00PM until I go to sleep after everyone else, doing it for experience, do not expect options to ever be worth much, we will reassess in August.

Getting your rules of engagement out in the open makes you not get lost in the excitement of good weather, good market conditions, or temporary good feelings. And this, my friend, brings us to the next point.

No matter how good or bad you feel, it won’t last

This is a very common saying in endurance racing and is quite effective in calming nerves during a monster event. I have experienced the thrills of riding my bike for eight hours and feeling completely invincible only to feel like I should call an ambulance 10 miles later. In my experience on long rides you start out feeling good then go through period in which you start to feel crappy and doubt why you are doing the ride at all. As the ride progresses these good/bad periods start to jump back and forth and the highs get higher and the lows get lower.

There are parallels in the startup world. For the first month building something from scratch you will feel great – no HR BS, no this no that, no limits. As you crank out code rather than sleep or see your friends you will let doubt creep in and wonder if your rules of engagement need to be adjusted. You will go through lows unlike anything you can experience at a big company. Short of rare events, you simply can’t have the same feeling as you do at a startup where you need to complete a feature of fix a bug for a demo the next day or *almost literally* you will lose big – let your partners down, potentially lose you and your partners money, and perhaps lose friends all the while having spent massive time away from those you love in order to execute your failure.

That being said, the highs at a startup are higher than anything you can do in a normal employment situation. Seeing a famous company use your application, getting chosen as a key partner, watching as an on-the-fly demo with a famous person goes crazy smoothly (all with software you wrote last week), watching the revenue numbers grow, watching as people go from beta to entering their credit cards, watching as customers start complaining because they see value and *want* what you are building. And of course the biggest high of all – the potential to make more money in a few years than you could in many more than a few years.

These highs and lows need to be tempered just like in long-distance riding. Sure – we are feeling good now but let’s still stop and eat and refill water while we have the chance. Ok, this looks like it’s working but let’s hold off on doubling-down on equity or buying that new mobile margarita machine just yet.

Embrace being different and enjoy its rewards

And, of course, the first similarity that struck me was this: you need to embrace the consequences of doing something different.

Cycling is a weird sport. Very, very tight clothing, little of the professional sport on TV, lots of tension between driver and cyclist, an unmanly image, and a big upfront cost (the bike) keep most people away from it and in company softball leagues and on their couch. But if you move away from what the “masses” do you will find bigger rewards – cycling trains your heart to recover unlike any sport, you get to travel massive distances that you can’t in other sports, you get to climb mountains that stress your car, you get to feel the raw joy of going fast that reminds you of flying, and of course, you get the simply joy of feeling that you were meant to move.

Working at a startup is also weird – isn’t a job sort of about getting guaranteed money on a frequent basis? Sure, but only at a startup can you feel what it means to create something from nothing. A startup is a no excuse situation – no I’m-the-only-one-who-cares, no blaming teammates, no blaming corporate IT policy, no reason to be any slower than the fastest you can be. And in that situation you will learn and move faster than you thought possible.  And only in a startup can you really feel that you were meant to create.

Computer Science Terms from Lunch

Today the family tried to eat at Chick-Fil-A but they were so pegged that the drive-thru had wrapped around itself and was deadlocked. This meant that the person who just got their food couldn”t leave as the last car in line was blocking them.

We ended up going to a sit-down restaurant where we waited our turn in line. There were four people waiting ahead of us so we were the fifth people to be called in the queue. While waiting the hostess pushed the buzzers back onto the stack on the top.

While ordering our waiter read our order back to us before writing it down. The order was uncommitted until he wrote it down, which allowed my wife to change my order from Fried Buffalo Chicken Wings to Grilled Chicken Salad. If somebody had asked me what I ordered they would have gotten a dirty read (and, er, a dirty look).

While eating I keep switching my attention from eating to making sure our 17 month old wasn”t throwing food, this sort of thrashing between threads lowered my efficiency and made me focus on throughput, while my wife was able to eat hers with great speed as our four year old”s interrupt clock cycle is much lower. There was heavy resource contention around the Ketchup bottle as my wife and I discussed future plans to spawn new threads.

Questions to ask in an interview

The questions you ask in a job interview are important.

They reveal your level of experience, ability to form complete sentences, how much you were actually listening, passion level, and how seriously you take committing to an organization. While the goal of an interview is to get an offer, deciding what to do with that offer requires you to proactively seek information.

Feel free to ask questions even if you get to the end of the interview and you’ve managed to:

  • pronounce C# as “C-tic-tac-toe-board”
  • use airquotes every time you said “database”
  • completely bomb the technical interview (“Sir once again – you use the other end of the whiteboard marker to write”)
  • made a joke so lame everyone just looked down in shame (“Well, I think I would fit here because I-BM’d this morning! Get it?“)

If the interview is going well the questions you ask will arm you with critical information to do the right things in your first months on the job, and if the interview is going badly insightful questions can still be valuable – every interview is an inside scoop on how a particular company operates, what they value, and a chance to meet new people who are good at what they do – don’t miss it.

How to ask hard questions

The real questions you want to ask are at times too direct; they are the equivalent of asking a first date overly-intimate questions (“So what does you mom look like – does she have that eyes-too-close- together-Will-Ferrell thing going on too?”). While figuring out what you can get away with is an art and not a science, I’ll offer these random bits from my own experience:

  • You’ll regret the question that you don’t ask. It will haunt your dreams and bring shame to your family
  • It’s much easier to read people 1-on-1. If an interview is six people each talking to you for 15 minutes you can pick and choose who to ask based on what you read off of people – who seems to like telling stories from the “good old days”, who is an engineer and therefore stereotypically honest, who is going to give you the brochure answer? [Side note – some of the best information can be gathered from asking good questions of multiple people and comparing notes]
  • If you get invited to a lunch interview it’s a much better situation than in an office; the bar is lowered and people are more honest with their answers.

In the end the basic information you are trying to pull out of the interview process:

  1. Is the work-life balance what I’m looking for?
  2. Is the risk/reward situation what I’m looking for?
  3. Will I fit in, learn, and thrive?
  4. What are they trying to do?
  5. What happens when things go wrong?

Is the work-life balance what I’m looking for?

First things first, I’ve seen the following three things crop up time and time again in my own personal experience as possible “two weeks later” deal-breakers that just aren’t always known upfront.

  • What percentage of travel is really expected? When is my first business trip? Is being on the road something that can affect how well I do here?
  • What is the real, honest, number of hours I’m expected to work? (The smaller the company the more important this question is)
  • “If Ricardo likes it then surely it is awesome so no further questions”. Research a company even if your friend works there. Sure you and your buddy worked great together at the last place, but little did you know he loves traveling, getting yelled at, and loud smelly office conditions and you don’t.

Will I fit in, learn, and thrive?

Is this a place where you see yourself being successful after 90 days? Do you see yourself liking or tolerating the people you’d work with and for? Will they help you to succeed?

  • Tell me about your mentoring and training strategies.
  • How many new hires have started in the last year?
  • What success rate are you seeing with new hires?
  • How has the company changed since you started working here?
  • Tell me a story that you are tired of hearing about this place.
  • What are you guys really bad at (i.e. what do you not value)?
  • Does anybody hang out together after work?
  • Has anybody here worked together elsewhere before?

Is the risk/reward situation what I’m looking for?

Before you leave you need to know how stable the company is and where it is on the risk/reward scale between a two person startup and the IRS. This is sometimes as simple as looking around and seeing how many people are in the building and how tired they look, but most of the time you need to dig deeper to confirm your gut feel:

  • How big is this industry?
  • Who are your competitors?
  • Give me a two minute company overview.
  • What are you current business challenges? (Dig deeper if the answers are: grow, become more profitable – how?)
  • When was the last time you had to perform layoffs?

What are they trying to do?

  • Why are you hiring?
  • Walk me through the architecture of your system(s).
  • What do you hate the most, and what’s the things you guys are most proud of?
  • What was the hardest part to get right?
  • What part of the system do you wish you could build?
  • What are you plans to fix your current problems? [Do people agree on the problems and solutions, or are they still up for debate?]
  • What sort of technical debt do you have?

What happens when things go wrong?

How a company acts when things go badly reveals who the company really is. In an interview they may paint the best possible picture, but asking about recent project delays is a great way to get passion and real information out of the interviewers.

  • Talk to me about the last time a project was late (or cancelled), and what actions were performed.
  • When you miss a milestone, do you start to track it more or less?
  • What has been your biggest project failure in the last year?
  • When a project is late, do you ever add people, and is it always the same group of people?
  • How does the client / industry react?
  • When things are behind do you expect more or less individual (vs. group) accountability?
  • Do you perform post mortems?

Speaking of post-mortems, make sure to be able to effectively close the interview by asking them if there is any additional information they need from you in order to make a hiring decision. Then hopefully in a few days you’ll be asked “When can you start?” and positioned with the knowledge mined in the interview you’ll be able to answer.

Out of space error when moving tempdb

Quirk in SQL Server to do with sizing tempdb that I ran into today: SQL Server checks the current location when you move tempdb to see if there is enough space not the new location.

From: Configuring Database Files for Optimal Performance

The below script will move TempDB from its current location to a folder on the T drive. Change the drive letter and folder location to suit your system. The script only uses a 1gb file size because of an odd behavior in SQL Server that checks the current file location to see if there”s enough space – instead of checking the new file location. If the user specifies a 100gb TempDB data file on the T drive (which does have 110gb of free space), SQL Server checks the current location (C) for 100gb of free space. If that space doesn”t exist, the script will fail. Therefore, use a small 1gb file size first, then after SQL Server restarts, alter the file to be the full desired size.

How to complain

Let’s talk about one of the ills facing any group of people who are passionate about their work – ‘complaining about how bad things are’.  I’ll list some personal do’s and don’ts on effective complaining, and ways that organizations can help it not destroy morale.Every person that cares about their job complains.  How they complain, the level of detail of the complaint, and who they complain to are different for each person. People complaining badly in an organization that isn’t listening will destroy morale and lead to the sort of stagnation that destroys good organizations.

Personal complaining

Complaining badly is a waste of your time. So, the rules:

Complain only to someone that can help you change what you are complaining about.

All other complaining is effectively gossip.  While it’s certainly not a bad idea to discuss possible solutions with coworkers, if you find yourself complaining about the same thing at lunch with your coworkers over and over without discussing solutions pat yourself on the back – you are part of the problem.

Don’t passively complain

Passive complaining is most complaining – “The government spends too much of my money.”, “I wish I was taller, I wish I was a baller”.  In a professional setting people need solutions.  If you complain about something, don’t let the complaint block you from a solution, and view the opportunity to give feedback as a chance to list problems and present your own solutions.

Common complaint patterns

“John sucks.  They should just replace John”

If they replaced him with you, what would your first actions be?  Take some time to think this over – imagine yourself giving suggestions directly to John.  Suggest those actions instead of replacing John.

What this company needs is leadership.  We need vision from above to tell us what to do.

Be more specific – do you need instruction on how to do your job, what to work on, or is there a clear vision ambiguity? If not, then you are basically sending the message: “I can’t self-manage, and need further instructions from my bosses to do important things.” Again, if you were in their position, what vision would you cast – what are your solutions?

If we could just take three months and clean-everything-up, then we’d be able to get our head above water.

The only companies that are able to stop current work and ‘clean-up’ are dying companies.  I hate to break it to you, but The Great Big Rewrite Of All Our Software isn’t going to happen.  Solutions that allow companies to continue working while improving are much more important than greenfield ones.  It’s a great exercise to actually write out a business case for the “Take three months” plan and really think through the true business value.  Pick five simple things that can be done now and figure out a way to do them.

We need better communication.

Be more specific – are two groups working on the same thing?  Is only half the office showing up to company parties?  Is half of the company selling poison while the other sells antidote? [Actually, that’s not a bad idea]  This statement is just too general.

Joe should be fired.

Even in extreme cases of ethical violations, the open criticism of someone in this way should be avoided.  In the extreme case the complaint should be registered with the proper channels given the rules above.  Focus should remain on the specific behaviors and consequences of said behaviors rather than the person, and you should feel comfortable telling the person directly everything that you tell anyone else.

This place sucks.  This has to be the worst-run company in the world.

If you aren’t allowed to complain, all solutions are blocked, and there is nobody to actually complain to you can still list out your complaints anyway.  You learn from every single experience if you allow yourself to pay attention and you never know when having your thoughts organized will help.

Organizational complaint handling

Let’s talk about the suggestion box.  You know, the one that people put old cigarettes in, and the one where in the movies some brilliant idea shows up from a low-level worker, causing them to rise to sudden fame?  The suggestion box is a sign of certain organizational fear of open criticism, not its opposite.  How can an organization accept open complaining without causing fear and demoralizing its employees?

First, only accept criticism that is inbounds.

See rules above.

Be open to criticism as a means to change.

When an employee complains ask questions, take notes, pay attention.  Even if you have heard the problem and same solution from a different person, tried it, watched the failure of it, and wept openly at the awkward post-mortem (“Seriously guys who thought the beanie baby Christmas bonus was a good idea?”) continue to listen to the new perspective.  The person in front of you is extending themselves and is passionate about the issue and dismissing them in anyway is dangerous.

Be open to criticism that follows chain of command, and discourage other forms.

Start conversations with “Have you talked to John about this?”  [Make sure that somebody named John works for you].Politics can destroy open communication of this form.  How to handle ideas within an organization is a larger subject, but in order to counteract the natural problems with the chain-of-command-only communication pattern, setup chances for higher level management to communicate with people in the details enough to see problems.  Sometimes the first level of management (tech leads, managers) cannot see process issues as they still have one foot in each camp.  In addition a more experienced manager might recognize a solution that others miss.

View complaints and criticism as a sign of a healthy organization, and don’t take it personally.  In the same way that hard conversations are needed in real relationships if you don’t hear complaining it’s a bad sign.  Your employees are complaining, but they might be doing it to the wrong people, and in the wrong way.

Fixing problems Part 1: Introduction and Attitude Adjustment

As DBAs, software developers, Homo Sapiens, and lovers we have to solve problems.  There is a common misconception that support is for the more junior folks on a team and thus being good at it is a sign of “being a little baby”.  While support is a great way to learn a software ecosystem and organization and thus “grow” a junior person into a senior one a lot of problem-solving falls to the rock star programmers or wizard DBAs in most organizations.

Yet some of these people aren’t any good at it.  In fact, they are awful.  I’ve seen people that are very good at doing very hard technical things – creating something from nothing, thinking of all the things that could go wrong, refactoring and integrating a large subsystem, etc. – fail at simply fixing a problem with an existing system. You can take your rock star and send them off to fix a support issue and they will return with a confused look, eight hours of wasted effort, a missing finger, and an STD.  Working with some truly gifted problem solvers I think that there are some differences in Attitude, Practices, and Skills that separate the junior and senior problem solvers.  Let’s start with Attitude.

First, there IS a problem

Never deny that there is a problem.  If someone is at your desk, on the phone, flooding your email with red exclamation points, or outside your house knocking on the window there is clearly a problem.  The problem might be that they don’t understand something and the problem might not be your fault, but the issue should be treated with respect as a real problem if they took the time to contact you.  Don’t deny it or argue.  Why would you deny it anyway, because you see support as negative.

Don’t see support as a negative shameful thing or a junior task

As long as things keep changing, there will always be problems.  (This next part is hard to put down in writing) If you aren’t writing bugs you aren’t writing software.  If you aren’t changing systems you aren’t working.  A support issue is not an insult, a bug is not a breakup.  Yes, you should make sure that you don’t write infinite loops, and yes you should make sure that you test the latest SQL Server upgrade before you install it in production at 10 am on the last day of the month.  But in most organizations there is a constant to and fro of creating new things and then fixing issues that crop up with them, so don’t pretend as if a problem is an anomaly.  True root cause and issue prevention are topics for another post, but don’t act surprised that software systems don’t always work as expected.

Confidence

In my home office I have a fortune cookie taped to one of my monitors that says: “You have the ability to analyze and solve any problem”, and over time I have started to believe it (by looking at it 27 times a minute).  The fact is that most people that are good at support are good because they believe that given enough time they could fix any issue.  ANY issue.  Given 20 years and enough coffee they could learn C/C++, reverse engineer SQL Server, learn about cross-platform multi-threading, and fix that bug.

The ability to not freak out and lose it

Something is broken but don’t be scared (its just moving electrons for the highest bidder).  The fact that someone is at your desk and not someone else’s is a good thing, don’t panic and freak out and yell things that you’ll regret later (Aside: yelling is always regretted – only thing I’ve not regretted yelling “OMG ITs MILEY”). Don’t blame people or come off as condescending; assume that they are your desk because they know you can fix it, not because they think you caused it.  Figuring out root cause and a long-term solution are separate things; as are fixing and blaming.  You are in charge of fixing for now, so just focus on that.  Besides, over time a calm person is going to be relied upon more while those freakout will only end up on reality shows.

Next post – Practices that improve your ability to fix complex technical issues.

Exception (Mis)Handling

Exception handling was originally created to try to give developers a way to separate out error handling so that it wouldn’t clutter up and distract from the core functionality trying to be accomplished. When done well, exception handling can provide a clean way to instrument and separate truly exceptional conditions from the core flow of your methods as well as a way to prevent nasty crashes and untraceable bugs. When done badly, exception handling can be misused for decision making, masking errors and bugs, and distracting developers from doing real work.Go read these when you have time, and you have time if you are reading this – admit it.Chapter 19 of CLR via C# and Guidelines: Exception Handling.There is not a lot that can be said that isn’t said here about .NET exception handling that isn’t in the resources above, but I’ve picked up on a common anti-pattern that stems from a misunderstanding of what exception handling is. I call it the “safety-net catch”, and it goes a little something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
public bool Import()
{
	bool isSuccess = false;
   	try    
	{     
		if (SomethingIsWrong())         
			throw new ApplicationException("Start panic sequence now.");
	        bSuccess = DoSomething(WithThis, AndThis);     
		foreach (string databaseThing in theDatabase)        
		{            
			if (bSuccess) bSuccess = DoSomethingUseful(databaseThing);
	        }    
	}    
	catch (Exception ex)    
	{        
		PublishException(ex);        
		isSuccess = false;    
	}    
	return isSuccess;
}

In this example, DoSomethingUseful, DoSomething, and most likely PublishException all have this same pattern of a ‘catch all’ at the end. We aren’t getting much benefit from this style of exception handling as it merely serves to make sure that this method always returns bSuccess so that execution should continue with any failure.I think it would be easier on everyone if this code was changed to simply not handle exceptions that it can’t actually handle. The top level threads of your application should have a “catch all” that publishes the exception, and then displays a nice message for the user in the case of a web application. So if you want to fail, just let it fail if you can’t recover from it. The contract of the method Import above doesn’t say that it doesn’t throw exceptions – if it can’t do its job it should throw or allow an exception to bubble up.There are cases where you don’t want the code to break out in the case of any exception, but these are rare. In the case of row level handling where we are parsing a file and you don’t want row to ruin the whole file you can simply move exception handling down to that import piece. In this case non- exception based mechanisms can be used or you can throw a custom exception type or an exception with well understood semantics like InvalidOperationException or ArgumentException.Please do:

  • Throw an exception in a method if you can’t do the method’s job. If you don’t know what the method should do because it does 23 things, 18 of which can happen if it isn’t given a valid value for AccountNumber, refactor until you have 23 methods, one of which throws InvalidArgumentException when it gets an invalid AccountNumber.
  • Only do a ‘catch’ if you are going to handle it (maybe the syntax should say handle). Just publishing or setting a variable doesn’t count in the pattern above.
  • If you for some reason catch and rethrow (like if you want to add to the exception object), do a throw and not a throw origEx to keep the call stack.
  • Remember that your most common ‘exceptional condition’ might be a database timeout or other database exception (System.Data.SqlClient.SqlException) and is not an ApplicationException but most likely can’t be recovered from anyway.

Please don’t do this:

1
2
3
4
5
6
7
8
9
10
try
{
	bc.Save(o); 
	DB.UpdateProcessFlag(conn, Util.GetInt32(dr, "Key"), 1);}
	catch
	{ 
		//Error on save, so mark this one as errored. 
		DB.UpdateProcessFlag(conn, Util.GetInt32(dr, "Key"), -1);
	}
}

There are many reasons that this .Save() could fail, and a SqlException or (ObjectReferenceException on dr) are going to be badly mishandled here. If you find yourself feeling like you need to use the try/catch mechanism for this sort of stuff perhaps take a step back and think about how to handle it in terms of design.And please don’t do this unless you like hearing the sound of me dying inside. (Use the TryParse pattern instead)

1
2
try { o.Something = -aNumba; }catch { o.Something = 0; }
try { amt = Util.GetDecimal(dr, "FieldName"); } catch { amt = 0; }if (amt == 0) return;

Key considerations for your next development job

As a developer, your basic job is to create things. Since the world needs software in every industry you might think that one is the same as the next, and you’d be shamefully wrong. Outside of the obvious questions you should ask yourself when looking for your next gig – how sharp are the coworkers, how good is the tech, how hard are the problems, how good are the tools – you should also ask: “Who are we working for?” It turns out that who you produce for can more directly impact how much you like your job than you might think.

Internal* vs. External client

Internal clients don’t matter as much as external ones. Yeah I said it, wanna fight about it? Working for a bank on their internal accounting software is not the same as working on banking software that is sold to banks. Period, close bracket, EOF. The reason for this is that one is a profit center with real financial pressure, and the other is a cost center, with pressure to simply exist cheaply. A profit center has direct competitors that you sometimes have to react to, but a cost center rarely implements new product due to hearing that an internal customer at another company is happy.  The internal vs. external switch plays itself out in multiple subtle ways:

Rate of Change

A profit center tries multiple things, watches competitors to match features, explores new lines of business, etc. A cost center does not take much risk, and thus is more setup for small improvement or the support of company growth.

Rate of spending

Another way this plays out is in lack of budget flexibility – since a cost center is under pressure to lower costs their budgets are smaller and less innovative. The type of managers that run these organizations are the special type of demon that is good at finding ways to save money – like on hardware or crappy coffee.

Rate of Respect

In a profit center the business leaders interact with the technical leaders and producers enough that they begin to understand their importance. Over time a mutual respect grows and is a healthy team behavior. In a cost center at times the cost center is in a servant position and the IT functions are not held in the same level of respect.

Producer vs. Maintainer

There is another subtle difference in “software developers” at times that can play out in affecting you position. Some people build tools and processes and some people build deliverable product. In the software realm the tools can include continuous integration modules, deployment tools, operational helper tools, code generators, etc. Product includes things that directly sell outside your organization. If you are a developer working on the toolset *primarily* you are secondary to those working on the end product – you are in effect serving an internal customer.

Deadline driven vs Shame-driven

When we interview someone we always ask how the end game of projects work – “Do you work off of deadlines?”, “Who sets these deadlines?”. Many internal customers have false deadlines because there is no competition – are they going to go use another internal accounting department? The same fire does not always exist in those working in internally-facing companies. Just because your coworkers are smart this doesn’t mean they have any hustle.

Industry and Economy

The industry that you are building solutions for can matter because the level of tolerated innovation differs across industries and product categories. My first full-time programming job was working on an audio-dispatching call center product that was sold to air traffic controllers and 911 call centers. The sales cycle for this product was very different than other industries – if a client saw a demo and a single thing went wrong they would typically say: “We will reevaluate changing our product in 5 years, get back to us then”. This is obviously different than the change cycle for a web-based project management tool that might receive changes every two weeks.

Customer distance

How “close” you are to the customer matters as well whether that customer is internal or external. While only some developers are interested in directly speaking to customers, the dev team’s distance to the customer can affect whether or not what they are building matters. As a producer you should care very deeply about whether you are building the right thing.

* Internal clients means full-time – if you are working on a project for an internal client the effects are more minor. Most of what is mentioned above is when you wake up everyday to a world of internal client demands only.

Complexity

In software development there are three levels of complexity that are in play.  You should focus on the primary complexity: taking a complex business problem within its native domain and designing a technical solution.

Examples of primary complexity issues are designing a strategy-based plugin architecture for mortgage calculations, designing a star schema to later use for predictive analysis of snack cart Skittle consumption on college campuses (spoiler alert: finals week and wild berry are a solid marriage).  Secondary complexity in software include items that the folks in the domain don’t understand but the technical folks need to do their work quickly. The QA team, the fact that you need to upgrade Visual Studio every two years in a Microsoft shop, your check-in policy and coding standards – these are typical examples of secondary complexity.  Tertiary complexity are items that the domain folks and the technical folks don’t fully understand. The fact that Steve and Ralph freaking hate each other, the morale impact of low raises on a team, the fact that communication can kill a team over a certain size, poor meeting practice, etc. are all examples of tertiary complexity. This type of complexity is colloquially referred to as “bullshit” (in some provinces “horseshit” or more rarely “dogshit”)

Consequences

The levels affect each other predictably. If the problems of one level are not solved, they affect levels above them. If half of your team are French and the other half hate French people, that’s a tertiary complexity problem that will lead to awkward code reviews, poor rework throughput and affect your ability to ship software that solves domain problems. If your QA and UAT databases “timeout” more than a cranky 2 year old, you can’t ship software.

Talent Effect

By “effect” I don’t mean slow down only. If you have too many secondary and tertiary problems, the people that work in the primary domain will grow frustrated. In software development these are creators – developers and those that feed them information (BAs, User Interface folks, reedit). This is a dangerous situation. If you hire a talented pizza chef but all the pizza shows up two hours late (or not at all) you have secondary problems (failed delivery channel – broken down Civic) or tertiary problems (your delivery guy likes pot more than tips) eventually the chef will feel that their work doesn’t matter and leave.

Management

If you make the move from developer to manager, you are effectively saying that your team will handle the primary work while you work on secondary work (processes) with their input, but try as much as possible to shield the tertiary problems (bullshit) from them. Where this gets more complex is when tertiary complexity is high in an organization. The most striking example of tertiary complexity in some organizations are middle managers jockeying for promotion position – this activity does not improve processes or help with production, but creates a situation in which these things are harder.The balance of tertiary vs. secondary complexity is indicative of where a company is and whether it can heal its key problems. If the tertiary complexity is low, then there is enough management capacity to solve secondary issues and allow producers to create great things.

Triggers Part 3: FAQ and FOP Continued

This is part of a 3 part epic mini-series on triggers, view Part 1 and Part 2.

How many triggers should you have per table?

Ideally zero.  If you have any then there should be one.  There is no guarantee on the ordering of trigger firings, they normally fire based on their age – newly-added triggers fire last.  So if you have two triggers that both run on update, you could get into a recursion situation.  If you do have more than one, you have to set the trigger order via a call to sp_settriggerorder to avoid this recursion, or rewrite the trigger

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
-- Test trigger recursion
IF object_id('TestTriggerRecursion', N'U') > 0
	DROP TABLE TestTriggerRecursion
IF object_id('tr_TestTriggerRecursionUpdatedDate', N'TR') > 0
	DROP TRIGGER dbo.tr_TestTriggerRecursionUpdatedDate
IF object_id('tr_TestTriggerRecursionUpdatedBy', N'TR') > 0
	DROP TRIGGER dbo.tr_TestTriggerRecursionUpdatedBy
GO
 
CREATE TABLE TestTriggerRecursion
(
	KeyID INT PRIMARY KEY clustered,
	Payload VARCHAR(MAX) NOT NULL,
	LastUpdatedDate datetime NULL,
	LastUpdatedBy sysname NULL
)
GO
 
CREATE TRIGGER dbo.tr_TestTriggerRecursionUpdatedDate ON dbo.TestTriggerRecursion
FOR INSERT, UPDATE
AS
	print 'tr_TestTriggerRecursionUpdatedDate ran'
	UPDATE t SET LastUpdatedDate = getdate()
	FROM INSERTED i
	JOIN TestTriggerRecursion t ON t.KeyID = i.KeyID
GO
 
CREATE TRIGGER dbo.tr_TestTriggerRecursionMaintainAuditField ON dbo.TestTriggerRecursion
FOR INSERT, UPDATE
AS
	print 'tr_TestTriggerRecursionUpdatedBy ran'
	UPDATE t SET LastUpdatedBy = SUSER_NAME()
	FROM INSERTED i
	JOIN TestTriggerRecursion t ON t.KeyID = i.KeyID
GO
 
-- delete from TestTriggerRecursion
INSERT INTO TestTriggerRecursion VALUES (1, 'test1', NULL, NULL)
INSERT INTO TestTriggerRecursion VALUES (2, 'test2', NULL, NULL)
INSERT INTO TestTriggerRecursion VALUES (3, 'test3', NULL, NULL)
SELECT * FROM TestTriggerRecursion
 
-- fails with
/*(0 row(s) affected)Msg 217, Level 16, State 1, Procedure tr_TestTriggerRecursionUpdatedBy, Line 6Maximum stored procedure, function, trigger, or view nesting level exceeded (limit 32).*/
 
-- exec sp_settriggerorder @triggername = 'tr_TestTriggerRecursionUpdatedBy', @order='First'

Do triggers try to run ON UPDATE even if now rows have been affected?

Yes, which is why you should always bail if no rows are affected like so: IF @@ROWCOUNT = 0 RETURN As you see below, your entire trigger will fire even on a failed update.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
-- Test trigger update behavior with zero rows
IF OBJECT_ID('TestTriggerUpdateBehavior', N'U') > 0
	DROP TABLE TestTriggerUpdateBehavior
IF OBJECT_ID('tr_TestTriggerUpdateBehaviorUpdatedDate', N'TR') > 0
	DROP TRIGGER dbo.tr_TestTriggerUpdateBehaviorUpdatedDate 
 
CREATE TABLE TestTriggerUpdateBehavior
(
	KeyID INT PRIMARY KEY CLUSTERED,
	Payload VARCHAR(MAX) NOT NULL,
	LastUpdatedDate DATETIME NULL,
	LastUpdatedBy SYSNAME NULL
)
GO 
 
CREATE TRIGGER dbo.tr_TestTriggerUpdateBehaviorUpdatedDate ON dbo.TestTriggerUpdateBehavior
FOR UPDATE
AS
	PRINT 'tr_TestTriggerUpdateBehaviorUpdatedDate ran'
	UPDATE t SET LastUpdatedDate = GETDATE()
	FROM INSERTED i
	JOIN TestTriggerUpdateBehavior t ON t.KeyID = i.KeyID
GO 
 
INSERT INTO TestTriggerUpdateBehavior VALUES (1, 'test1', NULL, NULL)
UPDATE TestTriggerUpdateBehavior SET Payload = 'testImpossibleUpdate'
WHERE 1 = 2 
 
SELECT * FROM TestTriggerUpdateBehavior

So, when should you use triggers?
You should use triggers:

  • When you have a clear understanding of how they work
  • You have no other option
  • You have performance tested your code thoroughly
  • You have informed your storage folks, DBA folks, and your mother
  • You have prayed about it

In all honesty I”ve only seen a few clean uses for triggers:

  • audit trigger: Audit mechanism for straight up insert/delete/update calls.  You have tableA, and you want to log all changes to auditTableA – a “copy trigger” does this quite well.
  • refactor trigger: A temporary bridge between two phases of a database refactor project.  You are migrating data from schema A to schema B, but that last pesty bit of code hasn”t been changed.  This release changed 80% of the code and put in a trigger to maintain the data or log to new tables for later testing the remaining 20%.  The discipline required to push through the 20% and remove the trigger is rare, so this is sometimes dangerous.
  • trap trigger: In a crisis, log where updates are coming from to a specific table.  Remove trigger quickly thereafter.
  • evil trigger: A trigger created for evil.

Where can I find more information about triggers?

Books online: Triggers