default

January 18

Stop SOPA!!!

jim leonardo on 1/18/2012 11:29 AM

Needless to say, sign me up with those who want to stop the SOPA and PIPA bills. Both bills are heavy fisted attempts by Big Content to protect their intellectual property without any concern for the ramifications.

Even if you strongly believe piracy is wrong (and I do), Congress's attempts to legislate technology to protect piracy have been ineffective at best and unconstitutional at worst. How we justify 100,000 USD per violation of copyright for a 0.99USD song under "no cruel and unsual punishment" is beyond me. How SOPA and PIPA won't produce both a chilling effect on free speech is also beyond.

More importantly, both bills CLEARLY violate the concept of Due Process. If you think this is minor, you should investigate some of the cases that have been brought under the old DMCA (Digital Millenium Copyright Act). Just the Righthaven fiasco should be a cautionary tale(and Righthaven at least had some justification for pursuing at least some of the cases, they just did it wrong). Even before DMCA, we've seen small websites (and bulletin boards before that) bullied by content owners into shutting off fan created content over copyright and other intellectual property concerns even though they clearly would fall under Fair Use. Software applications have been gutted to comply with a content owners perception that it could potentially be used for pirating content. Now, some people think that prior to trial and with little review of the merits of a case, a site should be effectively shut down.

The sad part is that it's completely ineffective from a technical perspective. All that will happen is that the pirates will move on to other means of contacting one another whilst the legitimate Fair Users or accidental infringers get clobbered by content owners who have their nose out of joint. As the content owners won't be able to stop it with their current laws, they will pursue an even more draconian set of laws in another 5-10 years time.

But: think how the copyright trolls will use this! Instead of their quasi-legal extortion letter saying "pay us or we sue", it will say "pay us or we'll have your site taken off the internet".

We've seen that there have been reductions in the Due Process violating provisions due to all of the pushback, but still, the idea that I would have to monitor every site that I link to in order to determine if it hosts violating content is nuts. What it means is that I will not be able to cite any references in any blog post for fear of the site I'm referring having some smattering of violating content. I not only need to check it before hand, but after as well. What does this mean for blogger, wordpress, tumblr, etc? All blog hosting websites would basically be put on notice that they need to monitor every single site that one of their users posts a link to. While we're at it, why not pass a law making restaurant guests liable for dining at a restaurant that buys its tomatoes from a distributor that buys from a farm that uses illegal immigrant labor. It's about the same thing.

Anyway, there's others who have articulated this much more clearly than I have. I'd link to them or mention them, but that would be a bad idea in a SOPA world.

November 17

Migrating a bootable VHD to Fusion in Mac

jim leonardo on 11/17/2011 11:29 AM

I just got a new Macbook Pro for work use and wanted to move a windows image which I used for client work. The image was a bootable VHD file (like this) and that presented a couple of issues for a simple move.
First, the bootable VHD is not the same as the vhd file as used for Microsoft's virtual machine technologies (whether Virtual PC or HyperV). VHD just means "virtual hard disk", but that does not mean the machine image contained within that disk is built for being run as a guest inside another machine. The bootable VHD machine is configured to be run as it's own machine, it's just running via a packaged up file sitting inside of another hard drive partition. The result is you cannot use VMWare Fusion's Import feature to convert this disk from a VHD to a VMWare image.
What you could do is use Fusion's "Migrate Your PC" option to migrate the machine, with a bootable VHD there may be a gotcha. The gotcha is that the Migration option will automatically try to import the host (naked) hard drive in addition to the VHD image if it's connected to your VHD machine. Your VHD image will not be big enough to hold it's parent hard drive! I suppose you could detach the drive, but doing so would require to alter the original machine. That doesn't seem such a good idea if you want to go back to that machine, even to keep as a back up.
What you can do is run the VMWare Standalone Converter from within the Windows machine you're trying to move over. This will allow you to tell it to not try to package up any drives you don't want. It also will let you put the packages elsewhere, not force you to fit it inside of the VHD's space. This creates the virtual machine for you, ready to run in just about any of VMWare's products. You can then simply copy the files over to your mac and then launch it in fusion.

If you did not take the drive with your boot launcher info, you will need your windows CDs (or isos if it's from MSDN) to run a quick Windows Repair to fix the boot info inside the new VM. It's pretty painless, just follow the instructions.
This is a new machine, you may need to run windows or office activation again depending on original activation requirements. OEM installs of windows will probably give you grief, but it's unlikely you were running that from a VHD to begin with.

Overall, once I figured out the need to run the stand alone converter, this was a pretty easy process. But, it did take a bit of noodling and googling to figure out.

September 07

A new little toy out there... Wordle

jim leonardo on 9/7/2011 11:27 AM

A new little toy out there... Wordle. Feed it some text (or a blog) and it creates a word cloud for you.

Apparently I want to Just Get Work Going People.

Clicking through may cause issues in some browsers, seems to work fine in Opera though.

August 05

A dice tool for 7th Sea.

jim leonardo on 8/5/2011 11:27 AM

I've put the first version of dice roller for 7th Sea up on my gaming related site, smellyoldgamer.com. As of right now, smellyoldgamer is still just an alias for jimleo.com, so you can just use jimleo.com as the root too. The full address to the dice roller is http://www.smellyoldgamer.com/sog/7thSea.html.

I've built it to play nice with keyboard, mouse, and touch displays, so please check out. Notes on keyboard use are on the bottom.

I kept the page completely self contained, so you can just save it and use it offline. It doesn't save anything, post it back to the server, or anything like that. It's completely intended to be locally run because everywhere your device goes may not have an internet connection. Simply save it as an html file and load it up in your browser for later.

A possible enhancement over today's version would be to keep track of the last x number of rolls so you wouldn't need to reenter them.

I've tested it using the major browsers, and as usual, IE is the only one annoying the snot out of me. There's a bit of page "dancing" going on that causes one of the sections to move a bit unexpectedly that I'm still working on. Firefox seemed to be the pickiest and managed to find bugs in markup that the other browsers compensated for.

March 17

A Few Very Useful Tools–Mar 2011

jim leonardo on 3/17/2011 11:26 AM

Wow, I missed a YEAR’s worth of A Few Very Useful Tools. You are now entitled to your subscription fee back. Given I pay most of you to read this, you owe me money.
[Warning, philosophy ahead]
Seriously though, a year ago I was running out of things to post, so it was time to let it rest for a little while. There’s really only so much you can use well enough to get a good idea of whether it’s worth using. Lots of reviews represent 5-10 hours of use of a product (if that) and often come down to what I euphemistically call “counting cup holders”. Feature count doesn’t matter if those features aren’t well implemented. I would much rather use a bunch of small utilities that did one thing well than one mega-app that did it all but not very well. Granted, there are times where the whole is greater than the sum of its parts, but that requires true integration.
Without further ado, some gems from the last year.

Git – Source control the way it should be (I believe I heard that from Linus Torvalds). Most of all, branching and merging are so easy you stop thinking about it. You also can ship changes directly to someone else without creating ugly patches/shelves and without sending it through the central server. Why? Git is distributed from day one. A central repository is a decision you make as a project team, not requirement of the software. You commit to a local repository, then merge that repository with a central or with your peers. Git is much more a command line tool, graphic tools are lacking. They’re available, just lacking. Mercurial is a close second and has better graphical tool support if you’re on windows, but Git’s the king right now in terms of functionality.
Resharper – Ok, it was on my Dishonorable Mentions list a year or so ago. That said, I think they’ve improved it, but I’ve also found that on a “real” machine it rocks. That said, a “real” machine is a Windows 7 box with min 4GB ram and a beefy CPU. An SSD and 6GB of ram makes for a nice experience with “R#”. This configuration will neuter my main complaint from a year ago, which was performance. The refactoring tools, the test running tools, and the various navigation tools make for a completely different world of development.
Remote Desktop Connection– Silly as it sounds, this little tool is just what the doctor ordered if you have to work with multiple machines. I’ve stopped trying to manage multiple machines with a Keyboard-Video-Mouse switch. Using Remote Desktop, you can just open a window on one machine into another machine. It is not screen sharing. It let’s you open another desktop as if you were another user logged into the system (note number of users is limited). As far as I know, there’s nothing like this for mac, although there is for Linux. XRDP let’s you pull this off in a Linux world and share between the two as well! Mac doesn’t get to play, sorry.
Moq – Unit testing is the sign of a professional developer. It’s also the sign of a sane developer. Moq is a mocking framework that makes unit testing in C# absolute cake. I would favor this over any pre-VS2008 frameworks as it makes extensive use of lambdas to make mocking out dependent classes absolutely easy. Of all the things I can say about it, the best one is probably this: just about anything I’ve thought would be hard to mock has already had a way to do it in Moq. You really don’t need much more than the Quick Start page for this, trust me.

Well, that’s it. Four stellar tools in one year, only two of which I’d say are new-to-me in the year. Back to the grind…

February 02

DateTime is Dead, Long Live DateTimeOffset!

jim leonardo on 2/2/2011 11:25 AM

Ok, maybe I’m late to the presses with this one, but given that no one I’ve talked to recently knows about this, I think it’s worthy of more press…

The .Net DateTime structure is a nice handy structure that we basically can’t escape. Funny thing about users, they have a tendency to want to know when they did something, when they should do something, and how far overdue it is when they didn’t do it.

The problem with .Net’s DateTime is the lack of time zone information in the structure. Believe it or not. Yeah, really.

In .Net 1, we had no indicator of what time zone it was for. In .Net 2, they introduced the notion of “kind”, but that was limited to indicating “Local”, “UTC” (i.e. GMT or Zulu), and “Unspecified”. Even with .Net 2, this led to most of us just taking the convention of immediately converting everything to UTC. Unfortunately, that led to more than one bug when someone forgot to convert or double converted (or triple or quadruple or…).

MS’s answer was always that DateTime was a 64 bit structure that they used something like 62 bits for it. They were able to add in the “kind” by using the extra 2 bits, but that only allows for 4 possible values if you recall your bitwise arithmetic (00, 01, 10, 11). Hence, Local(10), UTC(01), Unspecified (00). Maybe you’re thinking that “we only need 24”, but that doesn’t account for India or Newfoundland (something + .5 hours offset) and leaves you scratching your head at what to do with daylight savings time.

Worse, SQL Server’s DateTime only stored the date and time with no idea on time zone. However, there at least we could just say “server time” and be done with it.

This is manageable and usually not too much of a problem in self contained systems. Where it becomes painful is in situations where you need to serialize data and send it somewhere else. Then you really need to worry about that time zone and it gets really goofy when you’re working with datasets. Once I get over the psychological trauma of dealing with datasets and time zones, I might be able to blog about that.

Since VS2008/.Net 3.5 and SQL 2008, we’ve had a new and vastly improved way of handling this. This is called DateTimeOffset. Unfortunately, the name seems to throw everyone off. DateTimeOffset does not just store the offset (I.e. –5 from UTC), it stores the entire date time (2/2/2011 9:56:12 PM -05:00). Now,the interesting part is that this takes a TimeSpan, so you really can make this as flexible as you want it. TimeZoneInfo helps you with finding timezones and applying the various rules such as daylight savings time. Both just plain work with SQL server.

Short of it is that I plan on forgetting all about DateTime and only using DateTimeOffset from now on. Now I’ll always know what time zone I meant and not have to figure out what rules I need to apply to get it into local time.

Once I figure out the gotchas, I’ll relay those too. Winking smile

See:
http://msdn.microsoft.com/en-us/library/system.datetimeoffset.aspx
http://msdn.microsoft.com/en-us/library/system.timezoneinfo.aspx
http://msdn.microsoft.com/en-us/library/system.timezone.aspx

March 08

Rescuing yourself...

jim leonardo on 3/8/2010 11:22 AM

Lately I've been blogging about the dimensions software project failure and what kinds of personalities you see in projects that have failed or are about to fail. Now, I'm the kind of person that says you need to find the negative to fix the negative, so I'm not a pessimist as much as a tinker. Everything we do can be done better than we're doing it now. That doesn't mean we should try to be totally perfect, just that we should be better than last time.

I feel that I need to offer up a little disclaimer before we go any further: Discussing project health and management is very difficult to do with some co workers and bosses. Please go careful. Charging ahead convinced that you are right is likely to only end badly. Open ears lead to persuasiveness. Closed ears lead to a pink slip. Reading body language is just as important.

The Roads We're On

I've seen projects that were successful all the way through. Any hiccups were just bumps on the road and even freshly laid pavement's going to have a bump or two. Then there's driving in NYC... you manage to get through, but the going is rough and suicidal taxi drivers seem like they are trying to ram you into the East River. These are those projects that seem to succeed in spite of themselves. Other times, you feel like Roscoe P Coltrain in a Dukes of Hazzard episode... you just know you're not going to make the jump over the creek yet you try because if you don't Boss Hogg is going to tear into you. These are the projects that are going and going, but everyone knows what's coming and yet people keep pushing for whatever reason until finally the plug is pulled and you're all scrambling to not be the one who gets to be one of the ones canned.

If life were simple, you'd know which type of project you were on from the get go. If it was all going well, you'd happily keep with it til the end and pat yourself on the back. If it was rough, but survivable, you might stick it through figuring you might get some kudos for pulling it off. In reality, projects rarely stay on the same footing, but you rarely see it trend upwards. Either a project is good all the way through, or it just gets steadily worse. Call it project entropy... unless something else influences it, it tends to degrade over time. However, if we understand what's going on and keep a higher level view of what's going on, we might be able to act to correct it.

What goes wrong?

Phaser Burns

This downhill trend of projects is something I 've seen tend to happen in phased projects. During phase 1, you're primarily focused on getting something done. Proving the basic premise, getting a rough architecture mapped out and executing on the first set of use cases/user stories/whatever you call it. This tends to be busy, but the atmosphere isn't uptight and things tend to go ok. Phase 2 is where trouble begins. Brooks calls it "the Second System Effect" but I think there's maybe more to it. Please go read Mythical Man Month. It may not be spot on all the time, but the observations are still often valid 30+ years later. His main point in Second System Effect, and I pretty much agree with it, is that everything that was highly desirable but not mandatory got cut out for version/phase 1. Unsurprisingly enough, phase 1 goes reasonably well and delivers on time. In version/phase 2, there's a tendency to pull in far more than can be accomplished and failure mode starts to set in. What was once a great example of how to develop software now becomes a boondoggle.

A few things contribute to this. Obviously, biting off more than you can chew is a huge contributor. But how exactly does that happen? We're all intelligent adults and most of us aren't too terribly masochistic. Thus, we can cut out the overt psychological oddities and focus on more subtle issues.

The Inevitable Gold Plating - Gold plating is of course a major curse in software development in general. It tends to be at its worst in phase 2. All the 'nifty stuff we didn't have time for' now is in scope and the laser like focus you had in phase 1 diffuses as every 'brilliant' idea someone had starts to get to be a must have. In reality, business value's not there, but resume stacking is.

Good Old 80/20 - Quite simply, there's a subtle mental push that everyone feels even coming from within themselves. In phase 1, where your focus was on what was an absolute must have, you probably were cutting edge cases left and right. This leaves phase 2 with an incredible backlog of 80/20 ruled features. You know the 80/20 rule: 80% of the features are done with 20% of the work and 20% of the features take 80% of the work. Yeah, its just nice round numbers, but it communicates the point... edge cases often consume far more time than the main features yielding a low return on time invested. Contributing to this is that subtle mental push I referred to. Let's call it the Feature Counting Effect. This is brutally simple and goes like this: "In phase 1, we were able to deliver 20 features. Therefore, in phase 2, accounting for variability, we can plan on at least 15 features." Alternately, you'll think "you've learned some lessons" and instead plan for 25 or 30.

Go read that bit about the 80/20 rule again... if that's true you should only be planning FOUR features! After all, you've 80/20 ruled out some hard, but still pretty important, features. Go ahead, tell that to the boss and let me know how it works out... Of course you'll have to use your personal email address for me to reply to you because after they can your rear end, I won't be able to use your corporate email.

Cheaters Get Caught - Little known... err... widely known fact: No software project gets done without a compromise or three hundred along the way. Ok, calling it cheating is harsh, but the reality is that very often, to get past today, you don't have time to polish everything up to the point that you're really happy with it and you know it's going to come round to bite you. Sometimes, this turns out to be Gold Plating and YAGNI. But, almost always, there turns out to be a few features that have turned into dead ends in terms of being able to be extended to accomodate new functionality and you'll have to re-work them. Ok, this often occurs irrespective of what phase of development you are in. However, it's more likely to become an issue when new requirements and scope come into play. At any rate, we should always be willing to refactor to improve the internal quality of our code. Sometimes, it's just a lot more expensive and a lot less optional than others. All of this adds to the investment you need to make in phase 2... the larger the amount of code you have and the more users of the system, the more it costs to make any changes. (One of the prime drivers for refactoring in the first place... clean your junk up).

Your Boss's Boss (Political Commitments) - Let's face it, external commitments get made. Sometimes, to get agreement to go ahead with phase 1, the Boss needs to tell the Big Boss that he's going to get a feature in phase 2. Sometimes it's not Big Boss, but the Other Sponsor, etc. Anyway, there's a reason sometimes that 'not quite mandatory, but we better get it' feature is pushed out to phase 2: it's hard! Or, it's really not a good return on investment. The only value is because someone with a lot of weight to throw around wants it and this is going to drain time away from other looked for features in phase 2. At any rate, the premise here is that sometimes phase 2 ends up 'oversold' before phase 1 is hardly under way. Said another way: phase 2 is already doomed while phase 1 is still a long way from even being code complete let alone tested and shipped. This is all because the idea that "we'll deliver it in phase 2" was the convenient answer to anything that couldn't get put into phase 1 leading to significant overselling. Been there, done that, now you're reading my novel.

Piling on the "Help" - When phase 1 is seen as primarily exploratory, the temptation of the Bosses to throw more people into the mix can become irresistible. Surely, we can accelerate development by throwing more bodies at the keyboards, right? This what I call "World War I" methodology. It is this sort of thinking: if you have to cross an empty field with machine guns at the other end, then obviously the best way is to get as many people to cross the field at once hoping there will be too many targets. Needless to say, it didn't work in WWI and it doesn't work in software development. Sure, there are times to use more people, but I've not seen too many cases where growing the team rapidly works once the project is in flight.

All analogies aside, what happens is that in phase 1, you've usually got a small team where communication, knowledge sharing, group design, etc all happen naturally. During this time, people settle into roles and everyone knows who knows what. The team gets to know one another and start to achieve some level of that nirvana of not needing to talk to one another to know what they will do as a group next (I did say some level). Now, double the team. This is putting new wine in an old wine skin... it will not work. All those connections that have been forged over time will be broken.

Furthermore, I don't care how much time you waste on documentation. There will always be bits of the system that will primarily be stuck in the head or heads of the team. I might be able to document the system I built to the point that when there is a bug, some random person can read the document, figure out where the bug is and fix it. But, take the same random person and ask them to add a new feature. Sure, they can do it, but they will do it by integrating their brand new code into the system. They will not know the 'tweak this, extend that, refactor over there' that will get it done in the context of the system in a fraction of the time. Sure, over time, they will get it. I'm just saying it's foolish to think they'll get there quickly.

It Was All Bad

Sometimes, it's not a downward spiral. It's not even a slippery slope. It always is bad and that's all there is to it. You've probably already decided that the reasons to tough it out exceed the reasons to bail out and are going to ride this horse for what it's worth. But, what's going on here? We're all intelligent adults and most of us aren't too terribly masochistic... wait, I'm repeating myself.

Anyway, usually, you'll see the same factors creep in while you're still in phase 1 or whatever you call it that you see in phase 2+ as factors that doom you. Certainly, over commitment is a problem anytime you run into as is Gold Plating and other activities that don't add value. So, we can put all those reasons on the list.

Above that, there's a few other issues that can come into play.

Analysis Paralysis - When you get so wrapped up in thinking about the problem, you over look the truth in the old cliche "the best is the enemy of good enough". This is so classic, I'm not even talking about it any further here. Go here instead.

This Project That is too Important to Fail - If ever you hear those words about a project you're on, run. Don't walk. RUN! "This project is too important to fail" is a code for "Paranoia, Sloppiness and/or Egotism will cause us to make one bad decision after another." Seriously, I think I've seen at least three projects fall into this category. The project is "mission critical", so no resource will be spared so when there's the inevitable hiccup: the meetings that follow will distract the team and suck away crucial resources. Furthermore, inevitably, "momentum" and "positive attitude" start to take over and it's almost eerie how the decision making process begins to cave in. Add in all the other factors, and you're playing jenga on top of only one block. It's not the importance of the project that causes it to fail as much as the fear of failure. Basically, no one wants to sound like the one that is being negative and contradicting leadership or take on the political heavyweights. So, the small problems grow and grow until they consume the project, and the result is inevitable: Projects that are too important to fail usually do.

Everyone Wants In - These projects often also are the ones "too important to fail." Guess the story here: It's important so lots of people want in on the action. It's going to make the careers of people who are involved, so the proverbial wood work is working overtime. Everyone who can fabricate a reason to be on the project or better yet, 'advising' the project is there. The kindest way of looking at it is that you run into the Piling on the "Help" issue. The unkind way of looking at it (and more true) is that many (not all, or even most) are assessing their political/advancement prospects far more than their actual skill and contribution. At best, they are dead weight. At worst, they aren't content to follow the leader and want to be the top dog. I can't complain too much about someone who is making a full contribution even though their motivations may be bad. What causes problems is when they spend more time on their ambitions than the team and the project.

Now, teamwork really just comes down to mutual interest. Altruism rarely factors in for most people. We all are on a given project because it benefits us to do so. However, when someone is only on the project to reap and cares little for sowing, it's now a problem. What really becomes a problem is when someone's idea of reaping is not even letting the crops grow! Now, it's common to see large, important projects have some level of observer/adviser appointed by senior management to be their eyes and ears. That's ok, if a pain. But when someone places themselves in the role and refuses to budge, you now have a problem. This is the kind of person that makes Everyone Wants In projects bad. They don't contribute, but they sure make noise. When they start degrading other people's ideas simply because they aren't their own, the project is all but doomed. Avoid these folks, and Everyone Want In may not be all that bad. Presumably, it means you get to pick the most talented people available. However, if the talent selection process fails to weed out those who are here to take but not give, it's trouble.

Solving All the World's Problems - Ok, realistically, no one starts up a project thinking it will solve all the world's problems. However, quite often, the project's goals are overextended before it even begins. Honestly? Every project probable has more ambition than it's going to actually deliver on. It really only becomes a problem when the level of ambition turns into expectations that will not be satisfied. One way I've seen this manifest on several occasions is when what I like to call "false reuse" occurs. This happens when you have two or more stakeholders who describe their problem statements in very similar terms. In short order, the impression that everyone's needs can be satisfied with one project begins to get stuck in everyone's heads. Unfortunately, their problem statements may have been "I need to be able to produce a document I can print". They miss out that whole bit about what the document is and you end up trying to build a spread sheet and word processor all at once. We also run into this idea of "false reuse" at a code level, but it's a lot less damaging (still bad though).

We Can Solve Our Own Problems - This one is may be the worst. On one hand, the team may be facing an issue that it can't resolve by itself, but keeps trying to resolve itself. On the other hand, the team may be asking for help, but for whatever reason, the message never makes it out. I identified this in Faces of Failure as the "All Mouth, No Ears" type. There's a project manager who has sold the higher ups on a vision, a time line, a budget, whatever. But, reality and vision aren't lining up and they won't take the message out. Frustratingly enough, when I've run into this in the past I've also seen instances where the higher ups would have been happy to hear about the problem and readjust expectations! The project ends up dying because someone without the brass to look the Big Boss in the eye and say "it's not going to plan" early is in between the team and the ultimate decision maker (this isn't always the Project Manager, they are just in the role where it's easiest to do this).

Of course, if you are the project manager, you pretty much are going to get blamed anyway. Avoid the failure if you can and so read on. Keep in mind I'm not really writing towards PMs, I'm writing towards the everyone involved in software projects which by shear dint of percentages is mostly the developers, testers, implementers, etc.

Throw Yourself a Life Line... or... Go Down Guns Blazing

So, another long article and I've managed to talk mostly about failing again. Wasn't this one supposed to be "Rescuing Yourself?"

Oh yeah.

You see, it doesn't have to be a downward spiral. On occasion, I've seen a project rescued. So what do you do when you find yourself in the middle of a project that is dying? Or, what was good starts to become bad?

Don't Go There - I've never seen a project where the team worked well together (at that nirvana of knowing what to do together often without discussing it) fail. Note, that is my experience, I'm sure someone will contradict me. However, I've seen projects where the team that worked well together got split apart by all those other factors creeping in. Working well today is no sign of working well tomorrow. In general though, always keep checking the pulse. It's a truism, but it's easier to solve a problem when it's still small. But, that's not really the answer you were looking for, was it? You're already in failure mode, which leads me to:

Don't just do something, stand there! - That's the old HazMat advice and poking around the interwebs it looks like someone wrote a management book to that effect. I haven't read it and so can't recommend/endorse it. Anyway, if things are looking grim, first thing is to ask yourself "Why?" Look around for some of the factors I've discussed in this series. Some of them have obvious answers. Other's aren't so obvious. But, you need to understand the cause before you solve the problems.

Talk! - I wrote about the Complainer in the post on the Faces of Failure. You don't want to just complain, but you do want to open a dialog with whomever you can. Ideally, you get everyone involved. Often, this won't work. You may have an "All Mouth, No Ears" somewhere in the chain of command. You may have a Pacifist who doesn't want to see conflict, or a Warrior who is spoiling for anything to fight about. But, for the most part, you may be surprised to find out that even if others aren't seeing what you're seeing per se, they really want to talk about it. Lunch is the best team building exercise on the planet.

Blow up the Dam - A bit the opposite of "Don't just do something, stand there". Sometimes, you just need to stop talking and do something! You've ended up in a situation where for whatever reason, the project is just gaining any ground. Once you've identified an impediment or two, get rid of it. The larger the project, the more likely you are to have needless bureaucracy (see Everybody Wants In). Do what you can to get rid of it. Also applies to Analysis Paralysis. Sometimes, the best way to solve the problem is to start writing some code.

Use People Effectively - Outside of obvious things like not engaging people in useless tasks, think of more creative solutions when you need more people. Rather than adding yet another coder, you may need someone just to manage the build process and deployments to staging. You may be surprised how much time this eats up. Rather than having it take 10% of 10 people's days, have it take 100% of one person's day (ok, the math rarely adds up so evenly, you get the point though).

Be Formal Enough - A myth of Agile Software Development is that there is no documentation in an Agile project. The reality is that the Agile Manifesto says only that we should value Working software over comprehensive documentation. You are seriously misreading the Manifesto if you take that to mean no documentation. One of my Rules Regarding Everything is that "the more people you have, the more formal you need to be". In short, the more people you have, the more you need of things like documentation, task tracking, etc. Add distance into the equation, and it only goes up. Trying to substitue email and Excel for proper tracking doesn't scale, and trying to keep it all in people's heads is just begging for trouble if nothing else. In short, not adding a little formality can actually cause you to do more writing.

All Requirements Are Subject To Change - Seriously. Work on understanding needs not requirements. Lots of times, someone put a requirement down because they were trying to envision a way of solving the problem. If you have an easier, cheaper, or better way, talk to them, and...

Above All, Chutzpah - I have to throw some Yiddish in somewhere, don't I? After all, I grew up in the Borscht Belt. Anyway, Chutzpah is simply "audacity".

Without courage, you are just saying "walk over me". Many failed projects are savable, but someone doesn't stand up to say "enough". If you're the developer and all you do is say "yes I can do that, yes I can do that, yes I can do that." Don't be surprised when you can't do it anymore as work piles up. Meanwhile, you're only transmitting the message "I have more capacity." If you don't have capacity, if you have to work lot of overtime to get it done, SAY SO! This is probably a separate article by itself, but people can work O/T only so much before they melt down mentally.

If you see there are other misbehaviors/malfunctions in the team, you need to step up to the team and constructively take on the issues. If external influences are driving the issues, you need to start talking up the food chain. I've seen it happen many times that people just didn't know how complex etc something was that they were asking for. When I've traced the requirement back to the source (this can take some fighting believe it or not), its very often very negotiable. Talk and be courageous!

Being courageous is not being pigheaded and it certainly isn't being a cowboy. You need to listen to others, be ready to admit you're wrong, etc. However, while I say it's not being a cowboy, it may take some creativity. For example: sometimes your problems are processes that don't make any sense for what you're doing on this project. I've generally been successful by complying, but isolating the impact of those processes on day to day activities. Showing compliance in the battles you can't win also helps you in the battles that you can win. If folks think you're reasonable, they're more likely to listen when you say something is a problem.

Keep Your Skills Sharp, Soft And Broad (Ready to Relocate?) - All statements about courage etc aside... if you have a sense of project failure looming, you better load up the resume machine. You probably don't know what will happen in the end. Most software folks I know who have a hard time getting reemployed have let themselves get over specialized. Keep your skills broad, not just all technical. I've also found being ready to relocate helps a lot, but isn't practical for everyone. Also, keep in mind that changing jobs/projects isn't always the same as leaving the company! One more reason to be courageous, but level headed.

Anyway, this ran a LOT longer than I thought it would. As usually, probably 30 typos that I'll fix tomorrow or something. Adios!

March 05

Faces of Failure

jim leonardo on 3/5/2010 11:21 AM

My last post discussed what I called "Dimensions of Failure". At the end of that post, I promised I'd look into some causes of failure this time. Unlike the last post which drew heavily on knowledge I'd accumulated from other authors in addition to seeing how things played out in real life, this one will draw mostly on those real life experiences.

It really needs to be said: Most software projects do not fail for reasons related to technology (please notice the use of the qualifier "Most"). Most fail for other reasons, those reasons are either a failure to understand the real nature of the need that is driving the project in the first place, or people issues. I'm going to look more at the people aspect this time around.

These may be machines we work on, but software development is a craft. There's nothing scientific about it at all. It is not created by artists, but rather artisans. I've ceased thinking of myself as an engineer (regardless of job titles), an architect (blah), an analyst (that's a job function, not a job position), and certainly not a manager (that sound was me running screaming out of the room). I am a creator of software.

I say that because the impact of the people that you have on a project is critical to the outcome of the project. Understanding what kinds of people you have is also critical. You may have had 3 really super developers last time, this time you have 5 wet behind the ears college grads. Last time you had some real passive aggressive types, this time you have people who are willing to fight to the last over any point. Without understanding the capabilities and personalities you have, you will operate far less efficiently. When a project begins failing, the personality types of the people on that project will dictate how they contribute to that failure. Usually, the very strengths that make them an asset when things are going well also dictate how they will be a detriment when things are failing. So, what kinds of people do you have on a failing project (again, from what I've seen)?

The Hero

If you just had three more people like the Hero, you'd be successful, right? If it's hyper-critical, if it looks hard, you give it to the Hero. The Hero has probably been around the project the longest, knows exactly what to do, gets it done quickly, etc.

The problem with the Hero in failure mode is precisely the fact that everyone starts leaning on him or her. This turns into a situation where Hero can't keep up to all of the demands. The situation is further exacerbated by the Hero becoming more and more intimate with the inner workings of the application while everyone else becomes less and less intimate because you're leaning so heavily on the Hero to do anything. You need the Hero knowledge sharing, mentoring, and driving, not being isolated to "just get things done".

Needless to say, the Hero becomes a high risk for getting another job at exactly the worst possible time, or turning into the Burn Out, Lone Wolf etc. Anyway you slice it, the Hero is an asset, so really, addressing the negative aspects of the Hero consists of getting the project back on track. Of course, that may not be possible. If that's the case, guard the Hero against slipping into one of the more dangerous types.

The Martyr

Related to the Hero, and perilously close to being a Wannabe Hero is the Martyr. The Martyr wants to be a Hero, but unlike the Wannabe Hero, isn't actually out and out destructive. The problem with the Martyr is they are willing to grind themselves (any anyone else willing to join them) into the ground to get things done. They put in long hours even when not needed. They volunteer for the worst tasks. They are the ones on the plane to Chicago in January and so on.

They are close to being the Hero, but they don't have the skills and/or knowledge of the application to quite make it to Hero status. Meanwhile, the long hours, the constant throwing themselves in front of the bus "for the team's sake" exacts a toll. Inevitably the quality of work suffers because they are tired and if you're lucky, they turn into a Burn Out. If you're not lucky, they turn into the Wannabe Hero.

Often, building trust with the Martyr can help them to get out of the habit of killing themselves. If you can show them that they can be appreciated for doing a more normal level of work, then they'll cut it out. Sometimes, that's not enough though because this really is part of their make up. In that case, I don't have an answer for you other than to try to understand their motivations.

The Wannabe Hero

The Wannabe Hero is pure evil. The things that make someone a Wannabe Hero are not at all strengths. They are selfishness, ambition and unmerited pride personified. The Wannabe Hero is the person who appears to be a Hero, or at least a Martyr to everyone but anyone who has to get intimate with the work they do. The work they do is pure garbage, causes more problems than it solves, and yet they get credit for a "wonderful job" and "working hard".

All to often, the reason they are "working hard" is because they are incompetent or mentally lazy. They are making mistakes and having to put in long hours to make up for it. Hang around the software business and you'll hear a story similar to this over and over again...

"I joined a new company where everyone talked about so and so and how wonderful so and so was. Whenever something broke, so and so knew just how to fix it. So and so put in long hours and was the darling of management."

"After looking at the work so and so did, I realized that all of the problems were caused by so and so. Within two weeks, I was able to straighten out almost all of the issues. Of course, I never got so much as a pat on the back for it, but I also slept at night and got weekends off."

Sometimes, you get a Wannabe Hero out of the person who was once the Hero because they got overwhelmed and pride got in the way. They couldn't keep up, but the promotions, bonuses, etc kept coming until the Peter Principle reigned supreme again. They don't know the way out without being looked on as failures, so they end up in a death spiral.

If you can catch the Wannabe Hero early enough, then you can probably provide a soft landing for them and get them out of the death spiral. Otherwise, you're going to end up having to "repurpose" them or flat out get rid of them. In short, avoid the Peter Principle, and the Wannabe Hero usually isn't an issue.

The Lone Wolf

The Lone Wolf was often a Hero at some point. Now, they are afraid of letting their work depend on anyone else, so they do it all. The main problems with the Lone Wolf is they never mentor, they want nothing to do with any kind of meeting or status reports, and after a while, no one really knows what they are doing. The Lone Wolf is a particular breed of Hero, Wannabe Hero, or Martyr in all reality. The main thing that sets them apart is the fact that they are really good at working by themselves and very bad at working with others. They kill projects because no one knows what's going on, but are relying on them anyway. Or they kill projects because they don't work with anyone else, so they only integrate their stuff at the last minute.

Force them into the light. They will either bail on you or get it together and work with the team. But, don't expect it to be overnight in many cases, so don't try to go too fast with the lightbulb. If they are a Lone Wolf because in real life they are a loner, it's going to be a slow process and you really are just looking for them to be communicative. If they are a Lone Wolf because of trust issues, work on the trust issues. If you find yourself a Lone Wolf, ask yourself why? Really, most people don't suck, and if you think you are that good, let me tell you, most of the really good people I've met are really humble as well.

The Complainer

It sucks. Whatever it is, it sucks. You know that meeting we had yesterday? It sucked. You know the team dinner we're supposed to have tomorrow night? It sucks.

The new tools? They suck. The PCs we have? They suck.

You get the drift. The Complainer can't see the silver lining in anything. They destroy morale so quickly that they turn everyone else into Complainers or Burn Outs. When you're particularly unlucky, the Complainer is also a Warrior, but usually they're more passive-agressive than that.

Now, I grew up in New York, so I'm legally entitled to complain. Complaining itself isn't horrible. When things are going badly, you need the cathartic effect of complaining to just air it out. Or, the person's got some other issues going on and complaining about work is 'safe'. Shoot, sometimes they're just the voice of the rest of the team and act as a safety valve. If that's the case, let them blow the steam! If it gets too much, then my advice is stop and listen to them! Sometimes, the Complainer complains constantly because they don't know how to affect change and they will calm it down if they know others are willing to listen and act. Other times, they can't deal with reality. If that's the case, put them on the hook for solving problems instead of just raising the issue and they might just learn the reason why everything isn't perfect is because there's just not enough hours in the day. Either way, this is actually a person who can be handled much of the time.

A subset of Complainers are actually Blamers. Whereas the Complainer is frustrated in someway, the Blamer is usually either afraid of losing their job or has low self esteem. The Complainer can haul a project into the ground because they destroy everyone else's drive. The Blamer shoots the project down by causing friction and division. The Complainer can sometimes be handled easily, the Blamer usually is a lot harder to deal with. The Blamer is usually a Blamer because of a lack of trust that they won't be the first tossed under the bus when it comes along, so you need to work on that trust with them.

The Warrior

Got someone willing to fight it out for a worthy cause? You've got a potential Warrior. The Warrior is a great asset to have when things are going well because they won't let people derail you. When things are going bad, they are also fighting to "make things better". Hey, they can remember when they stuck to their guns and it worked, so it's got to work again, right?

The Warrior often gets to a point where they get frustrated and they fight with everyone. That's where the problem starts. They can drive a team apart quite quickly. Sometimes you get lucky though, they get everyone else pulling together because they all hate the Warrior. They also burn a ton of energy fighting and frequently sit and stew when they can't fight for whatever reason. This leads to lost productivity in lots of ways.

Really, the only way to deal with the Warrior that I know of is to come at them head on. Confront them with what they're doing and let them know you don't like it. If they listen, great. If not, you're in trouble. Otherwise, try to deal with them like a Complainer, because they probably are just a more agressive Complainer anyway.

The Pacifist

Worse than the Warrior is the Pacifist. When things are going well, the Pacifist calms people down when they are getting a bit excitable. The problem is that it's not because the Pacifist is a sworn peacemaker. The problem is the Pacifist is afraid of conflict and can't stand when there isn't peace. When it comes to Fight or Flight, they flee. Always.

The problem is they can create a team situation where no one can air the dirty laundry. When a problem needs to be dealt with that will cause disagreement, as soon as the volume goes up, the Pacifist tries to table the discussion. The sense of repressed feeling builds until no one is talking to one another. No communication makes failure a very short road. If a project is in trouble, you need to deal with problems and sometimes, or oftentimes, there will be disagreement. If you can't get people talking, you won't solve the problems.

A manager who is a Pacifist and not a peace maker can turn a successful project into a failure just by ending every disagreement by declaring it over before everyone who has a point of view can say it. Problems are swept under the carpet and never dealt with until the backlog of problems overwhelms the team.

Short term, the team needs to figure out how to not punch the Pacifists Fight or Flight button when in a disagreement. Some Pacifists will be able to get over their pacifism if they see the team is able to argue without imploding. Others never will. Work on calm disagreement.

The Defender

You know the Defender. They have their territory and won't let anyone else into it if they can help it. In this way, they can be a bit Lone Wolfish, but the Defender is probably willing to share their territory with someone else who they perceive as having the same project focus. Usually, the obvious place for defenders is as DBAs. Business Analysts and UI designers often can turn into Defenders as well. The short of it is that they probably have some special skill, or special understanding of an area of the application and are afraid to trust anyone with less knowledge than them. Sometimes this also just comes out of pride... They think they did a really good job in building "x" and don't want anyone else messing with their "baby".

Defenders can sometimes be dealt with by requiring anyone who needs to work in the Defender's "territory" to have their work reviewed by the Defender. If they see that it's not the end of the world, they will loosen up. If they find lots of problems, maybe they weren't so wrong!

The Burn Out

Anyone can turn into the Burn Out. You know the Burn Out. They've fought and worked hard, but got nowhere. I once heard a pastor describe burn out this way: "Working hard doesn't cause burn out. Burn out is caused by working hard and not getting anywhere." Now, the Burn Out just coasts if you're lucky. If you're not lucky, the Burn Out can't really do quality work anymore and primarily operate as a bug factory.

Usually, giving them a break from the same old grind is sufficient to snap them out of it. Vacation can help acute Burn Outs, but worn in chronic Burn Outs probably need re-tasking. Even a week of re-tasking can sometimes help dramatically. Or, shake up the project in other ways.

Overall, Burn Out is usually a condition, not a personality type. A change of pace, scenery or a breather can remedy the condition.

All Mouth, No Ears

I've saved this one for last because unlike the rest, this is a real project killer. All Mouth and No Ears is that decision maker who has the plan and doesn't mind telling you. The problem is that they won't listen to anyone else. When contradictory evidence comes up, they raise the reality-denial-bubble and pretend it will go away.

This is a big problem when this manifests in management and stakeholders. The team can scream til they're blue in the face that the deadlines are unreasonable, but they will still find themselves committed to the same death march. In fairness, often this manifests due to the downhill nature of pressure and may not be avoidable in that instance.
A phrase to look out for is "This project is too important to fail." You are dealing with All Mouth, No Ears. In this case, they will just try throwing more people, working longer hours, etc, etc even when alternatives are presented. "This project is too important to fail" often leads to singleminded-ness. I'm here to tell you that every time I've heard that, the project failed.

What to do? Initially, fight the battle. I would not focus on just shouting about them trying to shove 10 pounds into a 5 pound sack. I would seriously recommend trying to find an alternate route or a minimum route. There may be some gold plating someone is trying to inject. Get down to essentials, and it may be achievable.

If you can't win it, then either resign yourself to taking the paychecks or get another job. Ultimately, it's their money, they can do what they want. Your responsibility is to advise. To do otherwise is unprofessional. It's well within your rights to refuse to death march, but the right way to do that is to not sabotage the project. Sometimes, you're just being nervous and things will work out. However, your resume is up to date, right?

Firing in the Dark

Now before you think I'm being a snob, I'm sure I fit into a couple of the categories above when things go bad. So do you. What is important is to realize what's happening and you can redirect the strengths of yourself and others back into a positive direction.

I once posted a comment on InfoQ containing this little rule of thumb...

CMM and its ilk are designed to minimize harm by bad behaviors and less skilled people, whereas an Agile methodology seeks to empower its people. CMM=minimize risk, Agile=maximize productivity.

The short of it is that when you have a team you can't really trust, you need to be heavier handed. I hope that lack of trust is just that there's less experience and folks need more guidance. Otherwise, why are they still around? I don't think this means waterfall type methodologies, but I do think you need to be, umm, agile in how you approach how light or heavyweight a process you use depending on relative skill level of the team.

Above all, don't treat people like machines. Treat them as individuals and you may get lucky. Most people in development actually want to be contributors. They want to reap what they sow. Make sure they can, and you will even be able to be successful when things do fail. At the very least, being able to say you gave it your best shot is worth something.

A last line disclaimer... I'm up way too late typing this. Some of this may not make much sense, so comment or private message me. I'm not afraid to revise. If I've offended you, I have no intention of doing so. I'm sure something I've written will hit close to the mark for some. As I said earlier, your strengths often dictate how you react when things go wrong. As such, if I am hitting close to the mark, I'm probably speaking as much to your strengths as weaknesses. We're all human. Failure is our normal state!

March 03

Dimensions of Failure

jim leonardo on 3/3/2010 11:21 AM

I made a slight reference to dimensions of failure in my last post. What I said there was failure to deliver was not the only dimension of failure. I would like to expand on this a bit more because I think it is extremely important from my perspective in the trenches. If this goes right, you'll probably be thinking "Jim, good job, but you missed..."

All of these are failures because they impede your ability to deliver value to the customer/stakeholder both today and in the future. Now that I've said that, I'm not going to dwell too much on that bit of terminology. I'll also say that you'll probably find much of this isn't really new thoughts of my own. There's plenty of people like Fred Brooks who've hit on at least some of these ideas themselves and Roger Pressman has covered most of these in one form or another in his software engineering textbook (it's good as a survey of ideas out there, I wouldn't recommend it as a manual on how to attack a software project).

Some Dimensions of Failure

Well, We Tried (Delivery Failure)

At the end of the day, Failure to Deliver is the most significant failure. Right? After all, you are saying that you undertook the project, but it didn't deliver. How can it be worse than that?

Well, there's a simple fallacy inherent in this thinking. Sometimes, it is perfectly ok to not deliver working software (or whatever) at the end of the project. You may embark on a project knowing that the problem you are trying to solve is extremely difficult, that others have tried it, and it hasn't succeed. Your customer may be perfectly happy to throw money at the problem on the off chance that this time a good solution presents itself. In this case, reducing the uncertainty that they could be taking advantage of something that they are missing out on has its own particular value.

Finally, Here You Go (Schedule/Resource Failure)

This is probably the most easily understood failure dimension and is certainly the most common. The project got done, was acceptable, everyone loves using it etc. However it was late even past the ideal date, the "we gotta have by" date, and three further "drop dead deadlines". It cost twice as much, and it involved, by the end, three times as many people.

The obvious cost overruns are bad enough. Far worse is the long term psychological impact on the organization as it doesn't ever want to go through that again or loses it's taste for taking on risky projects. The long term cost of not taking on good, but risky ideas is hard to measure and has led more than one organization into fatal stagnation.

We Gotta Keep The Contractors For A While Longer(Completeness/Correctness Failure)

Completeness/Correctness Failure (do not confuse this with Iterative Development!) takes a couple of forms. The easy one is that the project was 'ready' at the end of the projected time, but there were key features missing. Inevitably, there's at least some features that won't make it in that you were hoping to get in to a release. That's ok. I'm talking about key features here. Think building the next Word replacement and not being able to print. That's a key failure, because very few users are going to want to use a word processor and not be able to print. They might be ok with manual duplex features missing, but if they can't kill a tree, they're not going to touch it. This may be related to the other form of this failure mode...

More commonly, this takes the form of a project that was delivered with the majority of features, but bugs abound. There's no such thing as a significant project that releases without some bugs. However, you want to keep them down to the sort that are cosmetic, are not stop work, don't cause data loss, and don't corrupt the system they are on. It all comes down to the impact on the user. If every 40th page, your printing suddenly inverts black and white, you're gonna have annoyed users to say the least. There's a bit of wisdom about waiting for the first service pack out there, but you need to also build some credibility. If you were hoping to direct the team in to Version 2 right after releasing Version 1, but have to have 90% working on bug fixes and not the V2 features, something is drastically wrong. What's worse is if you were planning on working on that Snow Leopard port and now have to miss a whole market segment because you have to put people on bug fixing.

We Built It, They Didn't Come (Market Failure)

People will say anything at a trade show if it's going to get them a free mug, even if all they ever drink are diet cokes. Market research done in a haphazard way has caused more than one organization to blow large sums of cash on a product no one wanted as badly as they said. Other times, people are dead convinced they need an app to help trim their toenails because they hate doing it, but the reality is no matter what you build, it's either still going to be inconvenient or the reality is they are so used to doing it that they can't fathom paying money to improve the situation. This last one happens a LOT. The other way this often goes wrong is that the understanding of the problem was totally wrong. Either you don't solve enough of the problem to have a marketable product or you have something that is just missing that magic 5% that makes people want to use it.

Market Failure can be incredibly frustrating. The development team does exactly what it was supposed to. Perhaps everyone involved, not just those inside the code, does a super job. But because of some flawed understand of the need, that perfectly working, bug free code turns out to be worth zilch.

What, You Don't Need It Anymore?(Opportunity Failure)

In the 1800s, a new type of sailing vessel called a Clipper was developed. These were the fastest wooden sailing ships devised and for a period were the kings of the global trade routes. However, very shortly after they came to prominence, this new fangled thing called a steam engine started becoming popular. The opening of the Suez Canal only sped up the process of the clipper becoming obsolete as steamships could navigate it with ease, whereas a sailing vessel had a hard time in narrow waters with uncertain winds.

Many projects suffer a similar fate. They are built. They are built on time, on budget, meet the initial understanding of what was to be delivered, and by the early definition, they should have been successful. However, in the meantime, the window of opportunity has closed. Opportunity Failure occurs because of sea changes in what people want, competing products come out, new regulations come into law, etc. But overall, the theme is that there was a right time, but that right time has come and gone. This is often exacerbated by or entirely because of Schedule/Resource Failure.

In short, those Vancouver 2010 t shirts may be really great, but if you didn't have them ready until the day after the games ended, they really aren't going to make you a bunch of money.

Now Can I Get Some Sleep?(Team Sustainability Failure)

This the one I hate most. This usually comes along with one of the other dimensions of failure. In order to get done on time, weekends cease being time spent recharging with family and people end up working 60, 70, 80 hours a week (yeah, do the math on 80 hours a week). They manage to deliver something, but usually it's still a Schedule/Resource Failure, often a Completeness / Correctness Failure, and frequently features other dimensions to boot.

I call this Team Sustainability Failure because this leads to burnt out people.

"That guy who was a real hero and had all the brilliant early ideas? Well he hasn't had a good idea in months. The woman who was solving every problem encountered almost instantly? Well, she's been stuck on the same bug for three days now. And what about that one developer actually born in this country ? What happened to him. He was really the best we had, wasn't he?

"Oh yeah, citizenship plus skills meant he got an offer from someone else 3 months ago and flew out the door. He hated the hours, and was just too blasted marketable. At least he didn't steal our source code, although I'm not sure why anyone would want to."

You probably see the problem here. When the project gets on a death march like this, the team's ability to mentally regenerate begins to drop precipitously. With drops in mental regeneration comes more and more mistakes. More and more mistakes lead to falling further and further behind, and you get the rest.

Parting Shots

It's rare to see only one of these dimensions of failure manifest by itself. If it does, it's probably Opportunity Failure or Market Failure. Usually, one dimension of failure is accompanied by others. Not all failures are easily visible enough to see what's coming early. However, most failures have plenty of warning signs along the way and plugs can be pulled before too much is wasted or rocking ship can be steadied before it capsizes. Catastrophe rarely occurs because only one thing went wrong.

Technology is rarely the reason projects fail. Next time... some reasons for failure from my own personal experience (and I won't just be ripping on how having too many Business Analysts and Architects spoils the broth). What I can share now is this simple fact... most dimensions of failure can be avoided by tight feedback loops in order to make sure you are constantly driving towards value in your product. Iterative development and continuous integration are just two ways to tighten up the feedback loop. Understanding the team you have and making sure you are doing what's right to keep that team productive is the other important factor.

March 02

Kai Gilb... Agile is dead?

jim leonardo on 3/2/2010 11:16 AM

So there's this fellow out there named Kai Gilb who is writing a series of articles about how bad Agile is. To increase the visibility of his arguments, and because his site requires log in, I'm commenting back here. I also don't want to lose the old thought-a-roos...

So, many of his points so far in the series are spot on. I will say that. In fact, given the so called "State of Agile", I'd give Kai probably an 85-90 out of 100. There are, however, a few things that I think merit attention in his arguments. This probably isn't my best writing, call it draft one.

Rebuttals

Hey, Quit Picking On the Manifesto

I'm a little concerned whether Kai understands the historical context of the Agile Manifesto. The Agile Manifesto was written in a time period when many organizations were drowning in process molasses. Kai really rips into the Manifesto, particularly around the idea that the Manifesto does not focus on delivering Business Value. This is a great example of how someone can be oh so right and still wrong.

I would first make the very simple observation that the Manifesto was written in 2001, that's nine years ago. It came out of a discussion culminating in a small conference between like minded individuals who all were fed up with the stifling environment being created by bloated, bureaucratic processes. The mindset at the time was that 1/3 or more of the effort on the project should be on requirements gathering and design. Many, many organizations interpreted this to mean that you should do all analysis work upfront, before development began. This was largely driven by a group of studies which indicated that the reason most projects fail is due to improper understanding of the requirements. The consensus answer was that we had to do more thorough analysis before we tried to build it. Guess what? Projects still failed (and failure to deliver is not the only dimension of failure). Why did they fail? I've been party to several project failures (either in the trenches, or coming after the fact to pick up the pieces). The reasons have not been for lack of understanding the technology. The technology is usually the one thing you can account for uncertainty in. The reasons have come down to various flavors and degrees of the following (nothing new here)...

Took too long/cost too much (time = money or opportunity cost)
Requirements were poorly understood.
It was built according to spec, but spec didn't meet the need.
The business just didn't want it anymore.

When you read the Manifesto, it really comes down to "If you follow the process, and don't deliver you lose. If you don't follow the process, and deliver, you win." What Kai rightly objects to is the lack of focus on "business value" and instead the focus on "product". However, the Manifesto was written at a time where "business value" was something simply calculated as Return on Investment (ROI). Since Agile has opened up the dialog, "business value" has become "Business Value" and I argue that what the initial signatories of the Manifesto had in mind was "Business Value". Jim Highsmith's commentary on the history of the Manifesto all but confirms as much:

At the core, I believe Agile Methodologists are really about "mushy" stuff about delivering good products to customers by operating in an environment that does more than talk about "people as our most important asset" but actually "acts" as if people were the most important, and lose the word "asset"...

Now, certainly, reading that, you pick up on Kai's argument that the Manifesto is more about the developer than the stakeholder, but you also see that it is about enabling the developer to satisfy the stakeholder (customers). The goal is "delivering good products to customers". The means is "operating in an environment...". Everything I read in that time period (say 2001-2003) about Agile from the signatories was ALL about the customer. No one had anything else in mind. We, as people who were practicing or fighting with their bosses (me) about practicing Agile, were concerned about nonsensical processes that seemed to be more about justifying jobs for particularly incompetent business analysts and 'architects' (washed up senior developers who couldn't hack management) and heading off the dreaded FUD (fear, uncertainty, doubt) than satisfying the people who needed the product.

In short, Kai's being rather unfair to the Manifesto and its authors/original signatories. His Claims in part 2 actually sound suspiciously similar to similar statements being made 9 years ago.

Great Points, But Don't Feed the Trolls

My second concern in his arguments has nothing to do with content, but is about the approach. He is taking a rather hyperbolic approach which will inevitably be taken up by certain brands of trolls who want to be in total control of everything to say "Agile isn't working and so we should go back to the 'good old days'." They won't read any further. They will just take his headlines and run off with them.

That's about as far as I'll go with taking significant exception to his arguments so far (up to post 2).

Let's Agree to Agree

This always happens

In the last three years, the idiot brigade has taken over Agile. Scrum (which is pre-Agile in all actuality) has become "The Agile Methodology" or "The Agile Process". I honestly believe if you jumped back 8-9 years to the early Agilists, you'd scare them out of their pants if you told them Scrum was pretty much it (go read that history again). Their whole point was more to do what was sensible for your organization, your project, and your customer than to follow someone else's cookbook. Sure, idea theft is the sincerest form of flattery, but slavishly following it is not what was in mind. These days, Scrum Master certification is almost mandatory to be considered an Agile practitioner. I've never seen certification in anything actually shed any light on anyone's actual competence. I've seen people without a relevant certification who were absolutely brilliant. I've seen people with technical certification who shouldn't be allowed anywhere near a computer, let alone operating one.

Useless User Stories

User Stories used to be what Use Cases were before Use Cases became too bloated. Now User Stories are what Use Cases became. Generally speaking, over-specification kills. My 'text book' case of over-specification was about 5-6 years ago.

Coming on to a project for a month or so to help out in their final phases of development, I got this requirement: "We need to be able to output all of the Web Pages associated with the client to PDF." I was a bit scared by this. At the time, there really weren't a whole lot of good PDF generators for server environments (particularly Windows) and knew that our usual purchasing procedures ran 2-3 months anyway. Not exactly what the project management wanted to hear as they were, as usual, way behind and way over budget. A few more conversations with the folks on the requirements side got the wheels turning. Back at my 'desk' (a folding table in temporary space), the light bulb finally warmed up and lit up. Tromping back across the maze of cables taped across the floor of our temporary accommodations, I cornered the BA.

"What do they do with the PDF," I asked.

"To print it of course," he said.

"So you don't need a PDF, you need to be able to print the pages in a decent format."

"Ummmmmm yes. But we can't do that."

"That's what style sheets are for. So can we agree that we need to make it printable, not generate PDFs?"

"Yes."

Yes, coming in at the tail end of a project and trying to make sense of all the style sheets is a real pain. But there was no way we'd be able identify, purchase, and implement a PDF component in time. It wasn't fun, but it was mission accomplished. Lesson learned... never put off printing til the end and always question the requirement until your certain you've separated out needs from someone else's solution to the needs.

This is where small user stories, use cases, what ever the heck they are called this go 'round really come into play. If you force them to keep it on an index card (in large print!), then you're more likely to get the need than their idea of a solution to the need. Don't let them start the problem solving without you! Sure, let them play, but really, the developer is the expert in what can and can't be done here.

If You Need Software Created, You Need A Developer

Really, we are back on the same roads that we were before. The same clueless wonders continue to adhere to the patently false impression that software development is a factory process. According to this mindset, developers are a commodity and you can drop someone who didn't even touch a computer until they started learning how to program 4 years ago into the same spot as someone who has been at it for 10 years and was using a computer 10 years before that. Fundamentally, there's a misconception (in my mind) that software development is a group of problems solved by thorough analysis. Once you understand the need, you will be able to get any reasonable competent coder to crank out an app to address the need. NONSENSE! If all you are is analytical, you will figure out what the problem is, but you will often not be able to solve your way out of it. The problems we face require creativity, an understanding of how it has been solved in the past, a certain level of battle scars, and just plain experience. Software development, like it or not, is about the people doing the work. I've seen plenty of cases where the team that was 'too small to get it done' got it done and the 'project that was too important to fail' did because management panicked and threw people at it until the project was paralyzed with meetings, documentation, and above all, Business Analysts. BA's are project killers in my opinion. I did an inventory of projects a short while ago that I've worked on or had intimate knowledge about and the results were startling:

Without fail, the success level of the project was inversely proportional to the level of BA involvement. (i.e. BA's did not help)
The successful projects without fail had the developers talking directly to people who would use the product (or were past that point in their career and were now managing those people).

Conclusion: Despite the all the FUD, developers actually do a pretty good job of listening to the user and know how to turn that into an app.

Drive-by Bullet Points

Agile is now firmly in the death grip of cargo cult mentality. Most 'adopters' are using Scrum and XP in a cookbook fashion without analyzing the philosophy behind it. When confronted with the problem, they blame process and not their own thinking. Agile is all about adapting to your customer's/organization's needs, not slavishly following the process to the detriment of what you are trying to do.
This cargo cult mentality is exactly what dooms everything in organizational methodology. Six Sigma worked for the early adopters who really bought into it and adopted not only the process, but the philosophy. Ditto that for RUP, CMM, XP and anything else you care to think about. It's when the success stories make it to the general population that the methodology starts to break down. The successes weren't aberrations, they just had the right conditions to be successful.
Agile got started with people who were really just saying "hook me up with the customer so I can deliver what they need. Get the cruft out of the way. Great things can be done with great people".
Kai seems to be on the road to making the very same argument. In fact, the Craftsman movement is starting to shape up (although this is the nascent form) and based on Kai's arguments so far, I think he's really leading in this direction. At the least, he's starting out on the same foot. I'm going out on a limb... within 10 years, someone will be writing something like "I have nothing but respect for Kai Gilb, but..."
Kai is right, the state of Agile is bad. It is now as bad as what it replaced. But, real Agile folks won't disagree. Not only are we not afraid to hear it, we've been saying it all along!
This is a case of Evolve or Die. If we tried to stick to 9 year old ideas without modification, either those were true geniuses (who should be working on cures for cancer, or at least quantum algos) or we're true idiots. The landscape has greatly changed in 9 years. Our approaches need to change as well. But the fundamental concept of getting what the customer needs into the customer's hands hasn't. Call it Business Value, call it Working Software, or whatever. It really is the same thing.

1 - 10

RSS Feed

Alert Me