Missed again...

One of the most popular dating sites, which claims to be chasing Match.com this year, is run by one man out of his home. PlentyOfFish.com claims to generate over $10,000 a day in ad revenues, as documented in this video. In the video you can see a check from Google, which he also posted a picture of, for just under $1M, which was earned in about 2 months, as claimed on his blog. The Wall Street Journal reported that Google did indeed verify that this was a valid check.

I must be in the wrong business. With technologies like Bungee Labs now making web development "so easy a caveman can do it", and with Google paying out the wazoo for ad placement, just about anyone can make a website and make themselves a ton of cash-ola.

Sounds a lot like the dot-com boom all over again...only this time the cash is flowing. I guess I need to come up with a great idea that I can support with AdSense revenues...

Agile and the "Big Picture"

Software Quality News posted an interview with Alistair Cockburn regarding Agile and how it has been implemented (or not) by companies since the creation of the manifesto. I really like the interview, but the thing that really hit a nerve with me was one of his top ten ways you can tell you are NOT doing Agile. Number 9 is as follows:

They have itty-bitty requirements on the order of "here's what happens when you click here," but they don't have long-term vision for what they're trying to accomplish.

This reminds me of a discussion we had in the last the CTO Peer to Peer regarding how some groups expect their architecture to "emerge" from their agile processes, so they rationalize that there is no need for formal architecture in the project.

I believe these are all-too-frequent rationalizations when people attempt to adopt an Agile methodology. Since Agile de-emphasizes documentation, people jump on the cowboy mentality bandwagon, and begin coding without ANY documentation, hoping that their architecture, and their "big picture" system, will slowly emerge from the clouds as their project comes together.

This is a recipe for disaster. If anyone tells you that because you are "doing Agile" you cannot document the big picture or design an over-all architecture, that is a clear indication that this individual doesn't really understand the Agile methodology.

Jumping into solution space and writing code before you have a well-defined architecture and big-picture goal is like taking off in an airplane before you have a flight plan.

Using agile methodologies in this environment will only help you get the wrong solution released faster.

Mac vs. Vista

OK - I have to admit, while I am not a Mac user, I love their ads. They are very well done, and hit the nail on the head when poking fun at Vista. I have not converted to a Mac yet, but I am seriously considering making my next laptop a Mac.

With that backdrop, I read today that Mac sales are up 35% in the first quarter over the same period last year. The same article also stated that the Vista uptake rate is less than had been expected (no surprise there).

This begs the question: Why? What is driving people to buy a Mac and not Vista?

It has long been a well-known fact that Mac users have a religious fervor to their support of that platform. Back in the 80's and even the early 90's, Mac users were well known for their zeal in support of the freewheeling, innovative Apple over the stifling corporate-focused Microsoft.

However, I cannot believe that the "corporate culture" difference is the cause of this zeal today. Rather, here are some ideas that I have heard that would make more sense:

1. The OS X is simply a better designed product. I referred to this in an earlier post, but the concept that Steve Jobs simply has a vision of how his OS should work, and requires nothing less than perfection from his employees in the creation of this vision. He maintains strict control over the implementation, and ensures it meets his high expectations.
   
2. People are getting tired of being disappointed by Windows. They start with high expectations for significant improvement, but end up with an operating system that is only slightly incrementally better than the prior incarnation. The Mac is the natural place for these people to migrate to, since Linux desktops are still only for the "techies".
3. Sales of the Mac are being driven by the iPod.

So, what can a software company learn from this? What do we take away here that can be emulated as small software companies fighting the Goliath in their vertical market? Here are some possibilities:

1. You need a Dictator with vision to run your software company
2. Be the only viable alternative to the Goliath
3. Create a market via horizontal expansion that will drive sales of your product
4. Have some cool ads and a good marketing strategy

Of course, none of these make for a viable, implementable strategy for a software startup. But, there has to be some lessons to be learned from the Mac success that small companies can follow. I would love to hear other people's thoughts on what those are.

I really don't know what single thing is driving Mac sales, nor why Apple has been able to create such a religious following - or if you can even boil it down to one or two key points. But, if other companies can figure out their magic recipe, we might find many more small companies building extremely innovative solutions and gaining extremely loyal customers.

E-mail signatures?

Interesting article in the Journal today regarding E-mail signatures. It mentions services like Meez where you can create your own "avatar" and include it in your e-mail signature. Evidently, this is a craze that is catching on - they report over 1 million registered users, with half of them joining in the first quarter of 2007. Crazy - I guess that is another entrepreneurial idea that passed me by.

I have to admit, I used to attach pithy comments to my e-mails, but have not done so for quite some time. This article makes me wonder, though, if we have now gotten to the point where the signature holds more "meat" than the e-mail message itself. I have noticed a trend recently where the signatures on e-mails are becoming more and more obvious and "in-your-face" than in the past, with graphics and company logos and such. It almost draws your attention to the signature before you read the message. And, my "pithy comments" didn't suck up extra bandwidth.

The article also mentioned John Edwards website, where it offers a download of an image that shows your support for his campaign. Great - more unsolicited advertising in my e-mails, only now it is coming from people I know, embedded in e-mails I want to read (or maybe not anymore...).

How far will this go? Maybe we need a new e-mail product that will strip e-mails of anything that is extraneous to the message in the body. Now that is a service I would pay for.

How much testing is enough?

It really doesn't matter when you plan to release, nor how much testing has occurred - someone will always be worried that it was not enough. Whether you are on an agile schedule of releasing every 4 - 8 weeks, or a more traditional waterfall process of 6, 12 or 18 month releases, it always seems that there was not enough testing time to really feel comfortable shipping a product. If you test too long, you will miss your window of opportunity in the market, and if you don't test enough you risk shipping a bad product.

So, how much is enough, and when do you know it? Test automation helps, but still nothing compares to putting hands on the product and getting a good feeling about it. There is much more subjectivity to testing than most people realize.

It has been said that good managers make decisions when they have only 40% - 60% of the data needed for that decision. The reason being that if they wait for all of the data, they usually miss the opportunity, but if they make the decision with less than 40% of the data, they risk making the wrong decision.

So, does this concept apply to testing as well? It certainly seems to fit (if you wait too long, you miss the window) - but then, if you ship a product where only 40% - 60% of the features have been tested, that doesn't sit well with anyone.

The bottom line here is that the decision to ship a product always comes with risk. You can never test all possible scenarios, because you never know exactly how someone will use your product. And you really never know exactly what features have been affected by the recent code changes, even though you might think you do. You can mitigate that risk with automated testing that has significant coverage, and you can also reduce the risk with acceptance-level tests that cover the most-often used features in the product.

However, there will always be risk when shipping new code - regardless of whether you are using agile methodologies or not. But, the agile world gives you two significant benefits: first, the amount of code change for any release is minimized, so the risk of something slipping through is smaller, and secondly with short iterations you can react faster to a "bad" release - which you hope you never have to do, but the reality is that there are just too many variables in todays software environments to avoid it completely.

Developers are their own breed

I recall having a conversation with one of my peers when I was working at Intel many years ago. We were discussing what motivates Software Developers, and how to best manage them. I have often thought back on that conversation, and thought I might share some thoughts about that this morning.

In the end, most developers just want to work on "cool stuff". They want to know that people like their software, and that their baby isn't ugly. Some developers don't even care if the end-user likes their code - they are motivated by knowing that they have "slayed the dragon", and came up with some really cool way to connect the dots under the covers of the application - the end-user will never even know just how cool it is.

So, how do you motivate developers? Personally, I have learned two things that I believe are invaluable:

1. Learn how THEY like to be managed, and change your management style to match it. Some developers prefer to talk out each step of their solution, and get your approval of everything they do. They like to me micro-managed, although that term has a bad connotation in today's management theory. If they are more successful because they talk through their solutions with you, then you need to be happy to do so.

On the other hand, some developers simply want to take an assignment and run with it. In this case, make sure you have clearly identified the desired outcome, and leave them to it. You hired them to solve the problems, so let them do it - and judge the results based on how well it solves the problem - not on whether it was the way you would have done it or not.

Which leads us to the second lesson:

2. There is not "one best way" to solve any problem. When I was a developer, I was quite arrogant, and really thought that I knew the best way to solve any problem. If someone else didn't do it my way - well, it might work, but it won't be as good, or last as long, be as maintainable, or as user-friendly as my solution.

I remember vividly when I realized just how wrong I was. I had been assigned to work as an Architect, and had to learn and understand three different product lines with the intent of bringing them together under one architecture. As I sat down with the top technical minds behind each product line, I realized that they were all solving the same problem, and had to answer the same questions as they went along. Of course, they all answered them differently (some with java, some with C++, some with client-server, some with SAAS, etc). I remember one day thinking that no one actually came to work one day and said "how can I really screw this up?" - they all had solid reasons for making their design decisions - and they all were now working on very successful products, based on very different architectures and technologies. They had all looked at many of the same technologies and architectures, but had made different decisions along the way. In the end, they all had happy customers and great products.

Developers, especially the good ones, tend to be a bit arrogant, and believe that there is really only one way to solve a problem. This reminds me of Abraham Maslow's comment that "he who is good with a hammer tends to think everything is a nail."

Bottom line is that in order to understand developers, you need to know what motivates them and help them achieve that goal. For some it will be more money, for others it will be a technical leadership role, for others it will be public recognition, and for others it will be to just work on cool stuff and have happy customers.

Regardless of what motivates them, however, the most important thing you can do as a leader is let them do what you hired them to do. They are smart people, and you hired them for a reason - don't try to do their job as well as yours or you will both fail.

Microsoft claims patents on Open Source code

The Wall Street Journal today reported that Microsoft has claimed that 235 of its patents are being infringed upon by the Free and Open Source Software (FOSS) movement, specifically mentioning the GUI, e-mail programs and Open Office as violators.

My first thought was "how did Microsoft get patents on GUI and e-mail programs?" I didn't know you could patent an idea that you "borrowed" from someone else...OK - maybe I'm being too harsh on Microsoft.

My next thought was "What took them so long?" These open-source projects have been around for quite a while - why did they wait until now to claim infringement? Maybe they thought that the best way to fight the Open Source movement was to ignore it, and that by claiming patent infringement would lend credibility to the movement. It has now gotten to the point that they cannot ignore the momentum of Open Source (actually, it got to that point quite a while ago...it is just like MS to play catch-up with the rest of the industry).

Of course, Microsoft has been on the other end of this battle many times. Gee, the first to come to mind is the Zune...doesn't that infringe on some patents by Apple? A large company like Microsoft is happy to enter the market knowing they are infringing on patents, and will simply take the "oops - my bad" approach when sued later for it.

I have to admit that I have not yet jumped completely into the Open Source world. I like Linux, and I actually have Open Office installed on a couple of computers at home, but I have yet to really dive into the Open Source movement with both feet. I am not a Microsoft hater, but I certainly would not personally like to ever work for them.

So, I consider myself a fairly objective outsider in this battle. I love to watch the open-source community eat into the strong-hold that Microsoft has on the operating system market, as well as any other market that they can gain a foothold in.

Having said all of this, the most surprising comment from this short article, and possibly the most troublesome comment, was from the legal counsel to the Free Software Foundation identified as Eben Moglen. He was reported to have said that software in general is a mathematical algorithm, and as such is inherently non-patentable.

We all know that the patent system is in trouble, and needs an over-haul. However, having spent many years in the software world, I believe that software patents do indeed provide some protection for small start-up companies. Not necessarily as a deterrent to large companies, but rather as a recourse if/when the large company infringes on the small companies technological patent. If we do away with software patents altogether, then the software industry as a whole will suffer. Innovation will be stifled, and the companies like Microsoft will continue to dominate in their non-innovation mentality.

Maybe Eben should re-think the defense for Microsoft's contention. If they win using the "software is not patentable" approach, it will increase Microsoft's stronghold over the competition rather than weaken it.

UTC CTO Peer to Peer

We had a great UTC CTO Peer to Peer meeting this morning. The discussion was regarding Architecture, and how and when to "do" architecture. We talked a bit about agile processes, and how to fit architectural re-writes into the iterations.

The most interesting part of the discussion to me was the evolution of a Product Architect (i.e, the career path that leads to this role, and how one progresses toward it), as well as how that role differs from a technical architect (i.e., the bits-and-bytes architect). The concepts brought out by the presenter, and others in the room during the ensuing discussion, validated some of the concepts that I have recently blogged about: namely, that there needs to be one Product Architect who is responsible for the entire product. The discussion took this one step farther, though, and determined that this role has many over-lapping duties with the Product Manager. So much so, was the conclusion of the presenter, that the roles actually should be one-and-the-same.

Very interesting discussion, and something I need to put more thought into.

The Train has Left the Station

Many have heard this when discussing software enhancements, using the analogy that the software release is a train, and it is too late to get the feature into the product (even though the product is not yet released to the public).

I have put some thought into this analogy recently, and it is very interesting how far you can take the analogy and have it still hold up. It actually makes for a great way to help keep an agile process (or any process, really) on track (pun intended...).

Here is how it works:

First, we need to understand our terminology:

• The Train is the Product Release (ours arrive at their destination every 5 weeks)
• The Train Station is where Passengers board the train
• The Destination is a public release (at some point in the future)
• The Passengers are the Enhancements, or new features, for that product
• The Luggage is the design documents, screen shots, etc that describe the feature (i.e., their clothing, makeup, etc)
  
• The Ticket Counter operators are the Product Managers, Interactive Designers and QA people (in a test-first methodology, you have test cases before development starts)
• The Ticket Takers are the Developers (they put the passengers on the train)
• And, this train has a "freight" car, which is the bugs that are being fixed for this release (you can't leave them out, as they take development and testing resources as well as the features)
  

Now, there are plenty of "potential passengers" who are no where near the train station. These are enhancements that are simply a gleam in someone's eye, and may or may not actually make it onto the train at some point. Some may live very close to the station, and may be on "stand-by".

The passengers in the train station are those features that are high on the priority list. Many of them are standing at the ticket counter, awaiting their ticket. In order to obtain a ticket, they need to have their luggage with them (if they have any). Applying the test-first process, this means that they have to have their test case, screen shot, etc with them (i.e., documentation regarding the feature has been created and is "attached" to to it in whatever form you use to track enhancements).

Once they obtain a ticket, they are allowed to board the train. The train that is in the station being boarded is the next-in-line public release, which is beyond the scope of the train that has "left the station" - that is, the development team is already working on getting a train to its destination, and it is too late to add more features to it. The features being "boarded" now are for the following train, which has not yet started development.

Some ticket takers have been known to allow a passenger onto the train without a ticket. This would be a developer who sees a waiting "passenger", and decides to proceed without waiting for the passenger to be ticketed - this is how some features "sneak" into a release without getting designed first. While this is not always a good thing, it is better to spend the time on a feature that is a high-priority than some passenger who is not even close to the station yet...and some passengers can travel much lighter than others, and don't need as much design.

Note that if the freight car is too heavy, it may require removing a passenger car from this train - that is, if engineering is too busy fixing bugs for this release, some feature enhancements may have to wait for the next train.

Well, you get the idea. As I said, the analogy holds up fairly well, as long as you are willing to stretch the imagination just a little bit. You can even have some fun with it - set up queries in your bug/enhancement database to match these concepts (potential passengers, passengers at the ticket counter, boarded passengers, and passengers already traveling), and post them on your intranet - it is a good, light-hearted way to keep everyone informed and on the same page. And, it keeps your trains moving, and helps to prevent stow-aways from derailing a release...

Software Design: How?

How do you accomplish "good design" in software? There are so many people/roles involved:

• Product Management, who is supposed to be closest to the customer needs
• Marketing, who is supposed to be closest to the customers wants/desires
• Development, who implements everyone else's ideas (and, ironically the farthest removed from the customer)
• Quality Assurance, who is supposed to be closest to the customers pain threshold
• Customer Support, who (unfortunately) is many times the only one to actually TALK to a real customer - but only the unhappy ones.

Everyone believes that they know the customer best, and all have ideas regarding how a feature should be implemented.

So, how do software companies resolve this? In order to completely understand what we should build, we need input from everyone, right? So, we get in a room and try to convince each other that WE know what would best meet the customers needs, then the developer returns to their computer and one of two things happen: either they implement what they THOUGHT they heard in the meeting, or they implement what THEY believe to be the right thing, regardless of what came out of the meeting.

This is called "Design by Committee", and it never works. It is based on self-referential design, where each person simply puts themselves in the position of "user", and decides how they would like the feature to work.

Frederick Brooks, author of the Mythical Man Month and No Silver Bullet talks about this in terms of "Conceptual Integrity":“Conceptual integrity in turn dictates that the design must proceed from one mind”... “I will contend that Conceptual Integrity is the most important consideration in system design”. He goes on to say that it is the single most important ingredient in success.

Wikipedia describes his idea of "Conceptual Integrity" in this way:

To make a user-friendly system, the system must have conceptual integrity, which can only be achieved by separating architecture from implementation. A single chief architect (or a small number of architects), acting on the user's behalf, decides what goes in the system and what stays out. A "super cool" idea by someone may not be included if it does not fit seamlessly with the overall system design. In fact, to ensure a user-friendly system, a system may deliberately provide fewer features than it is capable of. The point is that if a system is too complicated to use, then many of its features will go unused because no one has the time to learn how to use them.

Donald Norman, author of The Design of Everyday Things also talks about this concept, He said:

"You don't do good software design by committee. You do it best by having a dictator. From the user's point of view, you must have a coherent design philosophy…The person who's done it best is Steve Jobs, and he's well-known for being a tyrant."

In this particular quote, he was referring to Open Source projects, but I believe the concept applies beyond that realm.

Finally, in addressing the same topic, Alan Cooper, author of The Inmates are Running the Asylum, said it this way:

“The hypothesis is that better results can be obtained by working in a team with equal representation from users, programmers, managers, marketers, and usability professionals. In my experience, this ‘Seat at the Table’ doesn’t work. The goals and concerns of the members diverge, and the one constituent whose goals are most relevant – the User – is often the poorest equipped to articulate his concern. What’s worse, the programmers – who always have ultimate control over the software-based artifact anyway – inevitably drive the team, usually from the back seat.”

The bottom line is that we need one true "customer representative" who can design the interface for the product, and everyone else needs to operate in a "disagree and commit" mentality. That is, you have a chance to provide your input to a feature, but if the "customer representative" (or "interactive designer", or "product dictator", or whatever you want to call this person) designs it differently, you stand behind it. Having a single design philosophy, or "conceptual integrity", and the right person to own it, works wonders when presenting functionality to an end-user, and maintains a simple user interface without too many "bells and whistles" that detract from the main goal of your application.

Parenting or Big Brother?

Where do we draw the line between being a good parent and being "big brother" when it comes to the Internet? Even the few comments to my post below shows the difference of opinion regarding how to help our children navigate the Internet safely. Where do our responsibilities lie as their parents?

Even more interesting is to read the comments on the "Family Safety on Windows Vista" blog. Some would have us believe that ANY attempt to monitor what our children do online, much less actually stop them from doing it, is a criminal offense.

The Internet is indeed a dangerous place, but then so is our own backyard. In the very recent past there was a case here in Utah where a young girl walked into her backyard and was lured into the home of a neighbor who almost immediately killed her. So, do we stop our children from leaving the house? Of course not, but I guarantee that any parent who saw that story certainly thought twice about letting their children play outside for a few days.

So, where are we on the continuum of Internet freedom with regard to our children? There are certainly many very good things that they can experience on the Internet today, as well as some time-wasting opportunities (my kids have recently discovered runescape), but there are also some very bad areas of the Internet that our children can wander into - or even worse, can search out. Even when we apply the "safe search" in the popular search engines, some inappropriate things can still appear.

Technology is far from perfect, and there are people out there who work every day to find ways to get their content in front of our children, regardless of what technology is in place to prevent it. Pornography is a huge business. Last year pornography generated $57 billion, which is more revenue than ABC, NBC and CBS combined, and more revenue than the NBA, NFL and MLB combined. The statistics boggle the mind.

So, what do we do? Do we turn off the Internet in our homes? Do we monitor their activity, and talk to them after-the-fact, or do we actively block access to the "bad" areas of the Internet? Personally, I believe that we are not fulfilling our responsibility as parents if we do not put some technological restrictions on their Internet access. Having unfettered access to the Internet is akin to leaving a loaded gun where the children can play with it.

However, as I said above: technology isn't perfect. Filters will block too much, safe search will allow bad things through, and monitoring software will miss some things. So, it still comes down to us. We need to keep computers out of bedrooms, we need to teach our children not to share personal information over the Internet, and we need to be constantly vigilant regarding how they spend their time online.

If we rely too heavily on technology to protect them from the dangers on the Internet, we will find ourselves dealing with some issues that we had hoped we would not have to deal with. The average age of first exposure to pornography on the Internet is 11 years old. The FBI has recently stated that if you frequent chat rooms, you have a 100% chance of interacting with an online predator.

Doing nothing is not an option. Installing monitoring or filtering technology and assuming you are protected is not responsible. It sad that we have to discuss these things, but the bottom line is that having the Internet in our homes requires that we take action to ensure it is handled responsibly - and that is more than just throwing more technology at the problem.

Agile Marketing?

Michael's comment from May 6th brought up a good point that I did not mention in my "Why Software Projects Fail" post. When a technical solution doesn't take off in the market as expected, it seems that the first thing most teams do is look at the technology and try to figure out how to change it so it will be better accepted. However, many times the problem is not in the technology, but elsewhere in the process or organization.

Agile development teaches us to react quickly to the changing landscape that is our customer base. Give them something to use, and then react to their requests quickly. However, this really only works when you have already established the end-goal that you are attempting to provide a solution for. If that over-arching goal is changing, then the only thing that agile development processes will do for you is help you to get things that don't solve customer problems into their hands faster.

Sometimes, when our software isn't being accepted as quickly as we had hoped, we need to look at the sales and marketing of that solution to ensure we are really trying to sell to the right market segment, or we need to review the company organization to ensure that the right people are working on understanding the customers needs and infusing that into the development process, or maybe we even need to review the customer that we are trying to serve to ensure that we really understood what their needs were.

The solution is not always to change the technology, or add more features - as a matter of fact, I would propose that this is hardly ever the solution to slow sales. Understanding the customers needs and helping them achieve their goals is.

Marketing cannot operate under the same "agile" processes that work for development. Trying to identify what customers want by giving them product and then watching how they react to it is not an effective marketing strategy. Understanding the customers goals upfront, defining what the end-goal is, and giving customer the solution one valuable and functional step at a time is what works.

Consider how Saturn entered the automobile market - they didn't come up with a line of cars immediately, but rather entered the market with a single car that they felt would hit a specific segment of the market, and they were right. As that car gained acceptance, they then added other cars and even entered the SUV market eventually. Each iteration along the way, however, they were meeting specific customer needs with a specifically designed automobile.

Sometimes it seems that our marketing efforts are trying to provide pieces of each car to all of the potential customers in an effort to see which segment will "bite" first. The theory, of course, is that we would then focus in on the needs of whichever customer actually does buy the product, and we could "iterate" until we solve their needs. We end up with our first iteration of the product being the wheels of an SUV on the body of a sport scar with lots of legroom in the backseat. It doesn't matter how many times we try to move the position of the radio and climate controls, or whether we add hands-free radio controls to the steering wheel, sales of this Frankenstein car will not pick up.

The logic is that we can't go after just one segment of the market, because that would be limiting our customer base. And, we can't leave anyone out of the potential market, because we may not know which segment is the most profitable. Or, we can't make decisions for the customer - we need to give them all the technology and let them decide which features they like the best. Rubbish.

Arguably the most important part of any project is the definition of the target customer, and a clear understanding of their needs. The technology can then be developed in a crisp, focused manner, and can be expanded to other market segments in the future. Without this crisp and clear understanding of the market segment we intend to satisfy, we are simply following an "agile marketing" approach of throwing spaghetti against the wall to see what sticks. Hardly effective, and usually tough to clean up.

Why do Software Projects Fail?

First of all, what is considered a failure? Shipping late, without needed features and with considerable bugs would be considered failures by most people. Shipping a product that has no bugs but doesn't meet anyone's needs certainly cannot be considered a success, and shipping "on time" but without a feature set that customers want could only be considered a failure.

So, why do software projects fail? Having participated in several software projects over the past 20 years, both successes and failures, I have come to the conclusion that failure is actually fairly easy to spot, and actually not that difficult to fix.

Some of the main reasons for failure are:

1. No Clear understanding of customer needs. The customer representatives in the project need to have a very clear understanding of customer needs - which the customer themselves may not always be able to clearly articulate. Henry Ford once said "If I had asked my customers what they wanted, they would have said 'Faster Horses' ". To provide a truly innovative solution that customers will love, you need to understand their needs and fill them rather than simply implementing the features that customers request. In the absence of a clear understanding of customer needs, many teams fall back to "Self-Referential design". This is very dangerous, as it leads to development of applications that are very well suited to the needs of the development and marketing teams that created them. Most of the time, this only partially meets the needs of the target customer.
   
2. No Clear Owner. All too often, software product development is divided between a marketing-owned group who identifies features and prioritizes bug lists, and an engineering team that implements the code. The leaders of the engineering and marketing groups are supposed to "co-own" the product, and supposedly there should be some "healthy friction" between them. This thought process is flawed, and only leads to "Design by Committee". Alan Cooper said: "When everyone has responsibility for Product Quality, nobody is responsible for Product Quality". I have never seen this type of process to be successful. Donald Norman, the author of "The Design of Everyday Things", has been quoted as saying: "You don't do good software design by committee. You do it best by having a dictator. From the user's point of view, you must have a coherent design philosophy... The person who's done it best is Steve Jobs, and he's well-known for being a tyrant."
   
3. Lack of Discipline. You need to know when the "train has left the station" (which, incidentally, is NOT when the product has shipped, but rather when the time for adding more features has passed), and to make the tough decisions regarding which bugs and enhancements to postpone and which to include in a release. Everyone wants the project to be successful. The problem is that "success" is not clearly defined at the outset, so everyone has a different view of what success means. Because of this, a gap exists between members of the team, and it becomes very difficult to draw the line of what will be fixed and working, and what may need to ship as a "known issue". Without this clear understanding, it is very difficult to determine when the "train has left the station", and scope-creep (or "feature-creep") ensues. It takes strict discipline on the part of development, quality assurance and product management to ensure that quality products are shipped on-time.
   
4. Putting too much on the plate. At the start of a project, some lines must be drawn. The entire team must understand what is needed, and when the product is fully-baked. Too often project teams put too much on the plate at the start of the project, and set themselves up for failure. The key is to define very limited-scope iterations that can be accomplished in a short time period. Everyone needs to understand what the goal is of this iteration, as well as what the over-arching goal is of the entire project (i.e., the company "mission"). Breaking down the overall goal into mini-goals, and defining short iterations helps to keep projects on-track, and ensures high quality as the project progresses.
   
5. Silos. Development needs to operate in a fishbowl, and there must be complete transparency into what they are working on. The development organization must have "windows" into their daily operations, and those outside of development need to be willing to look through the window. Whenever a failed project ships, I inevitable hear that those outside of development didn't know what the developers were working on until it was too late. There are two possible reasons for this:
1. Development is operating in a silo. Many development managers feel that in order to maintain discipline and remove feature-creep, they need to keep the development team insulated from the outside influences of marketing and sales. While this is quite common, it is dead wrong.
2. The windows do indeed exist into the strange world of development, but those in other departments don't take the time to look through them. Development may indeed be operating in a very transparent model, but if marketing, sales and others don't take the time to look through the window, they will always be surprised at the end of the project.
   
6. Developers are not Normal People. Managing software developers is counter-intuitive to most non-technical managers. The motivation is different, and most standard management techniques do not work for these "artistic scientists". Too often, when upper-management feels a project is getting off-track, they get more involved, and begin to take over the project by applying standard tried-and-true management techniques from the world of Sales, Marketing and other groups of "normal" people. These attempts to apply standard management techniques that would work with sales, marketing, customer support or other job functions, usually backfire and lead to a downward spiral of more failed projects, more non-developer management techniques to solve it, etc.

These are all fairly standard pitfalls, and I have seen them in many different companies over the years. The solution to these problems is quite simple, but requires a software development culture shift in order to make it work. In future posts, I will attempt to describe each of these in more detail, and provide guidance on how to create a software culture that removes these obstacles from your software development environment, which leads to projects that are positioned for success.

Why do we test?

The question has come up here at work on a few occasions: "Why do we test?" The idea being that when a tester finds a bug, it could get prioritized down below the "fix" line, and the product would still ship with this as a known defect. That being the case, they (the testers) sometimes express frustration, and wonder why they even take the time to find bugs if we are going to ship product with them. Good testers have a strong desire to ship a perfect product.

The problem, of course, is that every defect has to be balanced against the list of fixed defects and enhancements that have already been added to the product. With an iterative process in place, the product is " shippable" on a regular basis, but it is never perfect. There comes a point of diminishing returns, where the latest development version of the product is so much better than the current customer version, that we simply need to get the development version into customers hands. As long as we are not introducing new defects, and as long as the new enhancements function to a minimal level of acceptance and quality, then the product can go out the door.

So, why do we test? The answer is: "To mitigate risk." We need to understand what the current status of the product is, and what our risks are with releasing it, before we can even consider getting it into customers hands. We need to ensure ourselves that we are not introducing any new defects, or at least any new significant defects, in our attempt to get new features to our customers. Testing plays an extremely important role in this Risk Mitigation decision making process.

Of course, the other key part that testing plays is in the "test first" methodology. The testing department should know how a feature is going to work before the developer ever starts writing a line of code. In this culture, testing ensures that the features are implemented in the way that best meets customer needs and goals, rather than simply ensuring that it works the way the developer intended it to work. If the test case is written first, with input from the Customer Representative, then testings goal is to ensure that the new feature meets customer goals. No small feat, given the myriad of software in the world today that really doesn't meet anyone's need but the developer who wrote it.

Software Process vs. Software Culture

So, I have just finished reading "The Inmates are Running the Assylum" by Alan Cooper. I know, I am only a decade behind...but, I am actually very glad that I read it when I did. I have had a few years to zero-in on an Agile-based process that I really like, which I believe allows an engineering team to focus on short iterations while still adding truly innovative solutions to real customer problems. When used to create a culture of customer-oriented development iterations, it balances solving difficult problems with providing the customer regular releases (we have averaged a customer release every 5 weeks for the past 2 years at my current company - each one iteratively better than the last).

In the book, Alan Cooper talks quite a bit about the need for clear documentation, and how short iterations, which he classifies as 12 months, are not long enough to be truly innovative. This is the only major assumption in the book that I disagree with - but, had I read this before narrowing in on an agile process that works, I may have been deterred from trying the agile approach.

As well as our agile process has worked, the one thing that has always been missing is the true customer focused input to drive that process, thus creating a "culture" rather than simply implementing a "process". This customer input has traditionally be the purview of Product Management, with some PMs actually owning and designing the GUI in addition to their other job responsibilities. The problem, as Alan Cooper so eloquently outlines it, is that GUI "design" never happens in that model - it is usually already implemented by a developer, and the PM is asked to simply review the interface, re-word the text and maybe move some buttons around. It is almost always an after-thought. Alan Cooper calls it "dressing the corpse" - I prefer "putting makup on the pig". Either way,it flies in the face of test-first design, which is another key Agile concept.

After reading Alan's book, and taking into account the Agile concepts that we have already proven will deliver innovative solutions to customers, I now believe that I have the final piece to the puzzle.

I believe that if we could combine the Interactive Design and prototyping techniques described in this book with the Agile-based process of short, iterative releases and minimal documentation, we would be well on our way to developing a culture where customer needs are met very quickly, with high quality and in manner that not only does not stifle innovation, but breeds it.

The problem that this notion poses is the implementation. In order to truly create a software development culture requires discipline and agreement to the culture by every department that participates in the process - much more than just Development and Testing. Delivering high-quality products that meet real customer needs requires solid GUI design, test-first methodology and creative development to fulfill the vision, as well as management that is disciplined enough to keep the train moving while allowing the culture to take root throughout the organization.

Possible? Definitely. Easy? No way. Fun? You bet! Sign me up.

Protecting Families Online

I just watched a news story from Australia regarding the Internet and its effect on teenagers - especially those who participate in online social networks, searching for others with similar interests. It has caused me to think: When those interests are self-deprecating, how do we as parents find these early-warning signs that are so easily seen by their peers online?

In the story, they discussed filters. They also discussed the ability for children to bypass them easily - most technically-savvy children can. The role of filters today is misunderstood by many - or, more appropriately stated, people today expect too much from Filters. Many years ago when I was working on other client-based software, we had the exact same discussion regarding our licensing mechanism - what was its purpose? We determined that its purpose was to keep the honest people honest, and to provide them with a means to tell when they needed to purchase more seats. As a side-effect, we would also try to make it more difficult for those who wanted to steal the software - but this was not the primary intent of our licensing mechanism.

Filters vendors today find themselves in a similar situation - but with much higher stakes. The filters today are intended to keep unwanted content off of our computers. However, if someone wants to get around the filter and bring that content on to their machine, it certainly can be done. There is a balance between being protecting the computer and being obtrusive and stopping the normal use of that computer. Move too far toward locking down the computer, and people will complain that they can't use it for the things they want (it blocks too much, it slows down my computer, I can't access the Internet at all, etc). Move too far toward unobtrusiveness, and you leave holes for people to exploit to get around the filter.

If people expect a filter to keep a determined individual, child or adult, from being able to find inappropriate content, then they will be very disappointed. As parents, we need to constantly understand what our children are doing online.

Recently, I am more convinced that there are two problems with the current thought process for filters:

1. Most filters are focused too much on keeping pornography off of computers, and simply "monitoring" children's activity, and are not focused enough on actually pro-actively trying to keep children safe from the many new dangers of the Internet. We need technology that will help parents to know what their children are doing online, who they are associating with online, and to provide them with the early warning signs of potential dangers. As an industry, we are failing miserably at this today.

2. Filters alone cannot protect our children. There are so many dangers and pitfalls, and so many strangers waiting to make contact with our children, that a filter cannot stop all of it. Filters keep unwanted content from the machine, but they don't stop our children from talking about suicide with their friends, or finding someone online to open up their soul to when they don't think they can talk to us. If I have a filter on my computer, but my children's friends don't, then nothing will stop them from accessing whatever they want from their friends computer, or from the local library for that matter.

The CP80 initiative is an alternative to filters. It certainly provides a more permanent solution, and one that cannot be bypassed as easily (it is still technology, and there are still holes - people stole porn and movie channels for years when cable was introduced). However, even this solution is focused on keeping pornography and other "content" out of the home - it doesn't address the issue of truly keeping our children safe, and making parents aware of early-warning signs of danger.

What we need is a way to understand what our children are doing online. What social networks are they a part of, what blogs to they actively participate in, with whom do they communicate? How can we get a handle on all of their activities online, and help them to know when they are venturing too far? In the real world, we can see when they are getting too close to the street, or when they are wandering toward a strangers house. In the virtual world, it is much more difficult to detect.

While there is a great need for filters, and there will continue to be, the real problem is in how we get a handle on the vast expanse of the Internet, and what our children are doing out there. The world is truly flat today, and keeping track of what our children are doing in that flat world is becoming more and more difficult.

It is not an issue solely of technology, but of parenting in a new world.