As a self-proclaimed Agile Evangelist (Agilist), I am keenly interested in the process of taking an organization from a traditional project management base, into a truly agile model. There are a variety of factors that come into play that make the transition from traditional to agile more of a challenge than it first appears.
The Greek Philosopher, Heraclitus long ago, identified that the only constant is change. Who among us can dispute this? Even at our most cynical, or our most precise, the best you can do to debate that fact is to say it probably isn’t the only constant. But at its heart, the statement resonates with truth. We see evidence of change around us on a daily basis. Change happens, whether we want it to or not.
In the latter part of the 19th century, as the world shifted into the Industrial age, change came in the form of taking the fruits of the labor of individual craftsmen, and discovering how to produce them in mass quantities. Mass production of goods promised to bring down their costs. Profits in mass production relied on keeping costs of production down. In other words: the more efficient the process, the greater the profit. It is from this simple relationship that the science of process management was born.
This model evolved throughout the 20th century, and ingrained its way into every facet of labor/production. The philosophies of Frederick Taylor established a simple division between the classes of manufacturing. Laborers labored. Managers managed. As technology overtook us, a new breed of laborer came to the fore. The engineer was needed to design the process that would create the product. Unlike Taylor’s ‘simple’ laborer, the engineer needed to be able to comprehend the design of the product, and unlike the Taylor’s manager, they did not need to understand the laborer.
As machines became more complex, the mechanical clockwork that made them function became more and more intricate. Machines became capable of performing more than one task, simply by moving a selector into a different position. Industrialization continued, and soon we let the machines themselves join in on the task of constructing the products. The laborer shifted in role from performing the same simple task over and over (say, inserting a rivet in a specific hole), into moving the selector that caused a machine to perform the same task. The Engineer designed the machine that constructed the product. The Laborer operated the machine to produce the product. Management was now able to focus entirely on pushing the buttons of the laborer, and slowly lost the need to understand the process of creating the machine.
By this time, management theory had evolved to focus on new ways to squeeze efficiency out of the production process. Management of labor was a relatively simple task. Keep the laborer happy. Keep the work environment safe. Apply minor tweaks to the process and trim another couple seconds off the process. Stress the system a little. Find the inefficiencies. Time is Money.
Then there was the concept of managing the Engineers. Setting up a new manufacturing job or facility was time-consuming, but thankfully didn’t happen all that often. Managing engineers was a whole different experience and needed to be approached in a different manner. If you stress the designer, you may get a faulty design. A mistake in the design of the manufacturing tools was costly both in terms of time and materials. Engineering management was at its heart, most successful when the engineers were allowed to do their jobs properly. Speed didn’t matter if the result was wrong.
Industry evolved again. Labor didn’t even need to push the button any more. Decision trees were incorporated into the machines, allowing the decision of when to push the button to rest within the machine itself. The decision tree needed to be defined and implemented. The role of the Software Engineer was born.
The advent of software as a product introduced a new dynamic. Whereas a mistake in mechanical engineering was costly and required a large investment of time and material to correct, software was easy to correct. Consider a cam in a tool and die machine. If the cam is the wrong shape, the resultant piece cannot be used, and production would stop until a new cam could be designed and constructed. But if software took the place of the physical cam, a mistake in the instruction code was almost laughably easy to correct. The concept of the strict quality control, that was so absolutely necessary in the physical world, became less necessary in the virtual world. The concept of the “bug”, an error in software became an accepted artifact of software development.
As software became more prevalent, the need for some sort of control over the production of software, both in terms of quality control and time to market became necessary. Unfortunately, Taylor’s labor manager was ill equipped for the job. The Taylor school of thought involves breaking a manufacturing process into tiny steps, then optimizing each step to speed the process as a whole. Management’s role was to identify inefficiencies and monitor progress. But those practices were born out of the desire to maximize simple reproduction of a product. One could argue that they were never intended for the act of creating the original prototype. Software development has more in common with inventing (and sometimes reinventing) the wheel, than it does with constructing wheels.
Yet here we began, and here we stayed throughout the end of the 20th century and on to the present day. Organizations around the world, are struggling to use the philosophies of scientific project management to impose some form of control over what amounts to a sheer act of creation.
The study of Project Management has been honed to a fine instrument. Software Project Management is an art in its own right. The most successful project managers are the ones who understand the process and the people enough to account for change. This accounting manifested itself in two ways. First, it attempted to reduce or control uncertainty by demanding better up-front planning. And second, it accounted for inevitable change by adding buffers into the schedule. The really good Project Managers didn’t find ways to eliminate uncertainty, and they didn’t find ways to mechanize creativity. They found ways to disguise the buffers.
Mind you, the mere existence of a buffer to account for uncertainty is a violation of Taylor’s philosophy—a fact that was not lost on senior management, who were still tasked with focusing on the bottom line and maximizing profit. This set up a basic conflict in management strategy. Project Managers, trying to produce reliable schedules were trying to add just enough buffer to avoid setting off the alarms of Taylor’s efficiency experts, but senior management, many of whom rose through the ranks of project management knew about the buffers and sought to eliminate them. Any act of forecasting a software schedule became less and less about good accounting and estimating, and more about good negotiation.
It must be said, that this concept of manufacturing theory applied to software creation, tempered by uncertainty buffering and close scrutiny did in fact work. Imperfect as it was, the combination resulted in successful delivery of product on what appeared to be a reliable schedule. It is this appearance of success that has dogged and hindered agile adoptions around the world. “If it ain’t broke, don’t fix it.”
Over the next few months, we’ll be exploring the ways in which the existing traditional product development cultures are hindering the very development efforts they seek to optimize. We are going to discuss the philosophy of Agile methods, and how they run counter to these traditional methods, and more importantly why they must run counter.
Agile adoption, for those with the courage to embrace it, offers a more natural model that not only recognizes the limitations and contradictions of the traditional methods, but offers real solutions to improve upon them. We will discuss the Agile Principals as they apply to different aspects of your organization, and how those principals will result in changes you haven’t even considered yet.
We will begin this discussion with the concept of Culture Shock—the first challenge you will experience in your adoption, and ultimately the most important indicator of how your adoption is likely to progress.
