Tuesday, December 8, 2009
There was one question that caused me to pause, unfortunately the woman who asked the question came up to me latter and apologized for putting me on the spot. Reality was it was those kind of questions that motivate me to give such talks and I thank her. If I can find her contact info I will forward this on. I probably didn't give the smoothest answer; so now sometime latter let me try again.
As I recall the question went something like this- "We've been trying to adopt an agile approach but we have a problem with one of our established metrics that people track. That metric is the days a bug report is open. With sprints sometimes we don't get to fixing a bug for many weeks and this is being raised as a problem with our agile adoption."
My quick answer is you should be considering resolution of the bug as part of sprint planning every 2-4 weeks. If the product owner and team decided that other work was more important than that's where the priority is. I recommend that you make sure all then necessary stakeholders are participating in sprint planning.
The longer story I didn't have on the tip of my tongue or the time to deliver goes like this. First when a team begins the agile journey they tend to carry over some habits or patterns from the waterfall world and just look at agile as a series of small waterfalls. They look at the work in a sprint as just the development tasks and don't establish any quality goals or metrics that help define what it means to be done with work in the sprint. If they are sprinting along and building up a bunch of bugs that aren't getting fixed then the person raising the yellow flag about the increase in duration for open bugs is raising a valid alarm. In that case they have sort of a cafeteria style agile where some practices are adopted and others are not. This is a common problem and one that the team should recognize and deal with. A healthy team adopting agile should not be building up an increasing stack of open bugs. Somewhere there should be the idea of "potentially shippable code" every sprint.
Second in the agile world one needs to look more carefully at what is means for something to be a bug. Let me describe two examples. In case A when someone clicks the OK button after a certain set of steps the system responds with "Access violation" or "Invalid object reference" This is clearly a unacceptable response and the result of some type of implementation error. This is a classic bug. However consider case B when someone clicks the OK button the system responds with a message indicating "no x selected". The user looks at the screen and there is only one choice for things of type x. They may think something like "why do I have to select black as a color when black is the only choice available." The program should be smart enough to realize that and make the selection automatically. This may also be consider a bug.
In case B the problem is most likely due to a flaw in the requirements or a use story. It is sometimes referred to as an emergent requirement. It was a difficult requirement to specify until the customer had some experience with the system and could actually see the problem. The bug represented in case B may be something that every recognizes is a bug but the severity or consequences are not as great as case A. The team and product owner may prioritize other work to be done before working on the bug like the one in case B. If that's the business decision then that's how it should go. Maybe the frequence of the user encountering this case is very low. The bug in case B may stay open for several sprints or even across multiple releases. The key point here is an explict decision (or multiple decisions) was made not to address case B in favor of something else that had higher business value.
Assuming these tradeoffs and decisions are being made, the the question of the standing of the traditional metric bug open duration is called into question. One would have to consider if the metrics truely supported governance of the development process or if they are a carry over from a different process? Hence it is hard question to answer with out context.
Monday, October 12, 2009
First while browsing the agile virtual community I encountered a reference to a youTube video with the headline "Learn the Agile software development methodology in less than 10 minutes. "
Second I was reading the abstracts of talks by some thought leaders that offer to clear up the multiple misconceptions about Agile.
Third, I was thinking deeply about adoption of Agile in my current situation.
I was initially impressed by the 10 minute video, it gives a nice concise definition of some common vocabulary and roles used in an Agile project. Yet, I fear it also contributes to the misunderstanding that experienced agile practitioners talk to others about. It may also lead executives with P/L responsibility to look at Agile methods with less respect and confidence.
I believe that there are also others who started their "agile journey" more recently or at a different place who tend to view Agile methods as light weight or more suited for the free spirited. They may be attracted to Agile methods as the least invasive of a variety of alternatives.
The more seasoned players my look at all of this with some degree of skepticism. They think how can something that simple support the claims being made.
In light of all this I have to offer my brief point of view. The process description is simple. Yet the organizational climate in which that process can thrive is no simple matter to achieve. Getting people to be accountable is hard. Getting out of the "it's not my job" mentality can be a challenge way beyond what one can learn in 10 minutes. How to create the foundational alignment and climate is what I'm learning and hope to continue to learn.
To do this I view presentations on motivation by people such as Dan Pink http://www.ted.com/talks/dan_pink_on_motivation.html and I listen deeply to team members who challenge me with statements like "why isn't it enough that I come to work every day and do good work".
The "agile manifesto" or the "declaration of interdependence" http://pmdoi.org/
are value statements not process prescriptions. The blood, sweat, and tears to create that type of alignment is how you pay your dues to achieve the benefits that proponents of agile methods offer. (faster, fewer defects) And in some cases your organization may not be able to fully implement an agile development methodology. It is not the "best practice" every one should adopt. In the end we must do what is necessary to create value for our organization in the current time and setting.
If nothing else agile methods should cause one to reconsider the value that any current practice or communication process brings to an organization and consider very carefully how "change" impacts the success of failure of your project.
Agile is not simple, and certainly not a quick fix. Rather it is a challenge and a cause to rethink and evolve our current current management and project management science.
Monday, September 7, 2009
How ever no one would consider it an English garden with neat rows of precisely placed and cultivated plants. The ivy ground cover grew up and covered the trunk of the candy apple tree. I never planned that but it looked nice so I kept it. At the edge of my garden I noticed that one of the sunflower seeds from my bird feeder was lucky enough to germinate and grow into a sunflower plant. A more particular gardener would have pulled out this plant as a weed growing among the ground cover.
This was a special summer. I got a new daughter in law, Heather and she loves sunflowers. So I let it grow. It grew almost 4 feet tall and does not belong among the 6 inch tall ground cover. But it is like an unexpectd gift that has evoked many special memories. The wedding has past and the three blooms have gone to seed. I have been thinking about pulling it out as it still looks really odd. But sitting at breakfast I got one last kick from this plant. I just enjoyed the entertainment of pretty yellow finch trying to figure out how to get some of the seed from the big downward facing seed had that has lost all its blooms. That was a garden pleasure I would have missed if I put the sunflower in the compost heap when the bloom had passed.
This whole experience reminds me of some of the unexpected pleasant surprises I've experienced introducing agile project management in a setting where I worked many years ago.
I was leading a group of about a dozen developers and a handful of testers and writers in a group that was growing rapidly. I was serving as development manager and architect. But I was burning out and realizing that I was personally becoming the bottle neck of the development organization. It was this situation that first prompted me investigate and latter introduce FDD Feature Driven Development. My team was familiar with iterative development but this was the first time I used what is considered one of the real agile development methods. This was two years before the agile manifesto. My past colleagues at TogetherSoft will recognize the appendix in the object modeling in color book as our bible on agile. The ground was fertile for this agile seed to grow. I was blessed with good people to work with and adoption of FDD went well.
One of the key attractive points about FDD was it enabled us to scale and run multiple feature teams as the same time. I saw this as a way to enable my group to grow and adapt to the multiple ideas that were coming at us from our sales force and other technology partners. Yet it also provided me a framework to monitor and when necessary become involved and pull a "weed or two". See http://en.wikipedia.org/wiki/Feature_Driven_Development for more info on FDD.
But during the introduction of FDD to this group I had a pleasant surprise that I didn't really expect. I explained the process to my team and the various rolls I monitored and let folks pick up the various roles and responsibilities defined by FDD. I encouraged as much self organization as I could stand. Some reasonably expected things happened regarding who became class owners. But what I didn't expect was a young reasonably inexperienced engineer pick up the role of Feature Set lead. Before I decided to introduce FDD I was considering breaking the team up and designating leads for each of the sub teams. It never would have occurred to me to select this person for any leadership role. But to may amazement she took off like a star. She really blossomed and took over making may of the decisions I didn't have time to do. In many ways I think what she did was very much like the sunflower in my garden this summer. I hope she is doing well and I am grateful for what I have learned and continue to be reminded about the limits of what I can really control and what I need to let grow.
Sunday, July 19, 2009
First -was in the book "How we Decide" by Jonah Lehrer. A fantastic book I strongly recommend to any one who thinks about how they think and what makes people do things. Near the end of the book he has an
antidote about a pilot and an crash that led the aviation industry to a process called cockpit resource management. (CRM) I believe that CRM is to the Aviation industry what Agile is to the software development industry. Unfortunately we've had many "crashes" in terms of projects that fail. Interesting thing is Lehrer goes on to talk about how CRM has gone on to the medical field as a practice that has yielded great benefits in the surgical operating room. And that was just one facet of the book. I believe that working on an Agile team or with a CRM culture is just so intrinsically satisfying. I just wonder why I feel that way about it that I find so cool. The accomplishment, the connection with others.... Humm. Perhaps a follow on book for Jonah Lehrer. A great example of what I'll call the shared owner ship pattern.
Second - last night I attended a Woodstock anniversary concert with the Cincinnati Pops and the rock band "Jeans n Classics". Very enjoyable and very talented people. What really struck me was how the Jeans n Classics folks all supported and complemented one another. They had a leader but each artist had special time to shine and show their own talents. The gelled as a team. Each song was a sprint and the concert was a project. I enjoyed the music and nostalgia. And I also admired the interdependent team. Each one contributing their particular talent. Usually at such a concert there a single star that is a focal point. This was a set of interdependent artists. Just like I'd like to see of a team of software developers. This is a great example of the interdependent pattern.
I think I'd like to develop this pattern idea some more. It helps to unify many of the ideas I've been bouncing around especially regarding the application of Agile to distributed organizations. More for another day.
Wednesday, June 10, 2009
Speaking to a friend today we commented that there seems to be a increasing popularity of seminars, lectures, and classes related to some Agile agile method or practice. I fear a coming backlash or pendulum swing in the opposite direction as various folks pretend to adopt an agile method, get into some type of trouble and then blame agile as not living up to the hype.
My analysis is based on two fundamental beliefs. One, is the amount of formal process you need is inversely propositional to the experience and skill of the team members. Inexperienced folks need more formal process and very experienced folks need less process.
Two, an agile process will expose problems and issues with the project more quickly than a more traditional development process. What to do about those problems and issues is not always very clear for an agile team. The natural conclusion here is that serious consideration should be given before the leadership endorses the adoption of agile for an inexperienced team.
Now, but what about the adoption of agile for a team operating under stress. Perhaps that team is comprised of experienced people but due to some pathology in the organization such as "mushroom management", fear, mistrust, wide geographic distribution or some other organizational challenge. The folks on this team have a good chance of identifying what are the obstacles but they may run right into a brick wall. These walls are may often be "organizational constraints". They may be political or geographic obstacles that are beyond what any development methodology can overcome. Consider the following:
- A project sponsor that doesn't have the depth of experience to understand the challenges of a software development project.
- An organizational culture where there is rift between the users and the developers that has made mistrust and suspicion the norm for the last ten years.
- A team that is spread across 3 continents. The communication challenges make it difficult to exchange technical information or to even build a climate of trust and cooperation. With the Internet and modern collaboration tools it is possible to mitigate the challenges but not to eliminate it completely.
When a team in such a situation attempts to adopt and agile method under these circumstances failures can often be blamed on the Agile method. "It just doesn't work" or "I don't believe in things like that" is often the refrain. When in fact the failure is to properly consider the environmental and organizational factors. Identification of these factors is right out of the PMI PMBOK. Or consider the CMMI based discussions on tailoring of development processes.
My sense is the key is developing the guidelines and science to consider the "environmental and organizational factors" or the skill in tailoring a software process to a particular situation. Then we will come to view an Agile practice such as Scrum as a very nice development method but not a one size fits all solution for every development team. When going agile I'd look for heavy customer participation, high bandwidth interpersonal communication between the team members, and a organization where trust can flourish and grow.
Thursday, May 7, 2009
It is a bit of an apples to oranges comparison, today I was really struck with an benefit for the agile approach that haven't really appreciated. On the classic project there is a distance or an understanding gap between the formal project plan and the people doing the work. On the classic project many people just don't have the time or care to become familiar with the schedule or the project plan. If they do, they are primarily just looking at the end date and maybe what is on the critical path to the end date. It is a project manager's job to drive the planning process and make the team aware of what is in the project plan. But most of the time they really don't care beyond the success of the project. They have developed some kind of understanding what what needs to be done and that's what drives them. Especially in a matrix organization, the project manager is often some one running ahead of or along side the project trying to keep the plan aligned with the reality of the performing team. In a matrix organization the resource manager will have language and vocabulary for talking about what needs to be done, it should be but it is often not exactly how it is described in the project plan. A good project manager will have some standards for describing tasks or doing the work brake down structure. This may be very professional and orderly but it is often not how the resources would typically describe the work to be done. IOn top of the semanict and vocablary issues the plan may be captured in a tool that not every person has due to license cost or just convenience. Consider a team of 12 where on the only the project manager and the resource manager have a tool like MS Proejct Professional and their dest top. The other 10 on the project get PDF reports, PWA access, or get to watch the project manager use an overhead projector to display the plan on the wall.
Some resources may passively resist the regimentation and structure that comes from working on a formal plan maintained by a professional project manager . I believe that all of these factors contribute to the gap between the plan and the resources.
On an agile team we have user stories that are written by a variety of people on the team. There is a template "As a
So in the end there are two alternatives each with their Achilles heal. A coherent professionaly maintained work break down structure that is some what distant to the resources doing the work. Or, a more chaotic structure that the resources doing the work readily understand. Generally I prefer the latter. I'd rather have a looser "plan" with inconsistnecies that people really follow than a really tight comprehensive plan that people doing the work have not internalized or even do not readily understand. In the end it is the outcome of the project that matters. On top of just understanding, there is the ownership issue. By far and away I believe that people work harder when they feel they own the plan and they have personally contributed to its creation. This ownership feeling is fosterd much more when they actually press the keys that make the plan artifact what ever form it happens to take. A perfect plan can never be achieved, however a team fully comitted to a successful out come is much more valuable than a perfect plan. In the end it is the outcome that matters. I'll trade almost every thing to get a team that owns the plan.
Sunday, April 19, 2009
I am very close to two very important people who will be getting married latter this year. I could offer my project management services to this couple the way a photographer or a florist may contribute. They have been busy planning and it seems that they have a good handle on the tangible aspects of the day. I worry still that all of this planning may ironically become an obstacle to that perfect day they both so richly deserve. Rather, I think a more valuable offering to my son and his future wife and any other couple planning a wedding is this adaption of the agile manifesto.
A wedding is a momentous event where family and friends gather to witness the vows of the bride and groom and the start of a new life together. It clearly needs a lot of preparation and planning. This is especially true when there are out of town guests and considerable financial expenditure is involved. All that planning, especially when the bride and groom are in the center of it all can take on a life of its own and over shadow the really important things that day. Through my wedding and being a guest at many other weddings over the years experience over the years I have come to value:
• Family, friends, hugs and tears over reception hall/church guidelines and “The Knot “ check list
• Precious memories of a happy day over hours spent posing for photographs
• A multi family & multicultural celebration over a party planned by a few people
• Being present in each moment over worrying about what has to happen next
That is, while there is value in the items on the right, I value the items on the left more.
I find it relatively easy to teach and mentor people about the classic aspects of project management what I struggle with is imparting the wisdom I have gained over the years about what is really important. That seems to be true for a wedding as well as many other endeavors. How to find the right balance is the central question.
Wednesday, April 1, 2009
What I fear is encouraging people to pick and choose from a set of Agile methods and then get in deeper trouble or further spread misconceptions. I believe processes like Scrum are the result of considering a particular development scenario and then putting it through a lean process filter. The outcome is a small tight set of processes where nothing is optional. (retrospectives, user stories, close customer collaboration, etc. ) Where some one to cherry pick a few of those processes and then claim to be doing Scrum it could be a big problem. Hence I think the difference between SCRUM and agile is like the difference between a set of values and a coherent integrated set of processes. Values are more abstract and processes are more concrete. It is like democracy and the government system of a particualr country. Agile is to democracy as Scrum is the the UK system of government.
On the other side of the coin I consider the multiple places I have worked or served as a process mentor and despite what ever label they apply: RUP, extreme programming, FDD or scrum there are still considerable variations among groups the profess to be following the same process. I believe that we tend to believe that things are more alike than they really are for groups who profess to be following a certain process. We under appreciate the tacit and implicit customizations that each group adopts. These adoptions are critical success factors.
Bottom line there is no magic bullet best practice. The art and science is in the tailoring and adaption of the broad range of best practices as identified by SEI or PMI with a values foundation as expressed in the Agile Manifesto. People over process, working software over extensive documentation, etc. Think like an agilest but make sure you have a story for dealing with scope management, risk etc.
I thought I was pretty smart 10 years ago when I had 20 years of experience. Now that I have 30 years of experience I realize how difficult this is and I have even less confidence. What I do appreciate is the wisdom I draw from all schools of thought and bodies of knowledge. Comming up with the most appropiate development and project management process is a very challenging endevor. The good news is there is now a lot of wisdom to draw from. Good Luck.
Wednesday, March 25, 2009
My best response is that we choose not to reduce the risk of undertaking a complex software development effort by writing detailed specs and requirements documents. Instead we choose (or I believe we choose) to reduce risk by incrementally create function and create frequent builds of working code. That is our risk reduction strategy. It is a corner stone for our plan to deliver a successful end product. I acknowledge that their is some extra work to keep things working - perhaps with some temporary scaffolding or temporary code. This enables us to demo the code and learn from it over the life of the project and not have a big crisis at the end and a long difficult to predict period of instability.
Next time I'll ask do you want to write specs and that start or keep the code always running and fix the bugs as they come up. You can 't have your cake and eat it too. We are all on a journey...
Wednesday, March 11, 2009
In a "traditional" project teams often work away according to a plan. Very often things don't go as expected and one of the places that suffer is any time planned for bug fixing. Or there is an accepted sentiment that "...they are just bugs and we can fix them at the end after all the hard work is done" we've got to stay on schedule. (In my experience more often than not a significant design flaw or requirements snafu is uncovered in the process of bug fixing and these issues are really harder than building new function. These are dirty problems people tend to avoid unless pushed by an experienced manager or agile mentor. )
Some teams even may have a bug review with the customer or product manager near the very end of the project. These are often difficult as during this review the only choice the customer has is to extend the schedule to fix the bugs or accept the bugs as known issues to be fixed in the next version. Hence they tend to want to avoid these sessions as they only present dilemmas not an opportunity for active decision making.
Some teams consider it OK to build up a large bug count. They also probably consider it OK to keep their charge balance at or near the credit limit. I recently pointed out to my lovely spouse that despite payments of a 100 dollars a month the balance of on a certain credit card had only decreased by 60 dollars in the last three months. In my minds eye that's almost 80 dollars a month wasted that could have spent on other things on tangible goods rather than interest payments.
The team that runs with a big bug back log faces the same waste. Granted there is the "cost of quality" analysis. We are not going to have a perfect product and fix every bug. But if in the course of doing business we make accommodation like:
- Skipping that error when it is reported again and again in the automated test run.
- Talking to one another via speech, email or instant messenger,or what ever about bugs that hang on for months.
- Some one else rediscovering the bug and attempting to report it only for the bug report to be deemed a duplicate.
- Reading past the old bug in the bug report when looking for new bugs. Considering what other bugs might be resolved by and open fix. A large open bug list makes the incremental cost of every new bug larger as the database grows.
- Writing up the bug as a known issue for the read me or release notes when we finally complete the project and didn't get time to resolve it.
- Not discovering other bugs that are masked by the bug due to people avoiding that functional area where the known bug was reported.
How do I cope.
1. Be vigilant and visible with test results. Keep them visible and on going.
2. Raise the threshold for what is considered a bug. Sure it would have been better if it worked that way.... but that wasn't in the user story and never discussed till now. Can we plausibly just make that part of user story backlog.
3. As part of the post sprint review, consider the new bugs that are still outstanding. Decide now if we can ship or put the product into production with these bugs. Don't be a fence sitter. It is OK to have up to 20% as "maybes" but if the customer/product manager can't accept certain bugs recently introduced then they get fixed before new user stories are added. This is part of what it means for each sprint to produce a potentially shippable product.
4. Remind the team what a crunch they use to have at the end of other projects. Use that as a motivator to not put things off and not be overly optimistic.
If I'm not running my credit card debt near the limit. I can be agile and make a decision to take advantage of great sale on new patio furniture or I can redo the basement the way I'd really like to after suffering from a flood. If I'm agile I pay as I go and I have choices. I have head room to respond when the unexpected happens. I can take advantage of business opportunities arise because I'm always just weeks, not months away from shipping or going into production.
Wednesday, March 4, 2009
Going back to the Agile Manifesto, it is primarily a values statement. If you don't buy in to the values you'd really be better off with a cook book approach. The teams I've met and worked with that are truly practicing agile have been and are very driven by process. Ironically this is the opposite perception of those who don't understand agile and speak negatively about it. I met one of those people the other day. He started the conversation with "I hate Agile it cost our company x dollars on a project we did using that some time ago." He elaborated "when we got to the end nothing worked and there was lots of finger pointing. " Our conversation on Agile was tangential to our primary discussion and it wasn't the place to start evangelizing about agile. I should have been quick enough to key in on the "got to the end nothing worked part" My gut feel is that perhaps it was one of those loose type processes that masquerade as an agile project. Instead I said well "Agile isn't appropriate in every situation." That is also true.
And here's the part where I make my self feel better. What I needed to do in that situation was build a new relationship, not get on a soap box about Agile. So I was living the values statement of putting people ahead of process. In the end I can only hope that if I can influence people by being genuine and living the values I profess. And, throw a few hardballs like I did today when I called our team on letting the bug backlog build up.
Monday, February 16, 2009
We've tried a few way's to make up for the daily scrum. Communication is really our big challenge. One of our first attempts was to hold a virtual daily scrum via a customized share point message board and a recurring Outlook appt. This worked well enough that we got good participation and people were itchy to improve it. The idea was that every day as close as possible to 8am Eastern time every one was to post a message to this share point discussion board under the root item of the current day such as 14 Jan. I figured that the folks in India would participate at the end of their work day and the folks in US would participate at the start of their work day, the folks in Europe well they could pick a time in the middle of the day. Well I was really rather ego centric. I didn't realize how self centered I was being based in the US Eastern time zone. This model of communication we really best for the US participants. It didn't make sense to be almost at the same time.
A thought I had for a while was what about all this social networking stuff. Can't I apply some of that technology to make my distributed agile team more social? One of my first thoughts was to use Twitter to host our virtual daily scrum. One our managers, rightfully so raised security concerns with twitter messages disclosing sensitive "intellectual property". So when we discovered Yammer it was something to get excited about. Yammer.com is like a Twitter for people at work. We've been using Yammer now for about a month. I've seen some discusions and disagreemetns arise from Yammer posts so I believe it is working. Very nice tool and thanks to the folks behind Yammer.com.
Wednesday, January 21, 2009
I was also pleased to hear another assertion that the number of people who are in the Agile camp and believe Agile is impossible in a distributed environment. Many of us in the audience supported the position that distributed organizations are a fact of life that we just can't change. And, that is the central question of this Blog. How can we best make Agile work or adapt pure agile values to a distributed environment rather than just saying it doesn't work.
Wednesday, January 14, 2009
My hopes here are to at least get the benefits one achieves by having a personal record my thought processes. Maybe even I'll get lucky and connect with others who are trying many of the same things I am.
Just a small step for tonight. I've decided to start.