A System User Story Holy War? On a Saturday night? :)

I’ve struggled for a while with the concept of System User Stories. I know about the debate and argument that about only User Stories should be used to measure progress. And I agree 100%. I am not proposing that these System User Stories would be used to manage the project and measure progress, but that they would be used to validate and confirm the system design and completeness of User Stories. In essence, they don’t remove the need for User Stories but are an additional activity and deliverable. I also feel they are an invaluable light weight model that documents System behaviour. I feel the lack of Application or System definition between User Stories and Test Cases and Code is just too broad of a gap. There is room for a light weight model to fill that gap and ensure our solution has a high level of completeness and cohesion.

In my definition, these System User Stories are stories that define how the system components are interacting with each other at a high level. They have been attractive for me for primarily three purposes:

  1. To examine the full behaviour of the Application/System to confirm that the design and concept is full fleshed-out and complete.
  2. To create a light-weight model to capture the Application/System behaviour for communication through out the team.
  3. To confirm that we have the appropriate User Stories and tests across the entire Application/System.

One thing that has always troubled me about User Stories is that unless there an end-to-end light weight model of the System, we may have the system architecture and design affected when User Stories are worked on in later iterations if the functionality starts to diverge from previous understanding. Having a light weight model of System User Stories that we can then validate Traditional User Stories again greatly reduces this risk and highlights any areas not covered. I think the System User Stories have more value for complex systems. I am currently working on a very complex system, and I feel System User Stories were essential to ensure our design and scope was complete, correct, and compelling. Since the project is somewhat R&D we also lack a Traditional Product Owner so this task falls on the project team. System User Stories helps us to clarify this concept.

So given the fact that I think they are required, what format do they take?

What I have found to work is that they should be in total no more than 2-3 pages in total. Any more than that and you are going too deep and not just confirming overall design and consistency. You are actually doing requirement definition.

I also like the User Story type of format. It is complete and concise. The format I have used is:

As an [Domain Specific Language Component] I will [do a Domain Specific Language Action] when [Event] using [Domain Specific Language Components or System Components

The Domain Specific Language Components and Actions can be anything from true DSL Components or Actions to just the abstraction to the high level components used in the System or Application. It definitely should not be at a low-level. I also have broken these System User Stories into major features just for easy categorization and organization.

And then finally I renamed these System User Stories as System Grammar. I did this to shy away from the System User Story holy way and also to reflect that these rules truly are the grammar that the system uses to build its language of functionality.

System Grammar. I think there is a place for it.

A Testing Phase – I just can’t quit you!

Finally I’m going to write about the problems I have had moving away from a sequential testing phase. I got side-tracked with some Monster analogies, but before I move on to discuss any other topics I need to fulfill my promise about writing about this.

To summarize, no other concept is personally easier to state and harder to do than a sequential testing phase. Even when I am coding or creating, my mind always thinks about creating sufficient inventory to then make the context switch to then test. Even though I know this is incorrect and I am losing quality and building inventory by doing this, I seemingly can not help myself. I was thinking about why this is and I may have a theory.

I wonder if this default behaviour on our projects is due to how we were educated? All through out our years we had the pattern of building up a sufficient inventory of knowledge in classes and then having a large test or two to validate that the knowledge was indeed mastered. I think back to university and my Mathematics exams worth 90% of the grade, the ultimate big bang approach. No concept of failing fast in those courses. Sure there was some assignments to practice, but even that was not consistent across all the courses. Ultimately most of my courses after Junior High had an exam of at least 60%. And of course the problems that caused were very similar to project issues:

  1. By the time you know there is an integration issue it is too late to do anything about it.
  2. You cannot correct mistakes early so if something is misunderstood the number of mistakes is huge.
  3. These large tests, don’t provide opportunities to learn. They only provide judgement.

This last sentiment has stuck with me the most and I remind myself of this fact to ensure I test as early as possible. I remind myself that the noble purpose of testing is learning, not judgement. With most traditional projects, the testing phase is almost viewed as a judgement being pronounced on the project declaring  whether it is successful or not. Almost like we are grading the project and determining if it has to go to summer school. We need to fundamentally shift our focus in testing to be that it is an opportunity to learn and not an opportunity to judge.

When you do this shift, something very interesting happens. The value of a Quality Assurance Departments and Testing groups lessens greatly. Those structures are all about judgement. If testing is about learning, then the people conducting it only naturally should be the ones that will learn the most! And that would be the developers and clients. (Hopefully together!)

I believe our educational system has now moved to more frequent, smaller tests to ensure that the tests are about learning, not judgement. It seems only now has our Information Technology projects also understood that it isn’t about judgement.

I would propose that rule #1.a of Software Development should be:

‘Thou shalt test the deliverables before each and every status report is given and status meeting is held’ (this would be daily but weekly at the latest’)

This would ensure that testing is not a separate phase and that testing really is a learning activity by being integrated in the entire lifecycles of the project.

And really, how can we report on status if we haven’t tested????

Any guess to what Rule #1 is? Share your thoughts and I’ll tell you mine on my next post.

User Story Mapping – Oscar for best presentation ever!

I now know why a colleague of mine at Protegra has been raving about this User Story Mapping presentation from Jeff Patton. I have read up on the concepts but until now did not fully review the presentation.

Simply put, this is the best presentation I have ever seen both in style and content. Awesome.


360 Performance Feedback “in the round” – Abolish the EAT Phase!

performance feedback is one of those issues that seems to confound Traditional and Agile projects alike. Even with Agile projects, sometimes the Performance Feedback gathering is left out of the retrospectives. Frequently the Performance Feedback is left to the end of the project or at major milestones to gather feedback.

I have worked at Protegra for over 8 years. Protegra is an extremely collaborative company that embraces the principles of Lean (& Agile), but we have also struggled to capture high quality, timely, and meaningful information to assist in Performance Feedback. I played the role of Delivery Manager for over 4 years and we frequently struggled with trying to counsel people in regards to Performance Feedback for whom we had little information on. Since we are a project based company and we fundamentally do not believe in supervisors or managers, we also did not have roles that were solely responsible for gathering and delivering feedback. It was the responsibility of each and every team member to do this. So what did we do?

Just like every improvement, we did this through a series of small improvements. Rather than go through each improvement, let me recap where we are now. To summarize, this resulted in three areas of change:

1) Reinforce the terminology of Performance Enhancement to fit the intent and communicate the reason behind gathering the information

We have found that language is very important to communicate the true value of a process. Many people viewed Performance Feedback as being required for compensation. We re-inforced the language of Performance Enhancement and stressed it is about helping people improve on the competencies they want to improve on and that are  important to the role. It is not a cookie-cutter approach where we fit people into a role description and rate them accordingly. This was a fundamental first step on the change. (And it is a continuous effort)

2) Create a Technical Performance enhancement Framework

The next step was perhaps the most difficult. We created a framework that recognized the true competencies of our culture, roles and team members that we are aware of. An important distinction was that these competencies were for our culture and roles and not positions. We don’t have positions at Protegra, but rather a variety of roles anyone can play on projects. Unlike other Performance Enhancement systems, this is a constantly evolving collection of competencies as new competencies and skills become valued at Protegra. Another key aspect was to ensure that the competencies also had objective criteria to be able to help assist people in the evaluation of the competency. Instead of just stating:

“Mike is a good .NET developer and has achieved an intermediate level of Programming expertise”

The competency framework will prompt the reviewer to evaluate competency aspects we incorporate in the terms “good” and  “intermediate level”. For example, for the role of a Software Developer on a project these would involve providing grades on such aspects as:

– Use of SOLID design principles

– Use of Standard Design patterns

– Creation of Change tolerant code

– Creation of Automated Test Cases and test coverage provided

– Among others

This level of information is gathered on less frequent basis due to the amount of effort required. It is very important, but we still needed to capture information quickly and frequently that could feed into this structure.

3) Hold Retrospective Performance Feedback sessions “In the Round”

The most important aspect we then implemented was holding Performance Enhancement sessions “In the Round” as part of each and every retrospective. Unless the Performance Enhancement is incorporated into the process of the project, it will be left behind. Although people were apprehensive at first, as they become comfortable with the process and more comments and feedback was gathered. To be honest we have only recently implemented this process but the results are quite astounding. In this setting we also asked people to consider the following questions when evaluating their teammates:

– What can they do more of?

– What can they do better?

– What can they do less of?

– What can they do differently?

It is still a work in progress but the results are going in the right direction. I believe our approach again aligns with the Agile approach.

The largest realization I had was that holding Performance evaluation Sessions at the end of a project or during a considerable period of time is EXACTLY like having a User Acceptance Testing (UAT) phase. But we were rather having an Employee Acceptance Testing (EAT) Phase.

We need to attack the elimination of the EAT Phase is projects like we have eliminated the UAT phase. This validation of team members needs to occur daily just like our testing!

Building a better monster – Naturally Agile

When we last left our intrepid Project Manager (Evil Scientist) he was fighting off the hordes of rioting Business Users and negotiating change requests. Not a good scene and something that could certainly be avoided.

It turns out that the answer to how we can build a better project has been done forever by nature. Nature in itself is intrinsically iterative and abhors big bang implementations. (except for ‘the’ big bang, but that is a topic for a different blog entry)

To start at the beginning, let us review what the evil scientist was trying to do. The very objective of the project had two essential criteria:

  1. To create life
  2. To have this new life follow his direction

Each of these essential criteria has a complementary concept in the Agile world. Let’s discuss both of them in sequence.

1. Creation of Life

As you can probably surmise, my analogy for being able to manage a project is exactly how nature creates life. The creation of life project has the following external milestones:

  • Visioning at Conception
  • Implementation at Delivery

Now there is an immense amount of work being done between these two external milestones and thanks to modern medicine we have more of an eye to this progress than previous generations. Although the creation of life is a 40 week project (for humans), the following systems are initially formed at the following milestones:

  • Nervous System – 8 weeks
  • Circulatory System – 4 weeks
  • Skeletal System – 6 weeks
  • Respiratory System – 6 weeks
  • Digestive System – 6 weeks

As you can see, although the implementation date is far in the future, nature really delivers initial functionality on all major sub-systems 8 weeks into the project. (I know I am hugely simplifying this, but I’m hoping you will cut me some slack. :)) So by 20% into the project nature has an end to end prototype (pun intended), that nature can then add subsequent functionality to before the final release. Some of this additional functionality is the refinement of these major systems which includes:

  • Full Skeletal Structure – 1o weeks
  • Hearing – 17 weeks
  • Breathing of Fluid – 28 weeks
  • Ability to Suck Thumb – 28 weeks

One important piece of information is that from weeks 33-40 it is reported that there are no major qualitative changes. During this period there is just more refinement and ‘hardening’ of the baby. (And possibly some refactoring?)

Let me leave you with two questions:

  1. How much better shape would our projects be in if we had an end to end prototype built by the time we are 20% into the project?
  2. How much better shape would our projects be in if we were only hardening our solution in the last 20% of a project?

2. Project Management Style

The other comment I would like to make quickly is that the Project Manager style of the Evil Scientist is definitely ‘old school’. I fundamentally believe that the new Project Manager style is more of a parent. Now I do not mean this to imply that the child project is beneath the parent in knowledge at all. But that the Project Manager is meant to facilitate the learning and growth of the project. And by doing that the project will also teach the Project Manager. At the end we really do need to look at projects as not something to be led, but rather as something that learn, grow, and succeed.

Perhaps we should term this concept Embryonic Project Management? Should the 20% milestones nature uses at the start and end of the project simulated in a project?

Stop the Madness and Frankenstein Projects

My apologies for the delay in the posting but I was under  the weather and also basking in the glow of the Packers SuperBowl victory. I know I stated that the next post would be on why the testing phase is so hard to let go of, but during a discussion with a colleague at Protegra we came up with a perfect metaphor for Waterfall or Traditional projects: A Frankenstein project. Please indulge me:

The Frankenstein Project Body Parts

The Torso: The Frankenstein Torso is the project structure itself. Not the business problem or the project objectives but just the overall structure that is required for the Traditional project to function and execute. This can be thought of as the project methodology, processes, and procedures. (yes the excessive meetings are part of this) In general, it is this body that holds the project together for good or evil. (Author’s note: copious foreshadowing)

The Left Arm: The Left Arm is the limb closest to the heart of the project and as such is the Business users. This arm defines the business objectives, outcomes, reasons, and rationale for the project being required. Unfortunately this arm has to be connected to the project in some way. Traditional projects have chosen this stitching to be in the form of extremely detailed documentation and business user sign off. Unfortunately this arm was grown in a lab separate from the torso, so the parts don’t match 100%. This is addressed by more detailed documentation and eventually the arm is attached.

The Right Arm: The Right Arm is closely connected to the Left Arm and represents the analysis required for the project. (This can be both Business and Systems Analysis) This is the translation of the Business objectives, outcomes, reasons, and rational for the project into Software Development Requirements. This arm is again connected to the torso by the Traditional Project artifact of documentation. Unfortunately the Right Arm was also grown in a lab separate from the Left Arm using mainly the documents used to attach the Left Arm, so the parts don’t match 100%. (Although the Left Arm and Right Arm do look somewhat similar) This is addressed by more detailed documentation to address the inconsistencies and eventually the Right Arm is attached.

The Left Leg: The Left Leg is the critical part of Software Development for the project. This represents the Software Development required to translate the Software Development Requirements into actual code. The code deliverable itself is what is used to connect the Left Leg to the torso. It is actually a pretty good stitching job but by this time the Left leg is not looking at all similar to the Left Arm. Oh dear.

The Right Leg: The Right Leg represents the critical part of testing on the project. Unfortunately the Right Leg can only be attached after the three other limbs are attached fully. The Right Leg is attached via the test cases that are generated from the documents that were used to attach the Left Arm and Right Arm. The stitching is looking pretty inconsistent with some areas covered quite well and others not so much. The Right Leg is really looking dissimilar to the other limbs. The other major problem is that the Right Leg is only half the length of the left leg as we ran out of material; The Schedule. I believe the project will have a limp.

The Head: The Head represents the architecture of the project. It is the Architect that is trying to put together these different parts and produce something that satisfies the understanding of the desired outcome both from a functional and technical point of view. The unfortunate thing is that the Head does not look at all similar to the Left Arm. It is similar in functionality but not intent. Typically the brain that Igor found for the Head is from a brilliant technical person who doesn’t quite understand the business domain. Pity.

The Mad Scientist: Ah yes, the Mad scientist. By now you may have surmised this is the Project Manager. 🙂 He or She is trying to understand all the issues that come with piecing the parts together from different parties but at the end of the day there is a fixed budget and thunderstorm coming to bring the beast to life and we need to make that date. He or She feels they need to come up with the solutions and push the team forward. We don’t need collaboration and team problem solving. He or she will solve the problems and then we just need more Igors!

At the end of User Acceptance Testing when the thunderstorm arrives to bring the beast to life, we really have the ultimate big bang implementation. The beast lumbers to life, crashes through the walls, and pillages the village of business users. The project did not resemble what was intended and the villagers are now rioting demanding the head of the Mad Scientist. Sequestered in his Ivory Tower away from the users, he wonders how it could have gone so very wrong. It seems the business forgot to quantify the non functional requirements.

Eventually the Mad Scientist and the Business write-up a plethora of Change Requests and a good part of them require the removal and re-attachment of the entire limbs. Some require parts of the limbs to be replaced. An elbow here, a knee there. Oh dear, the project is looking even worse than before.

Finally the tamed project lumbers out and does the base functionality that was intended in a way that is not elegant or satisfactory. But we have produced something and on the date! And the business did sign off after all!

Next Post: To build a better Monster.

Incremental Development

I think most people’s forays into Agile Development is done via Incremental versus true iterative development. Sadly, I think all of us who do incremental development actually think we are being very agile and doing iterative development.

The project was to create a Campground Reservation service for the Province of Manitoba and was quite successful. We implemented a public website to allow the reservations of campgrounds in 92 days and we made the date. Although we did a lot right, we also did quite a few things wrong. We developed incrementally, we had a separate test phase, our iterations were a month long, we had people owning parts of the application, etc…

The why did the project work? Well I believe we did four things extremely well from Agile/Lean

  1. We had the client decision maker on site 100%
  2. We had an all star team and implemented visual project management and daily stand ups
  3. We allocated a person for the entire project to design the testing and then test continuously
  4. The style of leadership on the project was very collaborative and respectful. The team together created as the estimates and lived by them. (Although we did not use relative estimating or planning poker)

Which goes to show you that rarely does the first Agile project ever do agile correctly and that even doing it incorrectly still provides benefits if you get a couple of the key principles right. I was the Project Manager for that project and have learned since about all the aspects of Agile and Lean and how I would execute the project again.

Next Post: Why I think a testing phase is a concept that is so hard to let go of…