A few weeks ago, Designer commented on Software testing, art and productivity. The question got lost in amongst the comment spam, so I thought I'd give my answer a bit more prominently than usual. The question was:

...Many people who want to get a web-developed project don't even understand the details of work. They just want to have a result and not to make a lot of efforts. How do you think - is there any solution? I think it is wide-spread problem.

If you're talking about smaller web-based projects, I think you're right. It can be really difficult to engage clients in the necessary up-front work to help reduce project uncertainty to a reasonable level. It's a problem given that those with less money to spend have much more business risk in any project they undertake.

I think the second aspect of this is that customers (both internal and external) often come to us with a solution, not a problem. As we build the solution they asked for, the problem becomes clearer and dissatisfaction starts to creep in.

The focus of my work is increasingly on trying to help people build the right thing in the first place. I'm lucky. In my current role, my employer has the conviction that it's important to make sure the project is heading in the right direction. This means making sure the project team has a shared understanding of the product vision, stakholders, those stakeholders' goals, priorities and (in the case of a consumer product) the market and opportunities. They also think that it's OK to bring development to a pause while we get our project bearings.

If you can't choose the projects you undertake, then I don't see any easy answers to these problems. And if you can't convince your customer to be involved appropriately, and to place *some* value on just talking about the problem, it's a hard road ahead for everyone. I guess the desire to do things 'right' is what drives many of us to start our own companies and projects. Without this option though, we can still focus on providing service - helping our customers better understand their problems and pointing out the benefits, costs and risks present in their solution(s). But choose your moments well, and don't stop dreaming that things can be better than they are.

Mike Cohn's "User Stories Applied" discusses using the INVEST mnemonic as a guide to writing better user stories. I was recently asked to dig up a reference for it, and found this presentation here, with the section on the mnemonic on pages 47 and 48.

As I read it, I noticed that there's been a change to one of the letters. Whereas the book uses 'S' to denote 'Small', now it's become 'Sized appropriately'.

I think this is a change for the better, as I noticed that every time I talked someone through the acronym, I would have a long-winded conversation qualifying 'Small' as 'Just small enough but no smaller'. This would come about as I tried to explain the tradeoffs between a story being small enough to estimate with some reasonable certainty, small enough to fit within an iteration, and still ensuring that stories are 'V- Valuable to the customer', needing to ensure that user stories continue to express clearly a problem or need of a person.

Overly small stories push us further from the original context of the problem, and thus force us to compensate with an increasing heirarchy of 'super-stories' to help us focus on the bigger picture. These become more noticable when working in shorter iterations than might be common on a Scrum project, so three cheers for Mike in spending the effort to come up with 'Sized appropriately'.

This was passed on to me via Paul Szymkowiak. Feel free to email me if you are interested in participating in this test for Security Innovation and I will pass on Pete's contact details.

-------------------------

Recruiting Beta Testers for Security Guidance System Usability Testing

We're looking for 50 beta testers for our new application security development guidance system. The testing is focused on usability and asks that testers run through a series of scenarios to qualify the navigation and search mechanisms, and provide feedback on the experience. The duration of the test is two weeks starting 1-Jul-2008.

Requirements for beta testers are that you are a) a software professional, b) comfortable reading & speaking English, c) interested in software application security and d) have a few hours to hammer our product on-line.

As thanks, each tester will receive a free 1 year subscription to all of the content we publish, and a formal invitation to participate on our customer advisory board.

Interested? Know anyone who might be interested? Please contact me.

In the Yahoo software-testing list, Shrini Kulkarni stated

"...productvity as a term is "bad" for creative work like "testing" or "art". That makes me to feel that I am a low skilled labour on a manufacturing assembly line (not that - it is a bad profession but that does not represent the kind of job I do or I would to do or I am capable of doing)"

It triggered some thoughts from recent projects, and my reply follows.

What bad things happen when we apply the word 'productivity' to testing?

Like it or not, unless we are on a fixed price contract with no deadline (and when is there *not* a deadline?), the person who pays is going to be evaluating the value that we provide relative to the money they spend. This seems to be a productivity measurement, and things don't usually go well when we ignore it.

We are often forced to balance the artistic/creative part of our work with giving our employer the impression that we are being 'productive'. My understanding of history is that most artists through history have also had to concern themselves with both productivity and art, especially if they want to make a living.

To me, productivity is the key to defensible testing. It's the idea that I should be striving to get the best coverage that I can for the least cost.

I think where we go wrong is when we define productivity as output or effort, not 'value'. Unfortunately, the former definitions are common, and I think this is a huge problem in many places that I work. That is, there is a focus on appearing busy and creating artifacts, and not on gathering the information that would most help the project right now.

There are negatives to the project in terms of our personal productivity if we are forced to ignore those parts of our work which give us the greatest satisfaction. For me, this does include the aspects of testing that I consider creative and artistic, but it also includes other things, such as the satisfaction I get from face-to-face communication, developing relationships, working ethically, collaborating and helping the development of others.

It's not all about art, but we all have days when our inner artist is challenged. Some days slamming the lid down on the piano and storming off the stage is what's called for. But mostly, we respect the audience...and the show must go on.

Alister Scott, an Australian tester located in Brisbane has started a blog. There are a few Watir samples, and it's always nice when a test automator puts their code up for scrutiny. I know my plans to do the same have taken far too long.

Check his writing out at http://watirmelon.wordpress.com/

Due to a persistent hacking attempt over the last few days, I've done an emergency upgrade. Things will probably look ugly until the weekend, so apologies to anyone visiting!

Jonathan Kohl pointed me at a position paper from Brian Marick for the Agile Coach Camp.

If you're in the middle of adopting agile, it's well worth a read. You can find it here:
http://wiki.agilecoachcamp.org/
tiki-index.php?page=BrianMarickPositionPaper

Another year, and a chance to bring enthusiastic testers together to network and learn from each other. To this end, I'm going to be organising public MAST gatherings. This will be a bi-monthly drinks night at a venue to be decided. In addition to this, I've set up a Ning group to allow members to chat to each other about local issues and to manage announcements and attendance for the group.

You can register your interest by following this link: http://melbournetesters.ning.com/?xgi=2edcWVP

I also recommend joining the other tester club on Ning at Driven QA - http://club.drivenqa.com and finding the ANZAC group.

Hope to see you all soon, and apologies to the person who emailed me about MAST recently. Thunderbird managed to destroy your email before I could reply. The tester 'black thumb' apparently extends to applications that I'm not trying to test as well.

During coffee with Agile-coach and all-round excellent guy Shane Clauson, in sympathy with yet another of my what's-wrong-with-agile rants, he pointed me to this blog post from Jeff Patton:

Don’t know what I want, but I know how to get it

While my opinions diverge on some of what he says must be true, I think the important message that he (and others - Check Alistair Cockburn's writing on this) make is that it pays to plan to iterate. That is, if you're on an agile project and you don't see anyone planning to rework things in response to feedback from using the product, you're probably in for some disappointment.

I think we frequently fail to give our customers an appropriate expectation when it comes to (agile) software development. Having them read this isn't a bad start, but you'll want to figure out how to make this message your own.

I've had a few common rants on most of the agile projects I have worked on. Developers bogged down in the detail of stories, while the critical goals of the system wound up ignored, or realising at the last minute that all of the stories built would do nothing useful.

The ideas I came to as a result of the problems I observed were -

- Compensating by starting with extremely high-level stories that defined the critical user and system goals, then progressively breaking these down into tasks.
- Ensuring that the intent and goal is clear from the story title
- Compensating by coaching the testers to ask the critical questions "Why is this feature being added? What problem does it solve and what value does it add?". The testers would try to ensure that the context was made clear in the story. Links to the higher-level stories in Jira (the tool I most commonly encounter) also helps to provide context.
- Writing high-level acceptance criteria at the top level of stories to help define alternate paths for each goal. These often provided clear boundaries with which stories could be broken down into sub-tasks or sub-stories.
- Evaluating frequently against the high level goals.

I've only had a few chances to really try this out, but from a recent project experience with a little more involvement at the early stages of the project, it seemed I was on the right track.
At some point, being aware of Alistair Cockburn's work, and being a regular reader of his blog, I realised Alistair had probably written about most of this, and I picked up one of his books on use cases. I expected that he's figured most of this out and more.

Then before I got around to reading the book, it was confirmed for me with this blog post, which I highly recommend taking a look at. Even if you're not going to apply use cases, it's worth asking his questions of your project and seeing what you can learn from them.

If you are using use cases, are you avoiding the problems described there?