How do you know if something is working? How do you evaluate something you cannot fully understand? How do you convey your intentions and expectations to others?
These are tricky questions that anyone who works in complex environment will encounter. One option is to be rigorously quantitative. It’s all about the numbers. Hundreds of pre-defined KPIs to be captured, collated and analysed. There is definitely value in formulating KPIs but this approach has drawbacks:
- Unexpected Outcomes: The numbers you define up front may not be the numbers that matter. For example, you may run a programme that aims to increase organizational collaboration – but its biggest impact is actually in staff retention. How would you know that with a narrow set of KPIs?
- Measuring the Intangible: Many organizations focus on the numbers that are easy to collect rather than the numbers that matter – e.g. number of courses run, number of reports produced. These numbers often do not describe the impact of these actions because impacts are hard to measure.
- Making It Meaningful: The numbers on the page may mean a lot to you but they are dry and dull to everyone else Even worse, the numbers probably do not provide a clear guide for action.
One solution to these issues is a technique known as Most Significant Change (MSC). MSC is a participatory monitoring and evaluation technique that uses stories. It was first outlined by Rick Davies in the 1990s for the development sector. It is not a replacement for quantitative reporting but rather a complementary qualitative technique.
The basic MSC process will be described in the next post.
Some attempts at implementing Most Significant Change (MSC) miss important aspects of the process. Here are some gotchas:
- Focusing too much on the production of polished stories. The stories you collect need to contain some key points: the context of the teller, their connection to the project, what happened to them and why that was important. However the more onerous you make story collection, the fewer relevant stories you will collection. Video and multi-media are nice but not necessary. You are not making a TV commercial.
- Collecting stories and then not selecting from them. Evaluating experiences can be quite confronting for stakeholders. It is tempting to say that all stories are equally valuable. And in one sense this is true. You should certainly thank all your story contributors. Nevertheless, the discussion around the relative importance of different outcomes is crucial to the process. The danger is that different stakeholders are working with varying assumptions that are never surfaced and never acknowledged.
- Feedback to participants does not occur. The MSC process can be presented as a loop and that loop needs to be closed. One way of showing to participants that you value their stories is by showing that you have read them, considered them and provided a response. This can be a participatory process as well.
It should be noted that MSC works best when monitoring and evaluating projects that are complex, produce diverse outcomes and are participatory in ethos.
The best single introduction to MSC is the online MSC Guide.
A story with heart and soil.
I finally ordered this great documentary/movie from the USA and it is just fantastic. It is an insightful and timely film that tells the story of the glorious and unappreciated material beneath our feet.
The movie is inspired by William Bryant Logan’s acclaimed book Dirt : the ecstatic skin of the Earth. The movie takes a humourous and substantial look into the history and current state of the living organic matter that we come from and will later return to.
Unfortunately for Australians it is only available in Region 1 format – but it plays on my Mac without any problems.
If you want to see some of the shorts and a preview, go to the web site of Dirt the Movie. I also ordered the canvas bag and Tee shirt.
This is a wonderful example of how to use story to get your message across. In the environmental science area we could be doing so much more of this to get the message across about topics such as climate change and water management.
Nerida
Well I am back from the Origins Asia Pacific Conference in Singapore and I consider myself to be so lucky to have attended and presented at this event.
The conference was part of the Singapore International Storytelling Festival and was organised by The National Book Development Council of Singapore. The aim of Origins is to foster the practice of business storytelling and narrative techniques in the Asia Pacific region and to build awareness among government agencies and corporations of the power of storytelling and narrative for business. This conference had the most amazing line up of presenters who are world class in the area of business narrative. I will be providing an overview of the conference in the Our Stories part of our website, as this seems to be the best place to highlight the strengths of storytelling in our day to day working and personal lives. I will also be drawing on my experiences at the conference in our upcoming Connecting through Conversation workshop.
Nerida
“An organization’s ability to learn, and translate that learning into action rapidly, is the ultimate competitive advantage.” – Jack Welch
The theme for this year’s actKM Conference is ‘Getting Clever about Knowledge’ and explores the question of how organisations can improve the way in which they generate and manage new knowledge.
The conference promises to be excellent, with several international and Australian experts covering topics ranging from the latest technology for storing and managing knowledge, through to cultivating wisdom, leadership and enhancing performance in organisations. Nerida, Matt and myself will all be presenting papers at the event, and are really looking forward to learning about the latest thinking and knowledge management approaches.
Why not make a trip to Canberra and spend three days immersing yourself in Knowledge Management (18-19 October actKM Conference) and Narrative Techniques (Connecting through Conversation 20 October), we would love to see you!
Siwan, Nerida and Matt
Resources | innotecture | 
29 September 2010 7:08 pm | 
Comments Off 
