Tori Hogan at the Social Edge blog “beyond good intentions” recently wrote about the problems with getting reliable feedback from projects in international development, and what GlobalGiving has been doing to improve feedback. Here is an excerpt:
“My old stats professor will probably kill me for this, but what ever happened to good old-fashioned gut instincts? Do you trust anecdotal reports by people who have visited field projects and come away with certain perceptions?
I started considering this concept recently when Marc Maxson at Global Giving introduced me to a new project they’re running that allows people who visit any of their countless field sites to submit ‘postcards from the field‘. These blog-like reports written by non-professionals (mostly by interns and student travelers) have an open-ended framework and are only guided by the question,'”what would you tell your friends about this project?’ According to Marc, ‘gut feelings about recommending a project are broad enough to predict deeper problems.'”
Tori dug through our more than 70 postcards from 2009 and found that this new Visitor Postcards program still needs more volunteers to scale up. I quote again from that post:
“However, as I scrolled through the “postcards” I had a really hard time finding any that were critical of the organizations they highlighted. Are visitors afraid to report on problems or are these organizations actually as perfect as they sound? Marc informed me that, ‘most volunteers try to self-filter and only say good things publicly, but privately send in negative comments.’ Well, it’s not perfect, but at least it’s a start!”
This remains true. The vast majority of comments on projects are positive, although I can think of two organizations who hosted visitors this year that posted negative comments and triggered larger dialogues. But anyone doing a spot check on our postcards is bound to miss them because they are not a significant fraction.
Bloggers and the public are our advisors. I personally thank Tori and look forward to doing what I can to get more honest feedback from visitors onto the site. Visitor postcards have played a major role in the outcome of one organization, and we will be presenting a case study on this at the upcoming Skoll “International Social Innovation Research Conference.”
I replied to Tori’s post:
We think visitor postcards have been a major success because more people are getting a first-hand account of what projects look like. And (as our case study will show) it also works to reform a problem project. The power of real-time feedback loops was enough to cause the organization visited to dissolve and reform under new leadership of a group of underserved beneficiaries. This happened in spite of the “self-filtering” problem we discussed.
Visitors often don’t realize that they omit inconsistent (negative) details when they have good rapport with the people the meet. This is human nature, and affects tourists and evaluators alike. I urge you to read “The Surprising Power of Neighborly Advice” from Science Magazine (March 2009), which shows that (a) strangers’ gut feelings are more reliable trust indicators than a set of facts and (b) most people DO NOT BELIEVE THIS even though they act on it.
I cannot underscore strongly enough that getting more people to walk through more village-level development projects would transform the way that money is spent, for a variety of reasons.
I hope you’ll attend our talk at ISIRC in September, 2009 – Oxford!