Nonprofit Effectiveness Posts

GlobalGiving’s Big Bet

Betting On Impact

How we’re using gamification, incentives, and behavioral economics for good.

We all want our donations to have the most impact possible. But how do we choose which nonprofits to support? And how does an organization like GlobalGiving, working with nonprofits in 165 countries, measure, compare, incentivize, and reward effective nonprofits doing everything from providing affordable housing in Nicaragua to restoring buffalo on Lakota land, to teaching organic farming to at-risk teens in Indonesia?)

Well, we’ve made a Big Bet about how we can do just that, and today marks a major milestone as we’re working to drive more money to more effective organizations. Starting today, the organizations on GlobalGiving that we believe are more effective—those nonprofits that are committed to learning—will be rewarded with more visibility and the chance for more funding through the crowdfunding community.

GG_Rewards_Logo_whiteWe’re using gamification, incentives, and behavioral economics to encourage organizations to listen to the people they serve, to act on what they hear by testing new ideas, and to learn faster and more efficiently. (We call this the Cycle of Progress: Listen, Act, Learn. Repeat.)

The Cycle of Progress: Listen, Act , Learn, Repeat

We’ve created a new ranking system, which we’re calling GG Rewards, that helps us identify which nonprofits are climbing the GG Rewards ladder as Partners, Leaders, and Superstars. We’ve developed this system in collaboration with our nonprofit partners, with researchers, and peers. We’ll continue to improve as we learn more in the coming months. (In case you’ve been following our progress, the GG Rewards system is a big improvement on the Effectiveness Dashboards and Partner Rewards rankings we’ve been experimenting with for several years.)

GG Rewards Status

When our partners log on to GlobalGiving today, they’ll see their GG Rewards Status, a list of benefits their status affords them, and then they’ll have immediate access to tools and resources that can help them log more Effectiveness and Engagement points that will help increase their scores. Nonprofits not only get points for reporting to their donors, but they can also earn points for interviewing stakeholders, collecting community stories, or collecting feedback from the people they intend to help, for example.

We know that 40% of our partners log into our system every week, and we have data that demonstrates that they’ll take actions to improve their ranking because it leads to more funding, so we’re making sure those actions will help the nonprofits not only become better fundraisers but also more effective at meeting the needs of their communities.

As nonprofits demonstrate their commitment to learning and improving, they’ll now have increased visibility on GlobalGiving, and we’re confident that Superstars will reap tangible rewards. Stay tuned as we roll out ways for donors to search for and find effective organizations!

Versions of this article have also been published by GuideStar and Markets for Good

How We’re Building GG Rewards Together

Next week GlobalGiving will be launching the new GG Rewards Program. Here’s a post by Marc Maxmeister that provides a sneak peek into the work that’s gone into conceptualizing, building, and launching the program. 

_______

GlobalGiving‘s goal is to help all organizations become more effective by providing access to money, information, and ideas.

That is a lofty, aspirational goal. To everyone else, it might look like all we do is run a website that connects donors to organizations. But internally, I serve on a team that has met every week for the past 3 years to pour over the data, to find an efficient way to help organizations become more effective. We call ourselves the iTeam (i for impact).

GlobalGiving's i-team. We try not to take ourselves too seriously.

GlobalGiving’s iTeam. We try not to take ourselves too seriously.

It is hard to move thousands of organizations in one shared community forward. We use gamification, incentives, and behavioral economics to encourage organizations to learn faster and listen to the people in whatever corner of the world they happen to operate.

Before 2014 we used just six criteria to define “good,” “better”, and “best.” If an organization exceeded the goals on all six, they were Superstars. If they met some goals, they were Leaders. The remaining 70% of organizations were permanent Partners – still no small feat. Leaders and Superstars were first in line for financial bonuses and appeared at the top of search results.

In 2014 we unveiled a more complete effectiveness dashboard, tracking all the ways we could measure an organization on its journey to Listen, Act, Learn, and Repeat. We believe effective organizations do this well.

But this dashboard wasn’t good enough. We kept tweaking it, getting feedback from our users, and looking for better ways to define learning.

What is learning, really?

How do you quantify it and reward everyone fairly?

The past is just prologue. In 2015, GlobalGiving’s nonprofit partners  will earn points for everything they do to listen, act, and learn.

LALR cycle-dark-bglalr-2015-explained

This week I put together an interactive modeling tool to study how GlobalGiving could score organizational learning. When organizations do good stuff, they should earn points. If they earn enough points, they ought to become Leaders or Superstars. But how many points are enough to level up? That is a difficult question. We worked with our nonprofit partner Leadership Council to get their ideas, and we also created some data models to help us decide.

Here is the data; the current distribution of scores for our thousands of partners, leaders and superstars looks like this:

learning_default_model

How to read this histogram

On the x-axis: total learning points that an organization has earned.

On the y-axis: number of organizations with that score.

There are three bell curves for the three levels of status. It is significant to notice that these bell curves overlap. It means that some Superstar organizations in our old definition of excellence are not so excellent under the new set of rules. Other Partner organizations are actually far more effective than we thought; they will be promoted. Some of the last will be first, and some of the first will be last.

The histogram shown mostly reflects points earned from doing those six things we’ve always rewarded. But in the new system, organizations are also going to earn points for doing new stuff that demonstrates learning:

new_learning_points

And that will change everything. “Learning organizations” will leapfrog over “good fundraising organizations” that haven’t demonstrated that they are learning yet.

old_vs_new_learning_points_model

Not only will different organizations level-up to Leaders and Superstars, everyone’s scores will likely increase. We’ll need to keep “moving the goal posts.” Otherwise the definition of a Superstar organization will be meaningless.

The reason this is a modeling tool and not an analysis report is that anyone can adjust the weights and rerun the calculations instantly. Here I’ve increased the points that organizations earn for raising money over listening to community members and responding to donors:

fundraising_focused_points_model

This weighting would run contrary to our mission. So obviously, we’re not doing that. But we also don’t want to impose rules that would discount the efforts organizations have made to become Superstars under the old rules.

So I created another visualization of the model that counts up gainers and losers and puts them into a contingency table. Here, two models are shown side by side. Red boxes represent the number of organizations that are either going to move up or down a level in each model:

status_change_table

We’d like to minimize disruption during the transition. That means getting the number of Superstars that would drop to Partner as close to zero as possible. It also means giving everybody advance warning and clear instructions on how to demonstrate their learning quickly, so that they don’t drop status as the model predicts. (We’ve talked this over with representatives from our Project Leader Leadership Council to get ideas about how to best do this.)

This is a balancing act. Our definition of a Learning Organization is evolving because our measurements are getting more refined, but we acknowledge they are a work in progress. We seek feedback at every step so that what we build together serves the community writ large, and not just what we think is best.

We’ll share more about the launch of our GG Rewards platform next week. This post is just the story of how we used data and feedback to get where we are. Here are a few lessons of what we’ve learned along the way:

Lessons:

  • Fairness: It is mathematically impossible to make everybody happy when we start tracking learning behavior and rewarding it.
  • Meritocracy: We will need to keep changing the definition of Superstar organizations as all organizations demonstrate their learning, or else it will be meaningless. The best organizations would be indistinguishable from average ones.
  • Crowdsourcing: The only fair way to set the boundaries of Partner, Leader, and Superstar is to crowdsource the decision to our community, and repeat this every year.
  • Defined impact: We can measure the influence of our system on organizational behavior by comparing what the model predicts with what actually happens. We define our success as seeing everybody increase their score every year, and earning more points each year than in the previous year. Success is also seeing a normal distribution (e.g. “bell curve”) of overall scores.
  • Honest measurement: I was surprised to realize that without penalties for poor performance, it is impossible to see what makes an organization great.
  • Iterative benchmarking: We must reset the bar for Leader and Superstar status each year if we want it to mean anything.
  • Community: We predict that by allowing everyone a say in how reward levels are defined, more people will buy into the new system.
  • Information is Power: By creating an interactive model to understand what might happen and combining it with feedback from a community, we are shifting away what could be contentious and towards what could inspire stronger community.

We were inspired by what others at the World Bank and J-PAL did to give citizens more health choices in Uganda. What the “information is power” paper finds is that giving people a chance to speak up alone doesn’t yield better programs (the participatory approach). Neither does giving them information about the program alone (the transparency approach). What improves outcomes is a combination of a specific kind of information along with true agency – the power to change the very thing about a program that they believe isn’t working through their interpretation of the data.

The model I built can help each citizen of the GlobalGiving community see how a rule affects everyone else, and hence understand the implications of their choice, as well as predict how they will fare. If we infuse this information into a conversation about what the thresholds for Partner, Leader, Superstar ought to be each year (e.g. how much learning is enough?), this will put us in the “information is power” sweet spot – a rewards paradigm that maximizes organizational learning and capacity for the greatest number of our partners.

I predict that giving others this power (to predict and to set standards) will lead to a fairer set of rules for how learning is measured and rewards doled out. It ain’t easy, but it is worthy of the effort.

How GlobalGiving Helps Nonprofits Become More Effective

This is our mission at GlobalGiving: catalyzing a global marketplace for money, information, and ideas.

In 2002 we began to build an online funding bridge between passionate individual donors—people whose gifts are often seen as too small to be meaningful—and smaller organizations whose impact has been all too easy to ignore because of their size or remoteness of their work.

Today that funding bridge has turned into a global marketplace where anyone in the world can support projects and organizations that they otherwise would never have known.

But democratizing fundraising was only the beginning.

Soon after we started GlobalGiving, we began providing feedback tools. The very same platform that revolutionized access to crowdfunding has also made it possible for us to collect and share information with our nonprofit partners. We soon found we could offer nonprofits the kind of feedback tools that would dramatically improve the quality of their fundraising.

For example, every nonprofit on GlobalGiving has a personalized dashboard that they see when they log in to the website. From the dashboard they can access details about their project page and fundraising progress:

 

We saw that our partners were hungry for this type of feedback, and they were eager to respond rapidly to new information if it helped them improve their work. Today, more than 2,000 of our nonprofit partners use our feedback tools to improve their fundraising.

The next step: introducing incentives

In 2011 we introduced the Partner Rewards system. Similar to an airline’s frequent flyer program, we give our nonprofit partners points for increasing their engagement with our platform and our feedback tools. Higher Rewards status (partner, leader, and superstar) translates to more visibility on the site (like being featured on our homepage, or our social media), and makes us more likely to refer an organization to our corporate partners. That extra visibility translates to more donations for projects.

In the same way that you might be motivated to purchase your next ticket from United because you’re only 1,000 miles away from Gold Status, we found that Partner Rewards levels motivated nonprofit partners to write that extra project update, or to rally a little harder for a campaign, because it would take them to the next level, ultimately driving more funds to their project.  

Our Partner Rewards Bonus Days are a great example of this. In June 2012 we introduced our first Partner Rewards Bonus Day where we offered different matching percentages for different rewards levels (donations to Partners were matched at 30%, Leaders at 40%, Superstars at 50%). One way for organizations to bump up their status is to submit a project report to fulfil the quarterly requirement. We saw a great increase in the number of partners submitting reports that month in order to qualify for the higher statuses. Clearly, the Partner Rewards System was a compelling incentive to drive behavior.

The cycle of progress: Listen, Act, Learn. Repeat.

As we began to study the links between feedback, learning, incentives and effectiveness, it became clear that our nonprofit partners informally follow the kind of Listen, Act, Learn and Repeat behavior that defines the most successful entrepreneurial businesses. Which made us realize we can do even more to help our partners:

Cycle of Progress

  • Listen: we can help nonprofits access feedback from the people they serve, share best practices, and discover new ways to improve performance.
  • Act: we can provide training and one-to-one consulting to help each partner experiment and try new ways of working.
  • Learn: as they try new ideas we can offer them feedback on how well it is working for the people and causes they serve.
  • Repeat: once they achieve the results they are hoping for, we can help them integrate the new way of working into their operations, so the improvement is sustainable.

While many of our nonprofit partners join GlobalGiving for the access to financial resources, they stay and thrive because we provide something that is just as vital to their mission: access to knowledge. We can use our web platform and powerful incentives (money!) to drive learning, and ultimately, higher performance.

That’s why we’ve created the Effectiveness Dashboard, a way to track the listening, acting, learning, and repeating that our nonprofits engage in both on and off the GlobalGiving platform.  We give our partner organizations points for listening to their stakeholders, testing out new ideas, learning from the results of their experiments, and for integrating learning into their daily work:EffectivenessDashboard

This dashboard is in MVP (minimum viable product) stage of the build-measure-learn product development cycle. Initially the majority of the opportunities to earn points  have focused on fundraising effectiveness, but in 2014 we integrated several external feedback tools into the dashboard, allowing our partners to earn points for listening, acting, learning, and repeating that cycle on the ground.

In 2015 we’re focused on integrating the elements of the Effectiveness Dashboard into our main Partner Rewards ranking and search algorithms on GlobalGiving. Our partners are motivated to improve their Partner Rewards status, because it translates to more funding for their work. We believe that it will also translate to higher performance on the ground that is informed by feedback and data about what works in their communities.

This is how the Effectiveness Dashboard is a powerful tool to help us align fundraising with performance, by  channeling more funds to the nonprofits that demonstrate the greatest commitment to improvement, and have the highest potential to do the most good.