The Art and Science of Measurement

It is a much discussed fact within the nonprofit community that funding partners are taking a growing interest in the performance of the agencies they fund. It is similarly much discussed that nonprofits – both large and small – face a real challenge in deciding not only what they should be measuring, but also how they should communicate those metrics to their different audiences, including funders, partner agencies, clients and the community at large.

I am not going to bore you with my take on measuring nonprofit effectiveness. But I would like to share with you a more fundamental challenge that is at the core of that issue. That issue is comparing like agencies that may indeed be nothing alike.

For the workforce development sector in particular, comparing like agencies and their metrics is daunting.

“Why is that?” I hear you ask, “Surely it is about getting people jobs, and a job, is a job, is a job, right?”

Not exactly.

Not only do different workforce development agencies start with different inputs (different missions, different target populations and different program services), but they also measure their outputs (and ultimately their outcomes) differently.

Consider that you want to compare two agencies that report on the number of people they place in jobs (If you are a funder, you might want to make a comparison to determine who you should fund. If you are a nonprofit, you might want to see how you match up against the competition or figure out whom to model your program on.)

These are some of the questions you might be faced with asking when making a comparison: What kind of jobs are being secured? What is the average wage? Are they only minimum wage jobs, and if so, then which minimum wage - state or federal, and if state, are these two agencies even in the same state? Is it a job that pays enough to break past the federal poverty line - and if so, is it 1.5 times that level, or 2 times? Is it a full-time job and is it a permanent job? Does the job have benefits? If so, what kind of benefits?

And that is just to start. Other questions might include: How long does a person stay in one of those jobs? Are there opportunities for advancement?

With all of these variables, how do I determine which of these agencies is most effective? If I am a funder, which program should I fund? If I am a nonprofit, which should I emulate if at all?

Fortunately for the workforce development sector, there are some people thinking about all this, and working together to chart a course. The P/PV Performance Benchmarking Project is funded by the Annie E. Casey Foundation and is made up of dozens of workforce development agencies from across the country. The goal of the project is to bring agencies together (disclosure: The Cara Program is one of these agencies), build a network, share data and start on the task of determining if there is a common way to benchmark workforce development agencies, and if so how it is to be done.

Three years into the project, some important major milestones have been met. The network is 70 members strong, performance metrics from different agencies are being compared and there is a lively conversation taking place about how to formulate a meaningful common benchmark – including a conversation about whether or not it is even possible to create such a benchmark.

But many challenges remain, including the fact that many organizations simply can not afford to participate in this effort. (If you would like to learn more about the results of the benchmarking study so far, or if you are a workforce development agency looking to get involved in future discussions, please click on http://www.ppv.org/ppv/labor_market/labor_market_initiatives.asp?section....)

Like the private sector, the nonprofit industry as a whole is making the important transition to heightened levels of accountability. Driven by an outcomes focus, and a real desire to show investors how their contributions yield real and lasting success for its population, places like The Cara Program report on results, and share with its constituents the successes and the challenges of empowering so many men and women in need. The Cara Program is proud to be a part of community of mission and outcome-driven organizations that are working to eradicate poverty in our city. Congratulations to those investors who are asking the hard questions. May we all play our part in raising the bar, embracing the challenges and pushing for real and lasting success.

Eric Weinheimer
President
The Cara Program

PS - If you have a few spare moments, check out The Cara Program’s website (www.thecaraprogram.org/performance) to review our Quarterly Performance Updates that highlight our activities year to date.