Thanks to Kat, Steve, Chhavi, and BFP for hosting this discussion.
At d.light we absolutely agree that market research and impact research are both critical undertakings and can often be integrated for nice economies of scale. When done mindfully - for example with an eye to cost-effectiveness and utilization of existing systems/processes - impact research need not be at odds with scaling. I say that from the perspective that not all impact research must be randomized control trials, much like not all market research needs to be 100% statistically significant, to still yield meaningful conclusions.
Moreover, I would argue that impact research is often most meaningful when approached from the same starting point as market research, ie what does the customer value? Those of us sitting in front of our computers can draw up theories of change and metrics that seem right, but all of this needs to be field-tested against the feedback from customers to strike the balance between measuring outcomes that matter to stakeholders like investors/policymakers and those that matter to the end-users themselves. For example, hearing from a customer that offsetting carbon emissions is an important outcome is likely rare in all of our experiences; this does not mean this is a variable we should not analyze. At the same time, we at d.light often hear about how happy a customer is that they no longer need to clean their home of soot every day, freeing up new time for more productive or simply enjoyable activities, or can now socialize at night, creating a sense of empowerment and freedom and building community ties. These variables are certainly harder to measure and may resonate less with policymakers interested in quantifying gains, but arguably need to be considered within a customer-focused impact framework.
I believe in both market and impact research, in as far as am concerned when we talk about the market we refer to the people on who we expect the impact. However because our intention is to see the impact it becomes apparent that the impact looks more important than the later.
For social enterprises it should be very essential to take both as exclusively important.
Yes, and to achieve significant scale in the development sector, there comes a point in the life of a business when social impact research almost becomes more important than market research.
I totally agree that impact should be part of the business model, but do you think it's the driving force for all enterprises? I don't think so. I think sometimes it's a secondary thought to check boxes to donors.....
As long as the social impact research is still informing the model and strategy otherwise we risk stopping 'learning' or 'listening' to the stakeholders.
Agreed, Conilh although if their main revenues are fundraised, they're not really businesses... or they won't be for long because as you say, they're listening to the wrong customer...
I'm in total agreement that all impact research shouldn't have to be RCTs to be of any value and in fact, for those that advocate that, I think they're missing the bigger picture. I think they're also making enterprises shy away from making a start and doing something in this space because it seems to daunting/complex.
I agree that looking at social impact from a customer perspective is really informative and more meaningful and I think that's perhaps where enterprises can be more dynamic than traditional charities as they often view their stakeholders as customers they are accountable to rather than beneficiaries they're helping.
I think it's very true that some of the biggest 'impact' areas are very challenging to measure. I wonder, how do we change the dialogue with donors/supporters/partners around this? How can we still talk credibly about hard to measure outcomes?
In value chain development research always influences projects. Our Off-Grid Dairy project has proven very successful and this has a great deal to do with understanding the market. We collect milk from the rural population but the most successful products are those destined for the urban market such as mozzarella cheese. Research allowed us to be able to satisfy the local market at an affordable price and also target the urban market where most of the money is made.
I once asked a customer what they spent their savings on, now they had bought a solar light. After an embarrassed laugh they said, "More cigarettes!" I thought that was wonderful. A customer giving us honest useful feedback... although not one for the funders to get excited about!
Social enterprises tend to use research findings more effectively, as they do not shy away from changing their business model or reinventing themselves from time to time. They use the information they collect to continually innovate in the design of their products and services as well as in their delivery and distribution models. This keeps things relevant, and they are able to better serve the needs of their customers, by being able to constantly adapt to the needs of the local situation and the market and to what their customers want. Charities, in order to prove the effectiveness of their programmes to their donors, demonstrate the impact that they are having on people’s lives, but they don’t often reinvent themselves, change their way of implementation or learn from their mistakes.
Personally I think anything business based has a better chance of using research effectively. This is not always the case for the development sector as survival is not based on the success of a single project.
Yes - so true; we've had customers say they spend it on alcohol too; we don't shout about that too much (and it's not the majority!), but it's great because it shows we're doing our research effectively and actually gathering true answers so we can trust the quality of our data to tell us what opinion really is.
I agree with Darin that you need to measure impact in terms of what customers say and also what you know. Customers may not articulate that they value carbon offsetting, but if you ask them if they want reliable rainfall they’ll say yes. So it’s about meeting underlying needs as well as explicitly stated ones.
As for those hard to measure outcomes - perhaps this is where we should get more comfortable building a case on the best outcomes evidence in the field, and then overlay the proxy outcomes evidence of our own programme on it. For some donors/supporters/partners at least, this imperfect case is better than not even trying. And it doesn't take years before you can make things happen.
OK, I can't comment on the wider international development sector with any authority but... my prejudiced answer is that I suspect business are more penetrating in their questioning and more honest in their interpretation. They have to be... they're driven by the need to please customers and that means listening to customers...
I agree that as enterprises our 'market' is the people we expect to impact, but I think there is a wider market which are those we're not reaching - but they help to inform how we can better adapt or reach out to them. I agree that there can be an overemphasis on impact research because that's what we want to see and in some organisations I'm apprehensive that they are proving impact rather than measuring impact i.e. they know the results they want to see and set out to prove it. i don't necessarily think that's wrong, but I think it's more informative to set out without any assumptions and have a truly open dialogue; that way you can better find out how to better serve (and ultimately garner more impact).
I agree with Chhavi and Steve. I think social enterprises can be more dynamic and are better at adapting. Some of the reason for this is that they have to to be successful and sustainable, but also I think they very basis is more about coming from the solution angle rather than the need angle. I think charities often see a need and try to address it, whereas I think social enterprises often see the need and the solution at the same time, so they can be more effective.
We carried out a consumer preference test for solar lamps before deciding on what products to work with. We came at it from the idea that local people know what they want more than we do. It was important to us that we understood what features were most important and why.
I agree not all social impact research needs to be RCTs, Darin. In fact, the point of a RCT is that you do it once to prove impact and then you don't need to do it again for the same business model in a similar context. Because if done right, RCTs are time, energy and money intensive!