Social Impact Evaluation in Interconnected Evolving Systems

Today there are many development economists that focus on social impact evaluation studies. These studies typically compare groups receiving an intervention to groups that did not. For example, did the group that received microfinance do better than the group that did not? Although they attempt to ‘control’ for variability between groups, I see some fundamental flaws in this approach. First, to be meaningful comparison, a ‘control’ group must be independent of the intervention group. In society, which by definition is an interconnected system, this is virtually impossible and you can never control this way for network effects.

For example, lets say the people who received microfinance in the village used the money to buy goods from the people who didn’t. Its not impossible that this could result in the 'control group' getting richer because they could sell more. Compared to them, the microfinance takers could then look poorer because, while they may also have overall bought and sold more, they also had to pay back their loan with interest to the lender. This is a gross and very direct example of course, and it will never be so simple to parse out the network effects. Recognizing this, if you instead decide to move your control group to a distant location this will not really solve the problem. For one thing, the more distant the ‘control’ community, the more the variability in the circumstances and the more confounding this variation can be. But more important, network effects can be far reaching and change dramatically as the size of the network changes. So physical distance does not help unless you compare two communities that are completely self contained economies (in which case they have evolved so separately and so differently that it will be a useless comparison anyway).

This brings me to a second extremely important point. Networks like society are open feedback systems that evolve continuously over time. This means that the impact you find that microfinance has today (network effects notwithstanding) is completely irrelevant to tomorrow and therefore cannot be prescriptive or even diagnostic in any way. For example, cell phone penetration and use is climbing rapidly in rural India today and dramatically changes the way people interact and transact, opening up distant markets and changing the rate at which people can buy and sell. Just imagine the impact of being able to call a buyer in the neighbouring town to coordinate an order and delivery instead of having to travel the three hour distance by bus! So if you conducted your traditional microfinance impact study today, when cell phone penetration was low, maybe it has little impact because the opportunities to interact rapidly were low and therefore business was slow. Tomorrow it may have magnificent impact as the network conditions change.
So, not to sound too harsh, but in the interest of progress, having read various social impact studies on microfinance conducted at great expense and unveiled with great fanfare, I have to say What’s the point? As a practitioner interested in seeding social progress there’s nothing whatsoever I can take from these studies. It doesn’t tell me what is wrong or what should be done right. Rather it is a flawed comparison of one slice in time.

So what’s the alternative? If we want to come up with something valuable prescriptive, what we need to do is stop wasting time with such social impact evaluations and start trying to understand the network and its evolution. What we need to do is begin asking and answering questions like these:
1) What is the topology of the transport and communication networks? How does information and trade flow in these networks? What kinds of simple changes in the network could result in nonlinear benefits for the flow of goods, money and information?
2) How do ideas and innovations diffuse in these populations? How does this compare to more advanced segments of society? What can be done to create more productive diffusion?
3) What do group dynamics look like in these groups? Can we identify aspects of culture and interaction that predict why these groups fail to organize themselves and innovate? If so, what kind of innovations can we develop that shift these behaviours towards more productive dynamics?

The Value of Money

Yesterday my mother called me to say that she is taking my six year old son to see a coin collector since he has been pestering her with questions about money. Motivated by his interest in buying yet another Ben 10 toy that I won’t get for him (jet ray, shake ray, I don’t get it...), he has become a little obsessed with finding ways to get money. Who invented it? Can he invent his own money, he wants to know. Would people take his invented money? Who makes the money? Can’t he just print some out on our printer? Who decides what its worth?

So back to the coin collector. My first reaction was that it seems a little silly and boring. What’s he going to show him, some bunch of coins from around the world? So what? But then I took a look at some of the coins and turns out it’s a very fascinating lesson in the history of money that sparked some interesting breakfast conversation today. The coins are all from ancient civilizations around the world and hark back to the first invention of money. Here are some of them:



They’re not perfect in shape and are pretty crude. You could so easily counterfeit these, was my first reaction. But that’s OK, my husband said, these coins have intrinsic value. They’re worth the metal they’re made from – x grams of brass or silver for instance. So if you wanted to counterfeit them, you would have to spend the equivalent value of energy and time to mine the metal and that has worth exactly equal to its value.

Our typical text book definition of money is as a store of value. In the last few years I have been thinking of money as being a proxy for potential energy that we accumulate on some level – since it provides the power to do work and reconfigure things. So that money should have intrinsic value (like metal) makes sense to me. You could actually calculate its value in units of energy, say joules. Today we have decoupled currency from any ‘actual value’ and created fiat money. Which means money is no longer linked to something with intrinsic value like gold bars. Given this, its ‘value’ can get progressively more distorted. With this disconnect it takes on a different life and dynamics that I’m still trying to get my head around.

So to my son, I have to say, ‘I have some notional answers to your questions and this is how I’m thinking about it, but in all honesty, sweetheart, I’m almost as baffled as you are. Why don’t you go ask Dad.’

'And, hey, I already told you that we are not going to buy another Ben 10 toy right now'.

Interpreting Poverty through Images

Here is an interesting post that I came across.

Exploring Different Perspectives of Poverty Through Photography
by Duncan McNicholl

Some excerpts:
Many people only experience sub-Saharan Africa through photographs. The teary-eyed child in rags is familiar to all of us as the portrait of poverty charities use to communicate a hopelessness in need of our pity and charity. I reacted very strongly to these images when I returned from Africa in 2008 after a 4 month volunteer placement in Malawi, working with Engineers Without Borders Canada. I compared the images I saw to my Malawian friends – people who embodied intelligence, resilience, and compassion – and I felt lied to.

It seemed that these photos presented by development organizations and the media were deliberately providing only one perspective of rural Africans like my friends in Malawi, which was despair.

.....This photo project, which I am dubbing ‘Perspectives of Poverty’, is not to say that people do not suffer in Africa. Malawi is one of the poorest countries in the world, and we must not lose sight of the fact that millions struggle to get by. But how we interpret the lives of others is critically important since it affects how we support those struggling to overcome poverty.....

The opposite is true of microfinance. As a community it projects only the face of success, making light of the difficulty and struggle of entrepreneurship and a landscape littered with failure.

Mobile Phone Data Capture

Madura is about to embark on a massive mobile phone enabled data capture exercise that will create some of the largest and most unique datasets in the world on the poor. We currently have about 160 locations that manage loan repayments through a mobile app that runs on a windows platform. We are now about to make an additional 500 of our field staff capable of capturing massive information through various mobile applications. The survey apps are being tested on these same windows phones that have worked well for us for our loan servicing needs. However, in this case, the apps are much more data intense. Consequently the phones have to work much harder, run longer, be able to withstand being dropped off the bus a couple times and have sufficient memory to store data given that network connectivity tends to be spotty. We are hoping for a 2 year life for the phones. Given the number of phones and platforms out there, choosing the right one appears to be no easy task. Anyone with experience with heavy usage of smart phones, please do tell me about it!

Engineering organizations

Something that's been on my mind lately is how organizational structure influences organizational outcome. Alex (Sandy) Pentland is a really interesting guy to follow in this context and in the context of social dynamics in general. He runs the Human Dynamics lab at MIT and has created a device that can measure subconscious signals of communication (tone of voice and things like that) that can predict the quality of interactions and therefore outcome. He calls them 'Honest Signals'. Take a look at this video where he talks about them.

Is organization an innate human trait or learned?

I've posted before thoughts on why the poor tend to be fragmented in their economic activity rather than organized into groups and it sparked some discussion (see Driving socioeconomic change by making women more dependent). I've been thinking now about the ability to organize in general. Our microeducation team at Madura just got back from the first pilot testing of our digital 'mini MBA' program. This is a video based training program that brings business education to the poor that has been developed in collaboration with Dr. Madhu Viswanathan at University of Illinois at Urbana Champagne and his Marketplace Literacy initiative. They tested with two different groups - one that was reasonably educated (10th or 12th grade) and one that was largely illiterate, and came back with some very interesting learnings. Here's one: There are a couple places where the video instructor asks them to pause the video, organize into groups of three or four and talk about some particular question or topic. The groups, particularly the more illiterate group were unable to carry out this instruction of organizing into groups. It had to be explained really explicitly and they are now looking at adapting the video to include a demo of how to break up into smaller groups. So, I wonder, is organization itself an innovation that we have taught rather than an inherent human trait?

Madura is looking for a postdoctoral scientist

Madura is looking for a scientist who will do cool things that haven't been done before. Here's the ad:

Postdoctoral Scientist: Rural Social and Trade Networks

This is a unique position where you will reconstruct the trade and communication networks of rural South India from various large empirical data sets and make it available to the greater scientific community. Real time understanding of the social and trade network topologies and dynamics will be used to design interventions that can significantly impact the dynamics of the network as part of an ambitious large scale social experiment to bring about poverty alleviation.

The key requirements are large imagination, a strong desire to work for social change, the ability to work efficiently with large datasets in MATLAB and C and the ability to design cool GUIs that will facilitate public access to the data.

This position does carry publication potential. However the primary goal is to create understanding that can result in fast productive interventions that positively impact the standard of living.

Please submit a cover letter outlining why this interests you along with your CV to kamalesh@mmfl.in

Spread the word!