What do we know about microfinance – often touted as the solution for the economic woes of developing countries? Practically nothing, say researchers from UConn’s Department of Agricultural and Resources and Economics.

Assistant professor Nathan Fiala and co-author Mahesh Dahal published research earlier this year in World Development where they showed that current literature about microfinance is statistically under-powered and flawed.

Fiala’s research focuses on impact evaluation. He works with governments, non-governmental organizations, and microfinance institutions (MFIs) across the globe to measure the impact of what they do. The recent paper was not as much a study of microfinance as it was a study of other studies – it is a post-publication evaluation of all eight randomized control trial studies of microfinance that have been published in peer-reviewed journals.

“In this paper, we are not making claims about the impact of microfinance,” Fiala says. “We are saying that the evidence of microfinance that we have so far is so bad that we should assume we have no evidence of whether or not microfinance has any impact.”

Microfinance started in the 1970s as a way to help those in poverty, says Fiala. The idea is straightforward: provide small loans to people who may not otherwise qualify, and those small loans could help launch a more prosperous life for the borrowers. The loans can range from $5 up to $200, says Fiala, and the interest rates are lower than loans granted by other lenders – 20-30% instead of 200%, for example. Originally the loans were subsidized, meaning that oftentimes the borrower ended up repaying less interest over the life of the loan.

Fiala says, “Fast forward to the early 2000s and the story had changed. During this time period, the industry was more of a for-profit business and less of a social mission, depending on who you talk to.”

Fiala explains that since then, there has been significant backtracking on the aggressive profitability of microfinance, but the overall effectiveness of these microloans and their social impact is uncertain; and that is where Fiala and Dahal’s research comes in. .

“I’m interested in research transparency, and you’d think everyone would be, but no, that is not the case,” Fiala says. “Transparency includes post-publication review. I’m doing quite a bit of work showing that many studies are flawed, this is something we don’t talk about enough. There have been quite a few high-profile cases where studies do not replicate.”

Fiala explains he and Dahal found that the studies – including work by Nobel laureates — they analyzed are very problematic, to the point which Fiala says some of them should have never been published in the first place.

Examples of problem areas identified in the studies include the take-up rate, or number of people who took out loans compared to the control group who did not take out loans. The take-up rates were extremely low, which gives researchers less data to draw conclusions from. Fiala explains that in some cases, the rates were around 11%. To make matters worse, in one of the studies, individuals in the control group took out loans on their own, therefore further compromising the research design.

“I as a researcher would say, ‘We’re not going to be able to do much with that, I think we should stop the whole study, let’s not proceed any further,'” Fiala says. “Not only did these researchers publish research that should not have been published, they also wasted a bunch of money by doing follow-up data collection with people. There is no reason to spend that money when you should know this is going to be a low-quality study.”

Fiala says there are other common issues they came across in the research that make it impossible to determine statistical significance between the control and trial groups in the studies.

“We show those six papers have such enormous problems that we would basically call them garbage, and when you add up garbage, you still have garbage. So we actually have nothing we can say about microfinance,” he says.

Fiala says that when designing studies, especially social science studies like these, it is necessary to learn from the mistakes made in previous studies to help advance the field.

“What we are doing is really hard, it requires a really careful diligence, and it’s being done by humans so there is lots of error in it,” says Fiala. “Social science is incredibly complex. I like to tell my students that economics is not rocket science, it’s way harder. People don’t all respond the same way to things, and can actually change their behavior when they know I’m looking at them.”

Flawed studies produce data that cannot be used to draw meaningful insights, and that is something Fiala is hoping to address in future work.

“This paper is meant to show that there is a huge problem in the literature, and hopefully we will come up with studies to confirm or deny common knowledge in the industry,” says Fiala.  “Every study of microfinance is way underpowered and produces a bunch of noise. By the way, one of the papers we analyzed is my own. I’m admitting my own work is underpowered. The entire literature is underpowered, results are basically garbage and we should pretend like we don’t know anything.”

Going forward, Fiala has partnered with a large microfinance institution to develop three randomized control trials that will implement changes to previously flawed studies all with the hopes of getting an accurate measurement of the impact, or lack of impact, of the industry. Fiala suspects that microfinance does in fact have a positive social impact, though he is unsure of the size of that impact.

Fiala says that one of the authors of a paper that was examined has gone on record stating that microfinance has zero impact, but Fiala disagrees.

“We should not be making those claims at all at this point. If I had to bet, I’d put my money on that there is a modest effect happening that these studies can’t find, rather than a zero effect that these authors have been claiming.”