We’re at an inflection point. AI is fundamentally changing how we research, learn, and solve problems in development economics. Yet the skepticism persists—and rightfully so. We’ve all seen the disclaimers: “ChatGPT can make mistakes, so double-check it.” We’ve all experienced AI confidently providing plausible-sounding answers to questions it has no business answering.

But here’s what I’ve learned: dismissing AI entirely is just as risky as trusting it blindly.

The truth? The future of research isn’t AI replacing human expertise—it’s AI amplifying
it. This is what researchers call Human-AI collaboration, and it’s transformative when
done right.

The Problem: Lost Hours in Literature Labyrinths

Let me paint a familiar picture. You’re a development economist planning a study on cash transfers. You need to understand what’s already known. So you start searching:

• NBER working papers
• World Bank policy research
• J-PAL and IPA evaluations
• Cross-referenced journal articles
• Grey literature and policy briefs

By the time you’ve synthesized findings across these fragmented sources, you’ve burned 40-60 hours of research time. And you haven’t even started your actual analysis yet.

Literature review—the foundation of rigorous research—has become a bottleneck. We’re losing time on the searching when we should be spending it on the thinking.

Introducing DERA: Intelligence With Guardrails

DERA (Development Economics Research Assistant) is designed with this exact challenge in mind. It’s an AI-powered platform that combines semantic search with Retrieval-Augmented Generation (RAG) to revolutionize how we discover, access, and synthesize research in development economics.

But here’s the key: DERA isn’t here to replace your judgment. It’s here to amplify it.

The DERA interface
simplifies research discovery.

Five Capabilities Built for Human-AI Collaboration

Seamlessly browsing
research evidence.

The Human-AI Contract

Notice what DERA doesn’t do: it doesn’t hand you answers carved in stone. Every feature is designed for verification, refinement, and human judgment. You’re not trusting AI to do your thinking—you’re using it to handle the grunt work so you can think better.

Balancing AI automation
with human judgment.

The Bigger Picture

In development economics, rigor matters. We’re informing policy that affects millions of lives. We can’t afford careless research. But we also can’t afford to waste half our research cycle on logistics.

DERA embodies a principle I believe deeply: smart tools make smarter humans. It
democratizes capabilities once available only to researchers at well-resourced institutions with dedicated research librarians and meta-analysis specialists.

The skepticism about AI in research? It’s not going away—and it shouldn’t. But the solution isn’t rejection. It’s smart collaboration.

What’s your experience been with AI in your research process? Are you finding the balance between efficiency and rigor?

About the Author

Aubrey Jolex is the founder of AJ Impact Evaluation Consulting, specializing in rigorous impact evaluation for development programs. With 7+ years of experience with leading research organizations such as IPA, IFPRI and IITA, Aubrey has designed complex survey instruments and managed data collection across multiple countries.

← Back to Blog | Home

AJ Impact Evaluation Consulting has designed and programmed complex SurveyCTO survey instruments incorporating advanced logic, real-time validation checks, and quality monitoring. Beyond
programming, the team understands the full survey lifecycle: ensuring instruments align with research
objectives, piloting, managing enumerator training, and troubleshooting implementation challenges.

But tracking changes across survey iterations? That was always the hardest part.

To address this challenge, AJ Impact developed the SurveyCTO Version Control tool with optional AI Agent—a Google Sheets Add-on that streamlines deployment management with one-click redirect form submission to SurveyCTO server and intelligent change logging.

Key Features

The SurveyCTO Version
Control menu integrates directly into Google Sheets

The AI Agent: Your Smart Documentation Assistant

Don’t want to write deployment logs manually? Let the AI Agent do it for you.

Instead of spending 15 minutes documenting what changed, the Agent tells you:

The AI Agent
automatically detects and summarizes form changes

Why This Matters

Every successful impact evaluation relies on well-documented survey instruments. When you’re managing complex RCTs across multiple sites and iterations, knowing exactly what changed between versions isn’t just convenient—it’s essential for maintaining research integrity.

The Full Survey Lifecycle

This tool was built from real-world experience across the entire survey lifecycle—from design through implementation. Complex instruments. Real-world constraints. Tools that actually solve problems.

Choose between manual
logging or AI-powered summaries

How It Works

Every deployment is
logged with version, timestamp, and change message

Survey Expertise for Your RCTs

This comprehensive survey experience and tool-building capability positions AJ Impact well for developing and managing survey instruments for RCTs. Whether you need:

About the Author

Aubrey Jolex is the founder of AJ Impact Evaluation Consulting, specializing in rigorous impact evaluation for development programs. With 7+ years of experience with leading research organizations such as IPA, IFPRI and IITA, Aubrey has designed complex survey instruments and managed data collection across multiple countries.

← Back to Blog | Home

You’re designing an impact evaluation. You’ve identified
your research question, chosen your outcome measures, and decided on a randomized controlled trial (RCT)
design. Now comes a critical question that many organizations get wrong:

How many people do you need in your study?

This is where power analysis comes in—one of the most important (and most misunderstood)
concepts in impact evaluation.

What Is Statistical Power?

In simple terms, statistical power is the probability that your study will detect an
effect if one actually exists.

Think of it like a metal detector:

• A high-powered detector can find small coins buried deep underground
• A low-powered detector will only find large metal objects near the surface
• With a weak detector, you might walk right over buried treasure and never know it was there

In evaluation, power determines whether you’ll be able to detect your program’s true
impact. A well-powered study can detect small but meaningful effects. An underpowered study might miss
real impacts entirely.

The Four Key Concepts

Every power analysis involves four interrelated components:

These four are mathematically linked. If you know three, you can calculate the fourth.

Why Sample Size Matters: A Real Example

Let’s say you’re evaluating a girls’ education program. You believe the program increases secondary
school enrollment by 10 percentage points (from 60% to 70%).

The difference? Sample size. Too small, and you’re flying blind.

The Consequences of Being Underpowered

When studies are underpowered, several bad things happen:

1. False Negatives (Type II Errors)

Your program actually works, but your evaluation fails to detect it. You conclude the program is
ineffective and shut it down, wasting a potentially valuable intervention.

2. Wasted Resources

You spent money, time, and effort implementing an evaluation that was doomed from the start. All that
investment in data collection, analysis, and reporting yields inconclusive results.

3. Publication Bias

Journals and funders favor statistically significant results. Underpowered studies that show “no effect”
are less likely to be published, even if they were conducted rigorously.

4. Incorrect Conclusions

Sometimes, underpowered studies do find statistically significant results—but these are often false
positives or wildly inflated effect sizes. This misleads future program designers.

Conducting a Power Analysis: Step-by-Step

Here’s how to do a power analysis for your evaluation:

Practical Example: Sample Size Calculation

Adjustments for Real-World Factors

Common Power Analysis Mistakes

What If You Can’t Afford the Required Sample?

Power analysis might reveal that you need 2,000 participants, but you can only afford 500. What now?

Conclusion: Don’t Skip the Power Analysis

Power analysis is not an optional luxury—it’s a fundamental requirement for any rigorous impact
evaluation. Skipping it is like building a house without checking if the foundation can support the
structure.

Remember: An underpowered evaluation is
worse than no evaluation at all. It wastes resources and generates misleading conclusions.

About the Author

Aubrey Jolex is the founder of AJ Impact Evaluation Consulting, specializing in rigorous impact
evaluation for development programs. With 7+ years of experience with leading research organizations
such as IPA, IFPRI and IITA, Aubrey has designed and powered a number of impact evaluations across
multiple countries.

← Back to Blog | Home

Every year, billions of dollars flow into
international
development programs. Organizations are driven by a genuine desire to create positive change. Yet a
critical question often goes unanswered: Are these programs actually working?

This is where rigorous impact evaluation comes in—and why it matters more than ever.

The Problem: Good Intentions Aren’t Enough

Development practitioners work hard. But here’s the hard truth: activity doesn’t equal
impact.

Without rigorous evaluation, we’re operating in the dark. We might be wasting resources on programs
that
don’t work, or worse, cause harm.

What Makes Evaluation “Rigorous”?

A rigorous impact evaluation answers a specific causal question: Did this program cause the
observed outcomes?

The gold standard is the Randomized Controlled Trial (RCT). When that’s not
feasible,
Quasi-Experimental Designs (QEDs) like RDD or DID can provide credible evidence.

Common Evaluation Pitfalls

Why Rigorous Evaluation Matters

1. Learn What Works: Discover the truth about your
   program’s effectiveness.
2. Stop Wasting Money: Cut losses on ineffective
   programs
   and double down on what works.
3. Improve Programs: Learn which components drive
   impact
   and for whom.
4. Build Credibility: Funders demand evidence.
   Rigorous
   proof gives you a competitive edge.
5. Contribute to Knowledge: Help the global
   development
   community learn.

Common Objections (And Why They’re Wrong)

Conclusion: Evidence Matters

Rigorous impact evaluation isn’t a luxury. It’s a moral imperative. We owe it to the people we serve
to
ensure our programs actually improve their lives.

About the Author

Aubrey Jolex is the founder of AJ Impact Evaluation Consulting, specializing in rigorous impact
evaluation for development programs. With 7+ years of experience at IFPRI and 6+ peer-reviewed
publications, Aubrey helps organizations generate credible evidence of program impact.

← Back to Blog | Home

Randomized controlled trials (RCTs) are the gold standard
for impact evaluation. But let’s be honest: sometimes an RCT just isn’t feasible.

Maybe your program is already running, political constraints prevent withholding services, or you’re
evaluating a national policy. Does this mean you can’t conduct rigorous impact evaluation?
No.

Enter quasi-experimental designs (QEDs)—methods that approximate experimental conditions
without random assignment. When implemented carefully, QEDs can provide credible causal evidence.

When to Use Quasi-Experimental Designs

Four Common Quasi-Experimental Designs

1. Regression Discontinuity Design (RDD)

Best for: Programs with strict eligibility cutoffs (e.g., test scores, income
thresholds).

RDD compares people just above vs. just below the cutoff. If the cutoff is arbitrary, people on either
side are likely very similar, making the difference in outcomes attributable to the program.

2. Difference-in-Differences (DID)

Best for: Policy changes where you have pre- and post-data for treatment and comparison
groups.

DID compares the change over time in the treatment group vs. the change in the
comparison group. This removes time-invariant differences and common trends.

3. Propensity Score Matching (PSM)

Best for: When you have rich data on all factors influencing participation.

PSM creates a comparison group by matching each treated person with a non-participant who has similar
characteristics (age, education, motivation, etc.).

4. Instrumental Variables (IV)

Best for: When you have a variable (instrument) that affects treatment but not outcomes
directly.

IV uses “natural experiments” (like distance to a clinic or lottery numbers) to isolate causal effects.

Which Design Should You Use?

Strengthening Your Design

Since QEDs rely on assumptions, you need to work harder to prove your results are robust:

• Combine Methods: Use DID + PSM together for stronger evidence.
• Falsification Tests: Test for effects on “placebo” outcomes that shouldn’t change.
• Sensitivity Analysis: Show that results hold under different assumptions.
• Transparency: Be honest about limitations and assumptions.

Conclusion

Quasi-experimental designs offer rigorous causal inference when RCTs aren’t feasible. But they’re not a
free lunch—they require strong assumptions that must be justified and tested.

About the Author

Aubrey Jolex has designed and implemented both experimental and quasi-experimental evaluations across
multiple countries. Let’s find the right approach for your context.

← Back to Blog | Home

Randomized controlled trials (RCTs) are the gold
standard for measuring program impact—but what makes them so powerful? And how do you actually
design and implement one correctly?

Whether you’re evaluating a health intervention, education program, or poverty reduction strategy,
understanding RCTs is essential for rigorous impact evaluation.

What is an RCT?

An RCT is an experimental design where participants are randomly assigned to either a treatment group
(receives the program) or a control group (does not receive the program).

The Key Principle: Randomization

Random assignment ensures that, on average, treatment and control groups are identical except for the
program itself. This means any difference in outcomes can be attributed to the program—not to
pre-existing differences between groups.

The Anatomy of an RCT

Types of Randomization

Implementing Randomization with Integrity

Common Threats to Validity

Best Practices for Clean Randomization

Power Analysis: Getting Sample Size Right

A critical step in RCT design is determining how many participants you need. Too few, and you won’t
be able to detect program effects. Too many wastes resources.

Common RCT Design Challenges

Conclusion

RCTs provide the most credible evidence of program impact when done correctly. Key principles:

About the Author

Aubrey Jolex has designed and implemented dozens of RCTs across Asia and Africa with 7+ years of
experience at IFPRI. Learn from real-world experience to implement rigorous evaluations.

← Back to Blog | Home

Randomized controlled trials (RCTs) are the gold
standard for impact evaluation—when done correctly. But we’ve reviewed dozens of RCT
designs over our years in the field, and we see the same mistakes repeatedly.

These aren’t minor technical issues. They’re fundamental flaws that can invalidate your entire
evaluation, waste resources, and lead to incorrect conclusions about program effectiveness.

Mistake #1: Insufficient Statistical Power

The Problem: You design an RCT with too small a sample to detect meaningful program
effects.

Mistake #2: Poor Randomization Implementation

The Problem: Randomization is compromised by field staff or logistical errors,
undermining the entire design.

Mistake #3: Measuring Outcomes Too Soon (Or
Late)

The Problem: You collect data before effects materialize or after they’ve faded.

Rule of Thumb: Map your theory of change carefully.
Education often needs 1-2 years; health might need 6-24 months depending on the outcome.

Mistake #4: Ignoring Implementation Fidelity

The Problem: You assume the program was implemented as designed, but it wasn’t. A
“null result” might just mean the program never actually happened.

Mistake #5: Multiple Testing Without
Correction

The Problem: Testing 30 outcomes and reporting only the 2 “significant” ones. This
is statistical cherry-picking.

Bonus: Skipping Pre-Registration

Pre-registration prevents “p-hacking” and “outcome switching.” Always register your study design and
analysis plan on the AEA RCT Registry or OSF before collecting endline data.

Conclusion

Great RCT design requires care, technical knowledge, and commitment to rigor. Avoid these five
mistakes:

About the Author

Aubrey Jolex has designed and implemented dozens of RCTs across Asia and Africa with 7+ years of
experience at IFPRI. Learn from real-world experience—avoid costly mistakes in your
evaluation.

← Back to Blog | Home