Below is the final research design from my African Politics course. For those who may not remember, this past semester (Fall 2017) I developed a research design for a hypothetical field study in sub-Saharan Africa. The question I based the study on is as follows: what quantifiable effect would online, NGO-facilitated tutoring have on public secondary school students in urban Uganda, specifically regarding their academic performance?
Back in November, I posted my review of scholarly literature on similar topics (though I have heavily revised that version since then). Now I finally have the research plan itself. Though it certainly still has its shortcomings (and may be full of jargon for those who are not political scientists), I am confident it could serve as a sturdy foundation for social science professionals.
While unlikely to happen anytime soon, Invictus Institute would be wise to someday collaborate with social scientists and test the validity of their tutoring regimen. Favorable results could lead to increased publicity and contributions. Less than favorable results could spark a re-designing of Invictus Institute’s methods that would ultimately serve its students and its mission more effectively.
One easily quantifiable outcome of interest would be O-level exams administered to fourth-year students at the end of their academic year. Exam scores are naturally quantitative measures; this would provide a straightforward path for statistical analysis. And all fourth-year students in the selected samples would presumably receive and complete the same exam, thus ensuring an unequivocal dependent variable to consider.
As convenient (not to mention standardized) a measure as it may seem, the O-level exam still leaves the possibility for confounding, outside variables. A student, for instance, may fall ill the day of the exam, and therefore be unable to concentrate as well as they would have otherwise. Some subjects may be inherently talented at taking tests while others experience overwhelming anxiety come test time. Still others may have received further assistance from their educated parents as opposed to their peers with uneducated parents.
Another contender for a quantifiable outcome is GPA. It is a naturally numeric metric possess by each students. However, it is not as standardized a measure as test scores, and still presents the likelihood for non-conformance between the subjects. Fourth-year students may not all take the same classes, for example. Some of them may have better-skilled teachers than others. Or some students could be afflicted with ADHD or some other mental/developmental disorder that inhibits their learning capabilities writ large.
Despite the myriad possible confounds, with proper randomization techniques, each of the possibilities mentioned in the previous section, and any others not explicitly spelled out, should be balanced across the treatment and control conditions. To ensure randomized conditions, the researchers would pool together a list of all public secondary schools in the Kampala district of Uganda. The website Schools Uganda: Uganda’s No. 1 School Site lists a number of secondary schools in Kampala, so researchers would hopefully need to merely double-check the area to ensure the list’s accuracy (Secondary Schools in Kampala, Directory Listing 2015). As the study is meant to assess the independent variable’s effect on public education, private schools would need to be eliminated. Excluding them, the list includes:
- Baptist High School
- Bilal Islamic Secondary School
- British School of Kampala
- East High school – Ntinda
- Emma High School
- Gayaza Road Secondary School
- Kawempe Muslim Secondary School
- Kibuli Secondary School
- Kings Way Schools
- Lubiri Secondary School
- Makerere High School Migadde
- Maranatha High School-Ggaba
- Mbogo High School
- Nsambya Secondary School
- Old Kampala Secondary School
- Pal and Lisa Secondary School
- Progressive Secondary School, Kitintale
- Rainbow International School Uganda
- Sam’s Park High School
- St Noa Girls Secondary School Zana
- Peter’s Senior Secondary School Nsambya (SPENSA)
- Stena Hill School
- The Aga Khan High School
After verifying and, if necessary, correcting this list, researchers would then assign each of the schools a number using a random number generator such as Microsoft Excel. Researchers would then command Excel to divide the randomly assigned values in numerical order and select the first five schools to serve as the blocks.
With the blocks selected, researchers would then contact school administrators asking for a list of students entering their fourth year in the upcoming school year. Once the full list of names is in the researchers’ hands, they would again utilize a random number generator to designate 100-125 students per school to receive the treatment. While these would not be the most sizable samples, they could at least provide some preliminary findings to be tested and possibly replicated by a larger-scale organization with greater capacities.
With Invictus Institute’s limited resources, as well as the fact that objective, unbiased tools will be used to select the treatment subjects, researchers could inform participants and their families, peers, etc. that they were appointed to take part in the tutoring through a lottery system. This should dispel any potential resentment or claims of unfairness from those not chosen to receive the intervention.
Given the fact that every student takes the O-level exam, the fourth-year students not selected for the treatment within each of the five blocks can be evaluated as a massive control group. The control group would overwhelmingly outnumber the treatment group this way, but as long as a sufficiently statistically significant number of students are assigned to the treatment condition, this should not be an issue. If anything, it would provide a highly reliable set of baseline data to compare the independent variable with.
Another important element to codify is the tutoring itself. Where and when would it take place? How long would each session last? What material would be reviewed? Again, the lurking danger is confounding variables that could bias the results. Invictus Institute’s tutors would ideally draw from the same source in their tutoring sessions, perhaps by referring to lesson plans provided by the schools’ instructors.
If that proves impractical, however, the mere fact that tutoring occurs for the treatment group should be adequate for this study’s purposes. The control group, on the other hand, would simply go about their school year with no assistance from Invictus Institute’s volunteers. To do otherwise would rupture the baseline results necessary to assess the tutoring treatment.
Also worth noting is the matter of cost. Bandwidth limitations make video conferencing a bit of a gamble in Uganda. Therefore, prior to deciding on a tutoring time and location, the schools and participating students’ communities would need to be surveyed for Internet accessibility. Researchers could then coordinate with the students being tutored on the handiest locations in terms of online activity, be it the schools themselves, Internet cafes, libraries, etc., and which would be the best locations for the tutoring sessions.
If worst comes to worst, there is the alternative option of using data. If adequate levels of funding can be acquired, then each student in the treatment group could be granted an inexpensive tablet device with an accompanying data plan so as to forego the worries over internet connection altogether. The devices could be stored in a secure location and only be accessed for the tutoring sessions to mitigate the possibility of damage or theft. This would likely require considerably greater solicitation on the part of the researchers and Invictus Institute; cellular data, while comparatively cheap in Uganda by Western standards, can still amount to a hefty sum over time, especially for so many individuals.
As far as protocol is concerned, perhaps the most important order of business would be to seek Institutional Review Board (IRB) approval. At first glance, this would likely be a challenge considering the subject pool consists not only of children, but of children in an impoverished country, two of the IRB’s vulnerable categories. But considering the fact that Invictus Institute has already been operating in Uganda for several years now, and has strong ties among its people already, the IRB would likely have no objections to these features of the study. On the off chance that they voice concerns, the untapped potential in online tutoring may be enough to persuade them.
This is still merely a speculative outline for the research study. But with time, additional consultation, and revamped fundraising efforts, I am confident that it can mature into something feasible and valuable for its stakeholders, and ultimately answer the question: what quantifiable effect would online, NGO-facilitated tutoring have on public secondary school students in urban Uganda, specifically regarding their academic performance?
“Secondary Schools in Kampala, Directory Listing.” Schools Uganda: Uganda’s No. 1 School Site. 2015. Accessed December 14, 2017. http://schoolsuganda.com/central-region/kampala-district/secondary-schools.html.