There’s a lot of talk on the need to diversify samples in social sciences, and especially in psychology. With a few exceptions though, most of us talk the talk and but rarely walk the walk. What’s amazing is that almost all of us know this. This is no longer about just an elephant in the room; it’s about ignoring an elephant that is staring into our eyes with a smirk on its face.
A recent special issue in the Proceedings of the National Academy of Sciences (PNAS) was devoted to this subject: Sackler Colloquium on Pressing Questions in the Study of Psychological and Behavioral Diversity. And Haruschka, Rogoff, Medin, and Henrich summarize the problem in the first line of the introductory piece:
Extreme biased sampling of research participants and the neglect of their cultural context are increasingly recognized as threats to the generalizability of much of what we know about human thought and behavior (see here).
We also contributed to these issues with an article showing that around 80% of samples used in studies published in the flagship journal of the APS, Psychological Science, between 2014 and 2017, continued to sample Western Educated, Industrialized, Rich and Democratic (WEIRD) societies. Moreover, we show that may characteristics of samples remain unreported (Rad, Martingano, & Ginges, 2018).
There could be many reasons but in our paper we focus on possible remedies. In general, we recommend editors and reviewers to treat non-WEIRD samples with a bias. The idea is that there should be an added incentive for authors and researchers who go out of their way to reach more diverse samples. This could also motivate researchers form non-WEIRD countries to conduct and report studies from their local populations.
We think it’d also help if we could build tools to facilitate psychological testing outside the laboratory and in remote locations. In 2017, about third of the world population used smartphones (Smartphone penetration statistics from 2014 to 2021). This network creates a massive opportunity to expand our within-reach populations. It could also improve ecological validity of our findings since we’d be testing people where they are, rather than where we are (in lab). What is missing is a platform that makes it possible to design and run studies that make use of specialized psychometric tools and are customized to run on smartphones. Cut is this platform.
In our current development cycle, we focus on minimizing the resources needed from the users’ devices (e.g., memory, processing power). We also ‘lighten up’ the data transferred between users and our servers. This reduces consumption of the users’ mobile data. Finally, anything developed on Cut can incorporate different languages with little effort. All that’s needed is table with translations from English to a target language.
We have more awesome features coming… More in future posts.