Who Was in the Room: The Sample Behind Ten Pieces of Self-Improvement Advice

Every record described below was opened and checked. Our method.

A number that describes almost nobody

Think of the last piece of self-improvement advice you took seriously. «Set specific, difficult goals.» «Take short breaks.» «Keep a gratitude journal.» Each sounds like a law of human nature. But each one descends from a particular experiment, in a particular room, with particular people in it, on a particular afternoon. This article asks an embarrassingly simple question about those rooms: who was actually there?

Science has a name for this problem since 2010, and it is a good joke: WEIRD. Most psychology findings come from people who are Western, Educated, Industrialized, Rich and Democratic — mostly American college students, the humans most conveniently located near a research lab. The researchers who coined the term showed that these subjects are «particularly unusual compared with the rest of the species — frequent outliers» (Henrich, Heine & Norenzayan, 2010). In other words: the people we study most are the least typical humans on Earth.

Since then, critics have mostly counted journals — what share of published studies sample which countries. We went the opposite way: we took ten pieces of advice you might actually follow this week, traced each back to its founding study, and asked one question. Who was in the room?

We expected the answer to be «American students», and often it probably is. What we found was stranger: for most of these famous studies, you cannot find out at all.

What we asked of each record

Ten pieces of advice, eleven foundational works (one piece of advice traces to two). For each work we opened the record a reader reaches by following a citation: the bibliographic entry, and the abstract as indexed. Not the method section, which sits behind a paywall for most of these papers. Three questions, answerable yes or no.

Does the record say who took part: students, children, cadets, adults? Does it say where: the country of collection, or an institution that settles it? Does it give how many?

The score: out of eleven foundational works, exactly one answers all three questions. Eight answer none. And five have no readable abstract in any public database — follow the citation behind «studies show», and you arrive at a title and a subscription form. That is the entire trail.

The eleven

The best-reported is the study behind «grit matters more than talent». Its abstract lists five samples with sizes: «2 samples of adults (N=1,545 and N=690), grade point average among Ivy League undergraduates (N=138), retention in 2 classes of United States Military Academy, West Point, cadets (N=1,218 and N=1,308), and ranking in the National Spelling Bee (N=175)», and the institutions place it unambiguously in one country (Duckworth et al., 2007). The same abstract also says the trait «accounted for an average of 4% of the variance in success outcomes», which is a second thing the popular version leaves out.

Next best is the growth-mindset study, which names «373 7th graders» and an intervention of «N=48» against a control of «N=43» (Blackwell et al., 2007). Kind of participant and numbers, no country. That population is children, and this page draws no parenting conclusions from it.

Then the silence. The if-then planning study reports that «difficult goal intentions were completed about 3 times more often» when participants formed them, and never says who those participants were (Gollwitzer & Brandstätter, 1997). The short-breaks study says «the vigilance decrement was averted» in a task performed by «observers»: no number, no country (Ariga & Lleras, 2011). The power-posing paper claims «a person can, by assuming two simple 1-min poses, embody power and instantly become more powerful», with «male and female participants» and no further detail (Carney et al., 2010), mentioned here as an object of examination, not as evidence, as is the depletion paper below.

And five works return nothing at all: the gratitude study, the willpower study, the original marshmallow paper, the deliberate-practice paper behind ten thousand hours, and the goal-setting summary. Between them they carry a large share of what the industry says.

The meta-analysis behind spaced study is a special case. It reports its scale honestly: «839 assessments of distributed practice in 317 experiments located in 184 articles» (Cepeda et al., 2006), but those are counts of studies, not of people, and no country appears. That is normal for a meta-analysis, and it still leaves the reader unable to say who was in the room.

One accidental discovery along the way. Reading eleven abstracts in a row, a pattern jumps out: the recent studies describe their participants properly, the older ones do not — and the two least informative records belong to the two papers most often quoted as settled science. Eleven cases is an observation, not a finding. But it is worth checking on a bigger pile.

How fast the denominator moves

Behind the individual studies sits a slow trend, counted three times on the same six journals.

The first count reported that research «focuses too narrowly on Americans, who comprise less than 5% of the world’s population» (Arnett, 2008). Repeated thirteen years later on the same journals: samples and authors «are now on average a little over 60% American based», and «the change is mainly due to an increase in authorship and samples from other English-speaking and Western European countries» (Thalmayer et al., 2021).

Between them, an analysis of one leading journal found that «almost all research published by one of our leading journals, Psychological Science, relies on Western samples and uses these data in an unreflective way to make inferences about humans in general» (Rad et al., 2018). That last clause is the half this page measures: not only who is sampled, but whether anybody writes it down.

For the two claims here that rest on studies of children, the picture is worse rather than better: developmental journals show «a habitual dependence on convenience sampling and little evidence that the discipline is making any meaningful movement toward drawing from diverse samples» (Nielsen et al., 2017). A later stock-take by two of the original authors is on the record as well (Apicella et al., 2020), though no abstract for it is retrievable either.

When it matters, and when it does not

Does any of this matter? Not always. If you are studying how eyes track a moving dot, a the eyes of a student in Missouri work like the eyes of anyone else. But the original WEIRD review names the areas where culture changes the answers: fairness, cooperation, moral reasoning, self-concept, motivation — how people relate to effort, success and the future.

Now hold our ten pieces of advice against that list. Spaced study and short breaks are about memory and attention — mental machinery with no obvious cultural flavour, the kind you would expect to work anywhere. But grit, growth mindset, gratitude, goal setting and delayed gratification? Those are all about self-concept and motivation. They sit exactly in the zone where culture rewrites the rules.

And this is not just theorising — transfer can be measured. When researchers reran 28 famous findings across 36 countries with 15,305 people, about four in ten effects varied meaningfully across settings (Klein et al., 2018). Honest reading of the other side: six in ten travelled just fine. The point is not that nothing transfers. The point is that you have to check — and for most advice, nobody has.

What a fair version would sound like

Take three of the ten and add the population limit that the underlying record supports. Not the limit we assume; the one the record itself states.

«Grit predicts success» becomes: in five US samples, including West Point cadets and Ivy League undergraduates, a self-reported trait explained about four per cent of the variance in outcomes. «A growth mindset raises achievement» becomes: among 373 American seventh-graders followed through junior high, and in an intervention with 48 pupils against 43 controls, believing intelligence is malleable went with a rising grade trajectory. «Ten thousand hours makes an expert» becomes: a 1993 paper about expert performance, whose sample this page could not establish from any accessible record.

Each is duller and each is true. The interesting question is not why the popular versions drop the qualifier (a qualifier costs a sentence and sells nothing) but why the qualifier is so hard to recover even when somebody goes looking.

How we counted

Collected 22 August 2026. Ten pieces of advice, each traced to the work the popular version most often names; eleven works in total. Every identifier was verified in Crossref, and every abstract was pulled from PubMed, Europe PMC or Semantic Scholar in that order. Each work was then scored on three binary fields, and every value is published with the sentence it came from.

The important limit, and it changes what the number means. We read the record a reader can reach, not the method section. Most of these papers are paywalled; several were published between 1970 and 2003, when indexed abstracts were shorter or absent. So the count partly measures how the literature was indexed, not how carefully the authors wrote their methods. A paper can describe its sample impeccably on page four and still return nothing here.

Call that a feature of the count rather than a defect in it. The claim being tested is about what a reader can find out, and the answer to that question is the same whether the information is missing or merely locked.

Departures from the plan: one coder, no second extraction, and no participant-level tally of who came from where; that would need the full texts. The second measurement the design called for, counting how many popular retellings mention the population at all, was not built and does not appear above.

What this does not prove

It does not show that any of this advice fails outside the countries it was studied in. Sample composition never refutes an effect; it bounds what is known about transfer. The correct word is «untested», and it is much duller than «wrong».

It is also not a complaint about American undergraduates, who did nothing except live near a funded laboratory. The complaint is about the step where a finding from one room becomes a sentence about everybody, performed silently, usually by someone who never saw the room.

There is a practical residue, and it is one question rather than a method. When a sentence begins with «studies show», ask which room. If the answer arrives with a country, a kind of person and a number in it, the claim is worth its confidence. If the trail stops at a title behind a paywall (which happened five times in eleven here), then what you have is a fact-shaped object of unknown provenance, and it should be held about as firmly as that description suggests.

The boring bottom line

Eleven studies underneath ten pieces of everyday advice. One tells you which country it was run in. Two tell you what kind of person took part. Five tell you nothing at all, because there is no abstract to read. Meanwhile the share of US-based samples in the field’s leading journals has moved about ten points in thirteen years, mostly by adding other Western countries.

So when the next sentence begins «studies show that people», the honest follow-up is not «that is false». It is «which people», and the discovery of this page is how often nobody can say. Claims checked one at a time on this site: growth mindset, willpower, the Pomodoro technique, habit stacking, twenty-one days, learning styles, the average of five people, and the six claims of a bestseller.

Sources

• Henrich, J; Heine, S J; Norenzayan, A (2010). The weirdest people in the world?. Behavioral and Brain Sciences 33(2-3):61-83. The paper that named the problem. Note what it does not say: it does not say findings from these samples are wrong. doi:10.1017/S0140525X0999152X
• Arnett, J J (2008). The neglected 95%: why American psychology needs to become less American. American Psychologist 63(7):602-614. The first count, and the baseline for the two that follow. doi:10.1037/0003-066X.63.7.602
• Thalmayer, A G; Toscanelli, C; Arnett, J J (2021). The neglected 95% revisited: is American psychology becoming less American?. American Psychologist 76(1):116-129. Over seventy per cent to a little over sixty, in thirteen years — and the movement is towards other Western countries. doi:10.1037/amp0000622
• Rad, M S; Martingano, A J; Ginges, J (2018). Toward a psychology of Homo sapiens: making psychological science more representative of the human population. PNAS 115(45):11401-11405. The half of the problem this page measures: not only who is sampled, but whether anybody says so. doi:10.1073/pnas.1721165115
• Nielsen, M; Haun, D; Kärtner, J; Legare, C H (2017). The persistent sampling bias in developmental psychology: a call to action. Journal of Experimental Child Psychology 162:31-38. Needed for the two claims on this page that rest on studies of children. doi:10.1016/j.jecp.2017.04.017
• Apicella, C; Norenzayan, A; Henrich, J (2020). Beyond WEIRD: a review of the last decade and a look ahead to the global laboratory of the future. Evolution and Human Behavior 41(5):319-329. Identifier verified in Crossref. Included so the original authors' own later assessment is on the page. doi:10.1016/j.evolhumbehav.2020.07.015
• Klein, R A; Vianello, M; Hasselman, F; Adams, B G; and co-authors (2018). Many Labs 2: investigating variation in replicability across samples and settings. Advances in Methods and Practices in Psychological Science 1(4):443-490. The direct empirical test of the question: run the same thing in 36 countries and see whether the answer moves. For most effects here it did not move much, and that is a finding this page reports. doi:10.1177/2515245918810225
• Gollwitzer, P M; Brandstätter, V (1997). Implementation intentions and effective goal pursuit. Journal of Personality and Social Psychology 73(1):186-199. The foundational study for if-then planning. The accessible record names no country, no sample size and no kind of participant. doi:10.1037/0022-3514.73.1.186
• Blackwell, L S; Trzesniewski, K H; Dweck, C S (2007). Implicit theories of intelligence predict achievement across an adolescent transition: a longitudinal study and an intervention. Child Development 78(1):246-263. The population is children, and nothing on this page is advice about raising them. The record gives the numbers and the kind of participant; it does not give the country. doi:10.1111/j.1467-8624.2007.00995.x
• Ariga, A; Lleras, A (2011). Brief and rare mental «breaks» keep you focused: deactivation and reactivation of task goals preempt vigilance decrements. Cognition 118(3):439-443. The study behind «take short breaks». A single laboratory vigilance task, and the record does not say who did it. doi:10.1016/j.cognition.2010.12.007
• Duckworth, A L; Peterson, C; Matthews, M D; Kelly, D R (2007). Grit: perseverance and passion for long-term goals. Journal of Personality and Social Psychology 92(6):1087-1101. The best-reported of the eleven: sizes for every sample, and institutions that place the work in one country. Also, in its own abstract, an effect of four per cent. doi:10.1037/0022-3514.92.6.1087
• Emmons, R A; McCullough, M E (2003). Counting blessings versus burdens: an experimental investigation of gratitude and subjective well-being in daily life. Journal of Personality and Social Psychology 84(2):377-389. Identifier verified in Crossref. A reader following the citation from a popular article arrives at a title and a paywall. doi:10.1037/0022-3514.84.2.377
• Baumeister, R F; Bratslavsky, E; Muraven, M; Tice, D M (1998). Ego depletion: is the active self a limited resource?. Journal of Personality and Social Psychology 74(5):1252-1265. Identifier verified in Crossref. On this site the depletion model has its own page; here the paper appears only as an example of a foundational citation whose sample a reader cannot check. doi:10.1037/0022-3514.74.5.1252
• Carney, D R; Cuddy, A J C; Yap, A J (2010). Power posing: brief nonverbal displays affect neuroendocrine levels and risk tolerance. Psychological Science 21(10):1363-1368. Included because the gap between the strength of that sentence and the absence of any stated sample is the whole subject of this page. doi:10.1177/0956797610383437
• Mischel, W; Ebbesen, E B (1970). Attention in delay of gratification. Journal of Personality and Social Psychology 16(2):329-337. Identifier verified in Crossref. The population is children; nothing on this page is advice about raising them. doi:10.1037/h0029815
• Ericsson, K A; Krampe, R T; Tesch-Römer, C (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review 100(3):363-406. Identifier verified in Crossref. The most-repeated number in this genre rests on a paper whose sample a reader cannot see without a subscription. doi:10.1037/0033-295X.100.3.363
• Cepeda, N J; Pashler, H; Vul, E; Wixted, J T; Rohrer, D (2006). Distributed practice in verbal recall tasks: a review and quantitative synthesis. Psychological Bulletin 132(3):354-380. Counts of studies rather than of people, and no countries — which is normal for a meta-analysis and still leaves the reader unable to say who was in the room. doi:10.1037/0033-2909.132.3.354
• Locke, E A; Latham, G P (2002). Building a practically useful theory of goal setting and task motivation: a 35-year odyssey. American Psychologist 57(9):705-717. Identifier verified in Crossref. doi:10.1037/0003-066X.57.9.705