Six rules of thumb for determining sample size and statistical power - Resource (2024)

This short guide from the Poverty Action Lab presents six rules very clearly, with helpful diagrams to explain or reinforce the points.

Four of them are relevant for planning quantitative data collection even if you’re not using a control group.

This resource and the following information was contributed byPatricia Rogers.

Authors and their affiliation

Poverty Action Lab

Key features

The guide presents six ‘rules of thumb’ that affect sample size and statistical power. Two of them relate specifically to random assignment (whether the treatment group and control group are the same size, and whether randomization is done at the level of individuals or larger units such as schools or villages) but the other four apply more generally to sampling in order to produce quantitative estimates.

  • Rule of Thumb #1: A larger sample increases the statistical power of the evaluation.

  • Rule of Thumb #2: If the effect size of a program is small, the evaluation needs a larger sample to achieve a given level of power.

  • Rule of Thumb #3: An evaluation of a program with low take-up needs a larger sample.

  • Rule of Thumb #4: If the underlying population has high variation in outcomes, the evaluation needs a larger sample.

  • Rule of Thumb #5: For a given sample size, power is maximized when the sample is equally split between the treatment and control group.

  • Rule of Thumb #6: For a given sample size, randomizing at the cluster level as opposed to the individual level reduces the power of the evaluation. The more similar the outcomes of individuals within clusters are, the larger the sample needs to be.

The rules are illustrated with useful diagrams to explain or reinforce the points.

How have you used or intend on using this resource?

I could imagine using this as part of the training materials for a skills development class in quantitative data collection and analysis. While the concepts would probably be included in a statistics textbook, the format of this guide is likely to be clearer.
The material could also be used to work with an evaluation advisory group or steering group to help them understand design choices and tradeoffs.

Why would you recommend it to other people?

It presents some important ideas very clearly in terms of the clear terminology, the diagrams and the uncluttered design.

Sources

The Abdul LatifJameelPoverty Action Lab (J-PAL) (2018).Six rules of thumb for determining sample size and statistical power.Retrieved from:https://www.povertyactionlab.org/sites/default/files/resources/2018.03.21-Rules-of-Thumb-for-Sample-Size-and-Power.pdf

'Six rules of thumb for determining sample size and statistical power' is referenced in:

Six rules of thumb for determining sample size and statistical power - Resource (2024)

FAQs

Six rules of thumb for determining sample size and statistical power - Resource? ›

While determining sample size, it is usually recommended to include 20 to 30% of the population as a sample size in the form of a rule of thumb. If you take this much sample, it is usually acceptable.

What is the rule of thumb for determining sample size? ›

While determining sample size, it is usually recommended to include 20 to 30% of the population as a sample size in the form of a rule of thumb. If you take this much sample, it is usually acceptable.

What is the rule of thumb for power quality? ›

Measuring Power Quality

The rule of thumb here is this: the greater the disparity between the two meters, the worse the power quality is, and the greater its harmonic content.

What is the rule for sample size in statistics? ›

A good maximum sample size is usually 10% as long as it does not exceed 1000. A good maximum sample size is usually around 10% of the population, as long as this does not exceed 1000. For example, in a population of 5000, 10% would be 500. In a population of 200,000, 10% would be 20,000.

How do you find the sample size for statistical power? ›

Sample size estimation with single group mean

N = (Zα/2)2 s2 / d2, where s is the standard deviation obtained from previous study or pilot study, and d is the accuracy of estimate or how close to the true mean. Zα/2 is normal deviate for two- tailed alternative hypothesis at a level of significance.

What is the rule of thumb for statistical power? ›

Rule of Thumb #1: A larger sample increases the statistical power of the evaluation. Rule of Thumb #2: If the effect size of a program is small, the evaluation needs a larger sample to achieve a given level of power. Rule of Thumb #3: An evaluation of a program with low take-up needs a larger sample.

What is the rule of thumb formula in statistics? ›

The range rule of thumb formula is the following: Subtract the smallest value in a dataset from the largest and divide the result by four to estimate the standard deviation. In other words, the StDev is roughly ¼ the range of the data.

What is the rule of thumb for power supplies? ›

Once you have a clear picture of your power needs, select a power supply with a wattage that comfortably exceeds your calculated total power demand. As a rule of thumb, it's wise to add a buffer of 10-15% to account for unforeseen power spikes or future additions to your system.

What is the rule 5 power of a power property? ›

Power of a power rule: If an expression of a base raised to a power is being raised to another power, multiply the exponents and keep the base the same.

What is the rule for calculating power? ›

Power: Power is defined as the rate at which energy is transferred and can be calculated using the equation P = W t . Power is given in units of Joules per second or Watts (W).

What is the rule of thumb in research? ›

The rule of thumb for sample size calculation is a commonly used method in statistical research to determine the appropriate sample size for a study. It is often used in marketing research, where small sample sizes can be used with partial least square structural equation modeling (PLS-SEM) .

What is the rule of thumb in data analysis? ›

The rules of thumb outline the key relationships between the determinants of statistical power and sample size, and demonstrate how to design a high-powered randomized evaluation. A larger sample increases the statistical power of the evaluation.

What to consider when calculating sample size? ›

In general, three or four factors must be known or estimated to calculate sample size: (1) the effect size (usually the difference between 2 groups); (2) the population standard deviation (for continuous data); (3) the desired power of the experiment to detect the postulated effect; and (4) the significance level.

Why does sample size affect statistical power? ›

Therefore, as sample size increases, the variance of the sampling distribution decreases. If the distribution is more narrow, then there will be less overlap between the two sampling distributions resulting in fewer type II (false negative) errors and a greater statistical power.

What are the assumptions for calculating sample size? ›

Assumptions: Whether the 2 groups are matched/paired or independent. The proportion with the outcome in each group. Rule of thumb: both the value of the proportions and the difference between them affect sample size – sample size much larger for small proportions.

What is the rule of thumb for the sample size of the t test? ›

A question that invariably arises is, “How large does the sample size have to be?” A popular rule of thumb answer for the one sample t-Test is “n = 30.” While this rule of thumb often does work well, the sample size may be too large or too small depending on the degree of non-normality as measured by the Skewness and ...

What is the range rule of thumb for sample size? ›

To find the sample size using the range rule of thumb, you need to divide the range of the sample by a constant, which is typically 4 or 6. In this case, you would divide the range by 4. For example, if the range of the sample is 154, you would divide 154 by 4 to get a sample size of 38.5.

What is the rule of thumb machine learning sample size? ›

The rule-of-thumb rule is that you need at least ten times as many data points as there are features in your dataset. For example, if your dataset has 10 columns or features, you should have at least 100 rows. The rule-of-thumb approach ensures that enough high-quality input exists.

Top Articles
Latest Posts
Article information

Author: Chrissy Homenick

Last Updated:

Views: 5956

Rating: 4.3 / 5 (54 voted)

Reviews: 85% of readers found this page helpful

Author information

Name: Chrissy Homenick

Birthday: 2001-10-22

Address: 611 Kuhn Oval, Feltonbury, NY 02783-3818

Phone: +96619177651654

Job: Mining Representative

Hobby: amateur radio, Sculling, Knife making, Gardening, Watching movies, Gunsmithing, Video gaming

Introduction: My name is Chrissy Homenick, I am a tender, funny, determined, tender, glorious, fancy, enthusiastic person who loves writing and wants to share my knowledge and understanding with you.