Learning resource / Statistical reasoning
Statistical software can calculate quickly, but it cannot repair a vague objective, a poorly defined outcome or a biased sample. This guide connects the research question, variable types, study design and analysis before a command is run.
Before the p-value
A credible result depends on who the data represent, how the variables were measured and whether the design can answer the stated question.
A sample is useful only in relation to its target population and sampling process. Eligibility, non-response, missingness and selection can limit how far a result travels.
Ask: who could enter the sample, who actually did, and who was left out?Record the role, type, unit, coding, timing and source of each variable. A number used as a category is not automatically quantitative, and categories coded 1, 2 and 3 are not automatically a scale.
Ask: what does one recorded value mean in the real world?Cross-sectional, cohort, experimental, paired, clustered and repeated-measures data carry different assumptions. The analysis must recognise those dependencies and design limits.
Ask: which observations share a person, site, group or period?Classify with purpose
Classify both the measurement and its role in the question. The same variable can be an outcome in one model and a predictor in another.
Examples include county, smoking status, blood group, treatment arm or an ordered satisfaction category.
Summarise with counts, proportions and clearly labelled denominators.Examples include age, weight, blood pressure, household size, income or a validated scale score.
Inspect the distribution before choosing the summary or method.The outcome should map directly to the research objective and be defined before exploring which analysis looks significant.
One objective can require one primary outcome and justified secondary outcomes.Predictors can be interventions, exposures, characteristics or control variables selected using the research question and subject knowledge.
Adjustment is not a contest to include every available column.A starting map, not an autopilot
This table gives common starting points. The final method also depends on sampling, pairing, clustering, distribution, missing data, sample size, repeated testing and the exact effect you need to estimate.
| Research question | Data pattern | Common starting method | Do not skip |
|---|---|---|---|
| Do two independent groups differ on a quantitative outcome? | Quantitative outcome + two groups | Independent-samples t-test | Distribution, influential values, independence and whether equal variance is reasonable. Consider a suitable robust or rank-based method when assumptions fail. |
| Did the same participants change between two measurements? | Quantitative paired measurements | Paired t-test | Analyse the within-person differences, not the two columns as unrelated groups. A paired rank-based method may suit non-normal differences. |
| Do three or more groups differ on a quantitative outcome? | Quantitative outcome + 3 or more groups | ANOVA | A significant overall test does not identify the groups that differ. Plan appropriate comparisons and account for multiple testing. |
| Are two categorical variables associated? | Categorical outcome + categorical predictor | Chi-square test | Review cell counts and the study design. Fisher's exact test or a model may be more suitable when expected counts are small. |
| How is a continuous outcome related to several predictors? | Quantitative outcome + multiple predictors | Linear regression | Assess functional form, residual behaviour, influential observations, collinearity and whether adjustment variables are justified by subject knowledge. |
| Which factors are associated with a yes/no outcome? | Binary outcome + one or more predictors | Logistic regression | Define the event and reference groups clearly. Report effect estimates with uncertainty and avoid interpreting odds as probabilities. |
Important: A method name is not a complete analysis plan. State the estimand or comparison, assumptions, missing-data approach, effect measure, uncertainty and sensitivity checks.
Sample size & power
Sample-size planning should begin with the primary objective and the smallest effect that would matter. It then considers variability or baseline risk, alpha, desired power, allocation, design effect and expected non-response or loss to follow-up.
Larger samples improve precision, but they do not remove selection bias, measurement error or a weak design.
Read beyond significant / not significant
The analysis is not finished when software prints a p-value. Readers need the magnitude, direction, uncertainty, context and limitations of the result.
It describes how compatible the observed data, or more extreme data, are with a specified null model. It is not the probability that the null hypothesis is true.
State how large the difference or association is, using an appropriate effect measure and a clearly defined reference group.
Confidence intervals help readers see the range of estimates compatible with the analysis and how precise the result is.
A tiny effect can be statistically significant in a very large sample, while an important effect can remain imprecise in a small study.
It may reflect limited information, high variability or an effect smaller than the design could detect. Report the estimate and uncertainty.
Adjustment can reduce some confounding, but an observational association does not automatically become a causal conclusion.
Worked pathway
A county health team has clinic data and wants to understand which recorded characteristics are associated with a clearly defined hypertension outcome.
Specify the blood-pressure protocol, diagnostic threshold, clinic population, eligibility and period represented by the records.
Report prevalence with uncertainty, participant characteristics, variable distributions and missing data using exact denominators.
Use a model suited to the binary outcome. Select age, sex, body-mass index, smoking and other factors from the objective and subject knowledge—not p-values alone.
Review functional form, collinearity, sparse categories, influential observations, model performance and alternative reasonable definitions.
Present adjusted estimates, confidence intervals, reference groups, missing-data handling and diagnostics in language the intended audience can follow.
Describe associations in the sampled clinic setting while acknowledging selection, measurement, residual confounding and limits to causal or population-wide claims.
Plain-language reference
Definitions are deliberately compact. In a real project, each term must be connected to the study design and analysis plan.
This Research Metrics Africa guide is an original educational adaptation and expansion informed by the article supplied by the site owner: Christian Vandever, “Introduction to Research Statistical Analysis: An Overview of the Basics,” HCA Healthcare Journal of Medicine, 2020;1(2):71-75.
It corrects and clarifies selected explanations, adds decision checks and locally relevant research examples, and does not replace project-specific statistical advice.