Step 3: judging the evidence

Epidemiological evidence for studies on cancer incidence

What causes and what protects against cancer? How does WCRF International gather, present, assess and judge the evidence that it uses to explain the risk factors and the recommendations associated with cancer prevention?

The Global Cancer Update Programme provides a comprehensive analysis, using the most meticulous of methods, of the worldwide body of evidence. The aim when judging evidence is to identify, with sufficient confidence to support a recommendation, what causes cancer, what protects against cancer and what is unlikely to have an effect. This work also reveals where evidence is inadequate and further research is needed.

Much of the human evidence on diet, nutrition and physical activity is observational, though it is reinforced by findings of extensive laboratory investigations. There is no perfect way to establish whether observed associations between these exposures and cancer are definitely causal.

However, the Expert Panel believes the rigorous, integrated and systematic approach enables them to make sound judgements and reliable recommendations.

How do we interpret the evidence?

The Expert Panel evaluates and interprets the evidence, making judgements on the strength of the evidence and, where possible, the likelihood that the exposures studied increase, decrease or have no effect on the risk of cancer.
Interpretation of epidemiological evidence is complex. A wide range of general considerations must be taken into account:

• How relevant are the patterns and ranges of intake examined in the existing studies to populations globally?
• Do the studies classify food and drink consumption, and physical activity, in ways that correspond to patterns globally?
• How accurate are measurements of the level of exposure in the study population, such as levels of intake of a food or its dietary constituents?
• Is terminology consistent between studies? For some exposures, such as ‘processed meat’, there are no generally agreed definitions.
• How reliable and complete are the data on cancer outcomes – on incidence and mortality, and subtypes?
• Is the study design appropriate? The hierarchy of evidence places randomised clinical trials (RCTs) at the top followed by cohort studies, case control studies, ecological studies and case reports but there are merits in considering different study designs.
• What is the shape of the association between the exposure and the cancer? For example, is it linear, with a uniform increase (or decrease) in risk for rising levels of exposure? Is there a threshold above which an association is found or a plateau where no further increase or decrease in risk is observed? Or does the direction of association (whether risk is increased or decreased) change with the level of exposure?
• Is there high heterogeneity, a large variation in the results of the studies, which would lead to less confidence in the overall summary estimate?
• Is the overall evidence limited to a particular geographic area and can results be extrapolated at a global scale?
• Do studies take the possibility of confounding, effect modification and reporting bias into account?

Uncertainty in epidemiology

Even though the best available evidence has been used, that evidence does not normally prove, beyond all doubt, whether the risk factors – diet, nutrition and physical activity – cause, or protect against, cancer.

The risk factors themselves are complex and difficult to manipulate in experimental studies. Furthermore, even if a person’s way of living does cause cancer, it may take years or decades for that cancer to develop.

Although randomised clinical trials have the power to test cause and effect vigorously, controlled manipulation of diet and physical activity in RCTs over the long period of time required to study these exposures is not possible.

Much of the data on cancer risk therefore come from epidemiological studies, and there is normally a degree of uncertainty surrounding whether observed associations in these studies are causal. Best judgement is therefore needed when interpreting and assessing results.

Best judgement and grading criteria

The CUP Global grading criteria require a range of factors to be considered. These include quality of the studies – for example, whether the possibility of confounding, measurement errors and selection bias has been minimised.

They also include the number of different study types and cohorts, whether there is any unexplained heterogeneity between results from different studies or populations, whether there is a dose–response relationship, and whether there is evidence of plausible biological mechanisms at typical levels of exposure.

The clearly defined grading criteria provide a systematic way to judge how strong any evidence of causality is. They enable evidence to be categorised as being ‘strong’ (‘convincing’, ‘probable’ or ‘substantial effect on risk unlikely’) or ‘limited’ (‘limited – suggestive’ or ‘limited – no conclusion’). Only evidence judged to be strong is usually used as the basis for Recommendations.

Epidemiological evidence for studies on cancer survival

Many of the same issues apply to studies on cancer survival as for those on cancer incidence, but this population have had a cancer diagnosis and been exposed to treatment for cancer which makes interpretation even more complex.

WCRF International’s grading criteria are designed to judge, for observational studies, whether an association is causally linked to a particular health outcome, or, for RCTs, whether a specific intervention affects the risk of developing specific health outcomes.

The panel consider several factors when judging the evidence, including: the amount of evidence; the consistency, magnitude and precision of the summary estimates; whether there was a dose-response relationship; study design and risk of bias; generalisability; the presence of biological plausibility and mechanisms.

For the reviews on cancer survivors, the criteria aim to judge the degree of certainty that a particular dietary, nutritional, adiposity or physical activity intervention in a population defined by a specific cancer diagnosis, would lead to a change in defined cancer related outcomes.

We have recently produced updated grading criteria specifically for our work in cancer survivorship. These criteria acknowledge the differences between published research focused upon cancer incidence versus survivorship. These support the Panel to judge how a certain intervention or exposure impacts the risk of an outcome occurring specifically within populations living with a cancer diagnosis.

> Get the original Judging the evidence chapter from our 2018 Diet and Cancer Report

Mechanistic evidence

The mechanistic evidence collated and synthesised by IARC is interpreted by the Mechanisms Expert Committee who makes an informed judgment as to whether biological plausibility claims are ‘strongly’, ‘moderately’, ‘weakly’ or ‘not’ supported by that mechanistic evidence.

The process followed by the CUP Global Mechanisms Expert Committee to issue their judgments is facilitated by a decision support tool, which guides the deliberation while fully enabling expert input.