.
The Fundamentals series aims to break down complex and deep subjects into a comprehensible understanding of the underpinnings. Often one of the first stumbling blocks in learning a new subject can be the overwhelming nature of a task fraught with it’s own language dependent idiosyncrasies. This series will aim to give an layman’s perspective and build up to providing sufficient evidence to continue your own research.
What is Meta-analysis?
Meta-analysis is a statistical analysis tool that can be used for combining evidence and results from multiple studies. You can think of Meta-analysis as a way of drawing out the relationships of interest within these multiple studies – in the Medical Research field Meta-analysis is often used to determine the efficacy of a treatment or risk factor for a disease – or more directly, the pertaining body of evidence.
Meta-analysis in Medical Research
One of the major benefits of Meta-Analysis in the Medical Field is the ability to analyse and draw conclusions from the effects of multiple small studies. As medical research is often undertaken by multiple researchers at multiple locations the results can be diverse and conflicting which proves highly problematic for the ethos of evidence based research.
Meta-analysis aims to combine the results of individual studies and draw out conclusions based on the results. Meta-analysis aims to discover if an effect exists, if the effect is positive or negative and to obtain a single summary estimate of the effect. The results can improve precision, answer wider questions, generate new ones and settle conflict of data between the results independent research.
Heterogeneity
Examination of Heterogeneity, that is – identifying the sources of variation in responses, is the most crucial task in Meta-Analysis. The success of Meta-analysis relies upon the integrity of it’s underpinnings and as such frameworks and protocols are allied to provide the requisite structure.
The Cohchrane Collaboration is one of the most comprehensive methodology developers in the field contributing to structural protocols as well as analytic and diagnostic methods and offers handbook of guidelines for reference.
It is through examining the heterogeneity that we are able to get a better view of the data’s integrity, and any underlying bias in the research.
What types of Meta analysis are there?
There are multiple methods for Meta-analysis based on different frameworks for differing purposes. Network & Pairwise meta-analysis for example, are statistical methodologies commonly used in treatment comparison. Standard Pairwise practice as the name implies can compare the outcome of two variables (i.e safety/efficacy/outcome) of two treatments that have been directly compared in head-to-head clinical trials where as Network meta-analysis offers an extension on this where by any number of treatments can be compared.
So pairwise analyses a ‘pair’ of treatments and Network Meta-analysis is useful for analysing treatment effects where there are more than two possible interventions within a ‘network’ of trials.
Statistical Models (for aggregate data)
In the Fixed Effects model the inverse of a weighted-estimate (estimate of the studies) variance is leads to larger studies contributing more than smaller ones to the weighted average. This presents a problem as substantially large studies often render any smaller ones useless.
In the random effects model takes the average size of a group of studies and inversely weighs the variance, then the un-weighting process occurs in which a random effects variance taken from the sizes of the underlying studies is applied. This leads to more variability in effects sizes, thus potential for heterogeneity.
Direct evidence model
The quality effects model allows for inter-study variability by taking the contribution of variance along with the contribution of variance due to random error used in either fixed effects model to generate the weights for each study. The advantage of this is that it allows methodical evidence over subjective inference as weight can be adjusted according to the quality of study, hence the name.
Indirect Evidence model (Network Meta Analysis)
Network Meta Analysis can be undertaken using a few different methods. The first method is the Bucher method, which is a single or repeated comparison of a closed loop of three-treatments. One of the nodes is common-linked to the two studies and and forms the beginning and end of the loop. As such multiple two-by-two comparisons are needed.
Statistical modeling on the other hand can include multiple ‘arm’ trials and comparisons between all competing treatment therefore eliminating the rigor of two-by-two comparison.
Does Meta-analysis have any problems?
Yes, many of the problems stem from bias. Bias can be influenced from sources such as Publication Bias, Agenda bias and Sentiment bias.
Publication bias is an occurrence within academic research in which the decision to publish the results of an experiment of research is dependent upon the outcome. The problem arises because the current body of research and literature reviews supporting a certain hypothesis can be inherently disingenuous if the source of available publication is only the derivative of studies which prompted a significant finding.
Statistically significant results are three times more likely to be published over those returning null result. It is, essentially the flaw of the human mind, non-response bias for example (most common cause of non-publication) is thought to be caused by a feeling of having made a mistake, failure to support a known finding and anticipation that others will be uninterested in the results.
The end result is termed the ‘file draw’ problem in which the theoretical non-published results being tucked away in the ‘draw, or cabinet’ result in a biased distribution of effect sizes creating a serious base rate fallacy.
Agenda bias is a straight forward concept in which bias of economic, social or political nature influence or potentially abuse the meta-analysis. Picking favorable research, favoring authors and consciously selecting smaller favorable data sets are all aspects of agenda bias.
Despite these problems, many still favor Meta Analysis as a method for approaching muktiple studies over Narrative reviews as they can be subjective and not suitable for larger quantities of studies. Meta-Analysis is in principle an objective method before the element of human interaction.
BigBrainData 