Not surprisingly, once we had established that the body of evidence about the effective ness of Bobath had moved from ‘no evidence of effect’ to ‘evidence of no effect’, the Children’s Bobath tutor and others supporters raised further methodological issues about the research.
How systemic reviews work
Firstly, the Children’s Bobath tutor raised that not all evidence had been included in the Novak review “Novak did not include at least 3 systematic reviews, which may have had impact on conclusions. In particular one by Franki et al⁷, which demonstrated level II evidence that NDT was effective on Gross Motor Function”.
I replied: The reason these reviews and papers are not included in Novak’s review is that they did not meet the selection criteria for the review. That’s the way a systematic review works; one sets out the research question to be addressed and the quality standards that will accepted and then selects the evidence that fits. Evidence is excluded because it isn’t relevant or isn’t good enough.
The Children’s Bobath tutor also raised that Novak et al had “excluded much lower level research. There is a fair body of lower level research which does reflect positive outcomes for Bobath-NDT. They recognise that level 1 RCTs are not the only way forward. Other types of research (eg lower level-non RCT) may be more suitable for this group” [children with CP].
I replied: “…….the Children’s Bobath tutor’s comment that Novak has excluded low levels of research that reflect positive outcomes for Bobath-NDT which she feels is more suitable, with an implication that Novak has been remiss in excluding this ‘supportive’ evidence. With respect, this suggests a misunderstanding of the levels of evidence. The whole idea of a systematic review is that one excludes low levels of evidence, because they are inherently open to scientific bias and therefore the conclusions drawn from that research cannot be relied on. Evidence-based practice is a matter of taking an objective view of the quality of the evidence available, selecting the strongest (ie with the least risk of bias) and summarising that. It is not a matter of cherry-picking and promoting the type that of evidence that give one the answers one wants. Randomised controlled trials of interventions for people with cerebral palsy (as with other complex, multi-faceted long-term conditions) are entirely possible, as demonstrated by Novak in her review, and many other reviews and reports of good trials.