Our Framework for Research Priorities
NCCAM, like all NIH Institutes and Centers, receives investigator-initiated applications for research funding that are based on ideas formulated by the applicant. As you might imagine, the research grant applications for complementary approaches cover quite a diverse and broad field.
Through NCCAM’s Third Strategic Plan, published in 2011, we have sought to clearly communicate the areas of research that we feel are ripe for investment now. The plan provides us with a roadmap for our research strategy and includes a framework of four factors we use to sharpen the focus of our research investments. These factors help NCCAM and our national advisory council to:
- Identify and plan targeted research initiatives
- Locate gaps in knowledge to help shape future research
- Balance the funding of specific initiatives with the funding of projects based on researchers’ creative ideas.
These factors, and some examples of questions NCCAM might ask related to each factor, are as follows:
- Scientific Promise
How strong is the body of evidence supporting the concept? - Amenability to Rigorous Scientific Inquiry
Are there reliable and reproducible methods—e.g., for diagnostics, outcome measures, biological effects, quality control? - Potential to Change Health Practices
Is it reasonably likely that the results will make a difference to consumers, providers, or policymakers? - Relationship to Use and Practice
Do the methods and approaches actually address the most important questions about use or practice in the real world?
I strongly urge anyone seeking NCCAM research funding to review their applications in light of these factors. NCCAM’s current portfolio reflects how these four factors are shaping our research investments.
Comments
Add your comment
It should be noted that:
1. Very few interventions promoted as “complementary” or “alternative” fare well in terms of factor #1. If they did fare well, there would be little need to refer to such methods with euphemisms such as complementary and alternative (instead of the more accurately descriptive terms: implausible and either non-validated or invalidated).
2. Very few advocates of interventions promoted as “complementary” or “alternative” prefer rigorous methods of scientific inquiry (factor #2) over uncontrolled investigations and poorly controlled investigations such as so-called pragmatic studies.
3. Very few advocates of interventions promoted as “complementary” or “alternative” would change their practices of offering or seeking such interventions in the face of negative outcomes emerging from rigorous scientific inquiry. It is quite easy to find promoters of invalidated methods labeled “complementary” or “alternative.”
4. Factor #4 represents the wedge by which investigators who are already devoted to interventions promoted as “complementary” or “alternative” will continue to rationalize the use of pragmatic studies rather than more rigorous scientific studies consistent with factor #2.
5. The questions offered under each factor are described as examples “NCCAM might ask related to each factor.” Let’s hope that this phrase doesn’t mean that NCCAM might choose to substitute these questions with questions that are less demanding of applicants.
As William London said, I fail to see how an entirely separate entity needs to be created and maintained in order to investigate specifically CAM interventions by such criteria. How does this criteria differ in any way from the standard investigations that the NIH performs? How does one define “CAM” in context to apply this principles of investigation to? What separates a novel medical intervention or therapy from a “CAM” intervention or therapy? And what purpose does such a distinction actually serve?
These are all genuine questions. I see no clear answer but would welcome Dr. Briggs to share her insights as I am always open to admit that I have missed something, despite the fact that I have explored these questions extensively and rigorously and have yet to come to a satisfactory answer.
I’d say if science/research is a service industry (to better the health of people) then in a world in which 30%-50% of what is done in medicine is waste, and much of it harmful (IOM Feb 2008, JAMA) #3 and #4 should be elevated to the top. If the integrative health postulate is correct that the best way to treat chronic conditions, and particularly those with multiple issues is through individualized strategies involving multiple modalities in a continuously iterative, responsive fashion, #1 and #2, as framed largely through the prevalent research mind, can be huge barriers to the service element in #3 and #4. Van Morrison sang that “the girls go by dressed up for each other.” He could be talking about many researchers.
Re nybgrus: the clear answer is that when the research industry scews up, focusing on #1 and #2, and doesn’t pay attention to a field of health care important to a growing number of those who research is purportedly supposed to serve (moving to #3 and #4), the industry need to be reminded that those issues and questions exist. Unfortunately, having the funds still hasn’t meant that they have been best applied to the optimal use - such as the kind of real world research Dan Cherkin and Ryan Bradley modeled for integrative, individualized treatment of people with diabetes.
John Weeks: I absolutely agree with the premise, though I don’t think that your numbers are correct. Of course, those numbers are very debatable and difficult to derive, so I won’t try and hash that out here.
However, do not confuse the application of “individual strategies involving multiple modalities…” with CAM or so-called “integrative” medicine. They are far from synonymous and indeed the actual practice of medicine is all about “individual strategies involving multiple modalities…”
I’ll be the first to admit that we often fail at executing this goal. But that is an indictment of the system in which we practice and administer medicine, not a nod towards some nebulous construct dubbed “CAM.” For example, since you brought up diabetes, in my own hospital we have a “Diabetes Boot Camp” wherein we take newly diagnosed diabetics (and those with difficulty controlling their disease) and sit down with them on an individual basis for an hour to discuss how to best tailor their diet, exercise, and medication regimen to best fit their life for maximal positive outcome. We educate them, describe the disease process, and discuss their every day lives to determine what barriers are in their lives specifically to optimum outcomes and adherence to healthful behaviors and medication regimens. We then tailor a plan to best suit each person, sometimes with the recognition that an A1c of <7 is not currently a reasonable goal and we should focus on getting them below 8 before moving on.
There is absolutely nothing “complimentary,” “alternative,” nor “integrative” about this approach. It is rooted in hard science with an understanding of patient centered medicine and population health. In other words, the barrier to this type of care is not something inherent to the practice of “mainstream” medicine, but inherent to the system of healthcare delivery and cost that makes such programs as this difficult if not impossible to administer on a wide and broad scale.
The “integration” of so-called CAM can add nothing to the paradigm to improve patient outcomes and optimal use. The restructuring of systems is the single best way to achieve the goals that you seek. If you look at Dr. Killen’s post on this blog you will note that he himself admits that essentially the last 10 years of NCCAM research would not have passed muster for scientific plausibility - things like homeopathy, faith healing, distance healing (such as reiki), and intercessory prayer to lower HIV titers simply are not plausible.
I will add that in my own medical training I have been and am constantly pressured to come up with plans and management options that will best suit the specifics of my patients - including things like level of education and understanding, socioeconomic status, living situation, access to care, ability to afford care - and tailor my management and discharge planning accordingly. I recently had a stroke patient with diabetes, hypertension, and no insurance. I worked extensively with the pharmacists and social workers to come up with a treatment regimen that she could afford, printed out forms for applications for drugs free of cost, referred her to free clinics, and had our diabetes educators come and sit with her to explain everything again and answer any and all questions.
How is that not describing “individualized strategies involving multiple modalities….”? And that is the standard of care and how we are taught as medical students to practice medicine. So indeed, the postulate of integrative medicine is correct - but the method for achieving that goal is simply not. Searching for highly implausible treatments and attempting to “integrate” them into actual medical practice is simply ineffective and inefficient. Any time something is found to be useful and integrated into medicine it is simply called medicine. Hence my inability to understand what utility a specific office for the research of CAM can do that would be in any way different than what the NIH does every day.
To put it another way, what objective criteria could one define to make a research project reasonable for NCCAM to study, yet the NIH at large would be unreasonable to study the same thing? If the criteria set above are to be applied uniformly, how could one possibly differentiate and say “This research project belongs in the NCCAM and this one in the NIH but NOT the other way around?”
Mr. Weeks’ argument is premised on what I call the “Trojan horse” of “integrative medicine.” Specifically, integrative medicine “rebrands” various modalities that are science-based (such as diet, exercise, etc.) as somehow “alternative.” That’s the Trojan horse. Lurking in the wooden horse are the scientifically implausible nonsensical treatments based on pseudoscience, prescientific thinking, or mystical belief systems. I’m talking about energy healing, chiropractic, homeopathy, and much of traditional Chinese medicine, to name a few examples. It’s been our argument that there is no need for a separate research center at the NIH to study the therapies that can potentially fall under #3 and #4. What makes anything CAM different than science-based medicine are the pseudoscience, prescientific beliefs, or mystical belief systems, and anything in CAM that can meet the four criteria described by Dr. Briggs isn’t really CAM.
CAM main function is to regulate the body’ s balance, NCCAM research should focus on to relieve symptoms such as pain, fatigue, insomnia…which do not result from diseases, or functional diseases, or integration with mainstream.
JX: That is exactly the function and purpose of actual medicine. How is CAM uniquely suited to do this? There exists no evidence that this is the case at all. Medicine is quite successful at, and continues to improve and do research on, things like pain, fatigue, and insomnia whether they result from “disease” or “functional disease” or not. And once something is proven to work it becomes mainstream whether it was the NCCAM or the NIH that demonstrated it. However, for the last 10 years the NCCAM has only had negative and highly equivocal studies, so I ask again - what is it that the NCCAM is uniquely positioned to do that the NIH or any other existing high caliber medical research group cannot?
The ‘trojan horse’ argument above points to exactly the reason what a separate research entity was and remains needed at the NIH. There are those such as some of those making comments in this forum that are and will always remain close minded to many of the systems of healing and therapies used in other cultures and ‘outside the mainstream’ of what is thought of as ‘western medicine’. As we have seen over time, closed minds in science prevent progress, exploration and getting to honest, ethical answers on benefit, safety, method of action, and dosing. Much has been accomplished, there have been stumbles along the way in some studies, and there remains much that needs doing. The advances in research that look at translational medicine such as Bayesian analysis for outcomes are well suited to the often complex, multi-pronged treatments of some systems of healing, in ways that a drug-model RCT approach has been challenging. Furthermore, there is much that needs developing in CAM on how to develop safe and effective research protocols involving children and individuals with disabilities. One major role of NCCAM is to fulfill the federal role in developing and overseeing research projects to insure the safety, adherence to protocol, and compliance with standards, policies and procedures. Let’s get good science done so that we can have accurate answers to the many research questions in the various fields - keeping in mind that there are those who will never been convinced that certain CAM therapies are legitimate or effective - no matter how much research is done. Those naysayers should not interfere with those who are interested in seeing research conducted.
I hope Beth Clay recognizes that the notion of closed-mindedness about “systems of healing and therapies used in other cultures and ‘outside the mainstream’ of what is thought of as ‘western medicine’ ” could be applied to those who fail to adequately consider how clinical illusions of effectiveness and safety contribute to belief in such systems. Vulnerability to seduction by fallacious, yet commonly offered, appeals to tradition or antiquity often closes minds to scientific evidence. It closes minds to the need to discard implausible methods for which support with solid scientific evidence remains elusive. Our minds should not be closed to the notion that extraordinary claims should be based upon extraordinarily solid evidence.
I’m afraid that the abstraction ‘western medicine’ isn’t meaningful or helpful since medicine that is consistent with well-established scientific theories and supported by rigorous clinical evidence is attractive throughout the Eastern world. The idea of Eastern medicine versus Western medicine is a false dichotomy that obscures how: (1) people all over the world have contributed to strengthening the scientific foundations of medicine, and (2) folk-medicine practices in the West and the East have developed based on pre-scientific notions such as vitalism, animism, representativeness, etc. The false East-West dichotomy of medicine is a distraction from a more meaningful dichotomy: medicine that has been validated in scientific investigation versus medicine that hasn’t been validated or has been invalidated in scientific investigation.
When people describe sectarian medicine based on pre-scientific theories as “alternative,” we should not take that to mean that, in a Bayesian analysis, that there is a reasonably good prior probability that the treatment will be a suitable substitute for scientifically supported treatment. When people describe such sectarian medicine as “complementary,” we should not assume that there is a reasonably good prior probability of the method adding favorably to science-based treatment in producing a favorable clinical outcome.
We should wonder what deficiencies a method has whenever its proponents use euphemisms such as “non-Western,” “alternative,” and “complementary” to imply the method is worthy of serious consideration. We should avoid closing our minds to the possibility that research investments into euphemism-based medicine will continue to yield meager, if any, advances in medicine.
@Beth Clay:
William London has said it quite well, so I have little to add actually.
I would like to point out though that even though you claim to have pointed out “exactly the reason [why] a separate research entity was and remains needed at the NIH” you actually have not done so even in the slightest. Furthermore, your claim of a “trojan horse” argument is quite ironically reversed - it is CAM that typically acts as a trojan horse, calling science based modalities such as diet and exercise “CAM” and thus attempting to claim legitimacy by association for the more ridiculous modalities such as Reiki and homeopathy.
The common *ad hominem* of “closed minded” also enters the repertoire. I can assure you that not only am I very open minded, that also the very principle of holding all claims to the same evidentiary standard is the epitome of being open minded.
You mention “many of the systems of healing and therapies used in other cultures and ‘outside the mainstream’ ” as if they had some demonstrated value that is being ignored and follow that up with “The advances in research that look at translational medicine such as Bayesian analysis for outcomes are well suited to the often complex, multi-pronged treatments of some systems of healing, in ways that a drug-model RCT approach has been challenging”
One thing to be certain of is that no matter what the attempted “system of healing and therapy” is employed, it *must* always be able to be measured by the same set of standards. While in some cases proper blinding and ethical consideration can make robust RCTs difficult to do, they are rarely impossible and furthermore there are other types of studies that can be used to lend insight. But regardless of that, outcomes can always be measured - if you claim that [X] can benefit someone then I needn’t know how [X] works and needn’t care about what “system of healing” it comes from. I can apply [X] and measure the outcome. The fact that essentially all of CAM cannot pass this basic test - as heartily demonstrated by 10 years of uniformly negative studies from the NCCAM itself - is all the evidence one needs to conclude it is time to move on.
Furthermore, Bayesian analysis has nothing inherently to do with translational research, nor with “complex multi-pronged treatments.” It is merely a way of quantifying the prior probability of *any* study in order to more accurately determine whether the p-value outcome of the frequentist analysis should properly be considered significant.
As an example, a p-value of 0.01 would be considered “significant” by any standard of frequentist analysis. If this were the result of a study measuring the effect of, say, a beta-blocker on heart rate this would certainly be significant. But if it were the result of a study measuring the effect of tooth placement on the amount of money left by the tooth fairy, we could reasonably conclude this was a spurious result. That is the application of the Bayes factor. So when a reiki study finds a p-value of 0.01 we can say that this is not actually likely to be significant, since the prior plausibility (aka Bayes factor) of reiki having an effect is extremely small.
Next you say “Furthermore, there is much that needs developing in CAM on how to develop safe and effective research protocols involving children and individuals with disabilities.”
That is jumping the gun. CAM needs to be demonstrated to be effective *first* and THEN we can go on to testing special populations. There is zero robust evidence of efficacy for anything currently labeled as “CAM” and until that is met, it would be unethical to run studies on special populations.
“Let’s get good science done so that we can have accurate answers to the many research questions in the various fields”
Indeed - and good science is that of a single standard. One that is well defined and used for a very good reason. So called “CAM” modalities have yet to meet that standard. Bad science would be creating a separate set of standards by which to judge something arbitrarily dubbed “CAM” and accepting it as equally efficacious to something “mainstream” which met a much higher standard.
“keeping in mind that there are those who will never been convinced that certain CAM therapies are legitimate or effective - no matter how much research is done”
And this is a ridiculous statement. Any reasonable scientist - myself included - will happily be convinced of the efficacy of ANYTHING with sufficient evidence. However, QUANTITY is not the key factor here, but QUALITY. And any study of sufficient QUALITY has yet to show any robust effect of CAM.
“Those naysayers should not interfere with those who are interested in seeing research conducted.”
Conduct the research. But do it by a uniform set of scientific standards. And employ prior plausibility to guide scarce resources to the highest likelihood of success.
Which once agains leads me to the question that always remains unanswered here:
What - SPECIFICALLY - can the NCCAM bring to a study that the NIH would not be able to? What are examples of studies that the NCCAM could somehow do that the NIH couldn’t, which would simultaneously meet rigorous scientific criteria and have sufficient prior plausibility?
I quite agree with David Gorski’s comparison of CAM to a “Trojan Horse”. I take great offense that CAM has co-opted the care that is presently provided to patients in our health care system, by physicians, nurses, dieticians and physiatrists…and have labeled that care as *holistic or integrated*.
The latest gambit of these alternative practitioners is to try and convince NCCAM to fund studies to push Medicare, Medicaid and private medical insurance companies to provide coverage and reimbursement to so-called “complementary” practitioners such as acupuncturists, homeopaths, naturopaths and “nutritionists”…based on some dicey studies…and based on *reduced costs*:
http://www.huffingtonpost.com/john-weeks/complementary-integrative-medicine_b_1916514.html?utm_hp_ref=healthy-living#comments
I sincerely hope that NCCAM will resist the efforts of the lobbyists representing CAM practitioners to undermine our public and private health care systems.
I agree with Dr. Gorski’s comment about the “Trojan Horse”.//The “claim” that alternative practitioners treat the “whole person”, not just diseases or disorders is ludicrous. Licensed physicians do recommend lifestyle changes, such as diets and exercise, low sodium food intake to control hypertension, etc. Physicians do refer patients to other licensed professionals such as registered and licensed dieticians (not self-styled “nutritionists” whose diploma or certificate is secured through on-line courses or through some schools. Nutritionists do not attend a university-based program for four years that stresses biology, chemistry and human physiology and they do not do internships in health care settings, then sit for the the rigorous boards that are administered under the auspices of the American Dietetic Association.//Chiropracters are a poor substitute for the care provided by licensed physical therapists who work under the auspices of orthopedists and physiatrists to restore function. What have all the studies funded by NCCAM proven about the effectiveness of chiropractic treatments…except for the temporary relief for lower back pain?//Naturopaths are not medical doctors; they cannot diagnose, treat or prescribe effective treatments for chronic health care problems such as hypertension, diabetes, pulmonary or cardiac problems. Naturopaths do not have the ability to diagnose or treat emerging health problems or make referrals to medical doctor specialists for treatment of serious medical disorders.//Homeopaths prescribe “medicine” that is composed of water that has not molecule of medicine left in it…after it has undergone multiple dilutions.//All of these “alternative practice specialists” sell supplements and other nostrums that do not undergo the rigorous testing that “traditional medicine” undergoes for effectiveness and safety, before licensing by the FDA.//Just this week we are hearing of the multi-state investigations of forty or more cases of fungal meningitis and at least 5 deaths, caused by contaminated steroid injections, that were concocted by one particular “compounding pharmacy” that were not FDA approved.
The Public Health Ministry of China has long since included in the “State Plan of Subjects for Priority Research” the topic of “How to raise the precision of Disease Pattern Identification in traditional Chinese medicine”.From qualitative description to quantitative analysis is universally acknowledged as the path along which a science must travel in its development. On the premise of strict conformity with the theoretical system of traditional Chinese medicine and the paramount rule of giving treatment on the basis of disease pattern identification, experts in traditional Chinese medicine arranged and studied information from three sources: the immense ancient and contemporary Chinese medical literature; the summing-up records and reports of famous traditional Chinese medical scholars’ experience in disease pattern identification and cases of their successful treatments; and the records of epidemiologic investigations. Information processing technology was employed to combine and quantify the information from these sources, whereby quantified data, the parameters of disease pattern identification, were obtained and used in compilation of the Quantitative Syndrome Differentiation Table.
Post a Comment
We welcome your comments and expect that they will be respectful. This is a moderated blog, and your comments will be reviewed before they are posted. Read our comment policy