The Effectiveness of Individualized Acupuncture Protocols in the Treatment of Gulf War Illness: A Pragmatic Randomized Clinical Trial

Abstract

Background

Gulf War Illness is a Complex Medical Illness characterized by multiple symptoms, including fatigue, sleep and mood disturbances, cognitive dysfunction, and musculoskeletal pain affecting veterans of the first Gulf War. No standard of care treatment exists.

Methods

This pragmatic Randomized Clinical Trial tested the effects of individualized acupuncture treatments offered in extant acupuncture practices in the community; practitioners had at least 5 years of experience plus additional training provided by the study. Veterans with diagnosed symptoms of Gulf War Illness were randomized to either six months of biweekly acupuncture treatments (group 1, n = 52) or 2 months of waitlist followed by weekly acupuncture treatments (group 2, n = 52). Measurements were taken at baseline, 2, 4 and 6 months. The primary outcome is the SF-36 physical component scale score (SF-36P) and the secondary outcome is the McGill Pain scale.

Results

Of the 104 subjects who underwent randomization, 85 completed the protocol (82%). A clinically and statistically significant average improvement of 9.4 points (p = 0.03) in the SF-36P was observed for group 1 at month 6 compared to group 2, adjusting for baseline pain. The secondary outcome of McGill pain index produced similar results; at 6 months, group 1 was estimated to experience a reduction of approximately 3.6 points (p = 0.04) compared to group 2.

Conclusions

Individualized acupuncture treatment of sufficient dose appears to offer significant relief of physical disability and pain for veterans with Gulf War Illness. This work was supported by the Office of the Assistant Secretary of Defense for Health Affairs through the Gulf War Illness Research Program under Award No. W81XWH-09-2-0064. Opinions, interpretations, conclusions and recommendations are those of the author and are not necessarily endorsed by the Department of Defense.

Trial Registration

ClinicalTrials.gov NCT01305811

About the Authors:

Lisa Conboy

* E-mail: [email protected]

Affiliation: New England School of Acupuncture, Newton, MA, United States of America

Travis Gerke

Affiliation: Department of Epidemiology, College of Medicine and College of Public Health and Health Professions, University of Florida, Gainesville, FL, United States of America

Kai-Yin Hsu

Affiliation: New England School of Acupuncture, Newton, MA, United States of America

Meredith St John

Affiliation: New England School of Acupuncture, Newton, MA, United States of America

Marc Goldstein

Affiliation: Boston Veterans Healthcare System/Jamaica Plain Campus, Boston, MA, United States of America

Rosa Schnyer

Affiliation: University of Texas at Austin, Austin, TX, United States of America

Introduction

Clinical and registry programs indicate that 25% of the 700,000 veterans of the first Gulf War (Operation Desert Shield/Storm, years 1990–1991), are affected by clusters of symptoms and co-morbid medical diagnoses including chronic fatigue syndrome, fibromyalgia, irritable bowel syndrome, arthralgia, digestive complaints, and mood-related psychiatric disorders, including depression, posttraumatic stress disorder (PTSD), and other anxiety disorders [1,2]. Defined by the Centers for Disease Control and Prevention (CDC) [3], Gulf War Illness (GWI) is a complex, poorly understood illness, often with a highly individualistic presentation, and symptoms difficult for conventional medicine to treat effectively; GWI has been shown to be remarkably stable at 5- and 10- year follow-ups [1,4]. GWI is twice as prevalent in deployed veterans, and seen in 15% of non-deployed veterans [5]. There is no standard of care treatment for this syndrome at this time.

Although there are no published studies evaluating acupuncture’s effectiveness in the treatment of GWI, acupuncture has been shown to successfully reduce many key symptoms of GWI including pain [5,6], musculoskeletal disorders [7,8,9], both acute and chronic pain after amputation in military contexts [10,11], fatigue [12], state, trait and situational anxiety [13], and depression [14,15,16]. Further, there is evidence that acupuncture may be effective in the treatment of other complex diseases such as irritable bowel syndrome [17], fibromyalgia [18], and post-traumatic stress disorder [19] and that acupuncture is well tolerated by patients, safe, and may be cost-effective compared to routine care [20].

Chinese Medicine, on which acupuncture is based, uses diagnostic and treatment procedures that are complex [21] and tailored to each individual’s specific symptoms. Although this individualized treatment ideal is often replaced in clinical research with standardized protocols for the purposes of reliability and simplicity, the ability of Traditional Chinese Medicine (TCM) to be tailored to each patient is a core concept and strength that can be maintained successfully in a Randomized Controlled Trial (RCT) format [22]. This unblinded Phase II clinical trial utilized individualized treatment protocols, testing the effects of individualized acupuncture treatments offered in extant acupuncture practices.

Materials and Methods

Study design

Full trial design details have been published elsewhere [23]. Please see S1 File: Protocol. To maximize study compliance we employed acupuncturists to provide treatments in their own offices in communities where veterans work and live. We began recruitment of subjects and practitioners with a catchment area of 30 miles from our research study offices, and widened our catchment area with increasing study duration.

Veterans with Gulf War Illness were randomized to either (1) acupuncture treatment twice per week for 6 months or (2) the wait-list comparison group consisting of usual care from baseline for 2 months, followed by weekly treatments for 4 months. We chose an active control group to maximize internal validity while allowing us to gather preliminary data on minimal effective treatment dose. The two month wait time allowed us to judge if GWI symptom presentations are stable in our sample, as has been shown in other GWI samples [1,5]. Our treatment schedule duration, dose, and specific Chinese Medicine techniques employed are based on the clinical experience of our expert practitioners, and informed by literature review. Details of the protocol and implementation were determined before the trial began via focus groups with senior acupuncture faculty at the New England School of Acupuncture.

The New England Institutional Review Board approved this research protocol on September 4, 2009. All human participants gave written informed consent. All of our study processes were approved and oversight is provided by: 1) The New England Institutional Review Board (http://www.neirb.com/), 2) United States Army Human Research Protection Office (https://mrmc-www.army.mil/rodorphrpo.asp). The study operated as planned between September 2009 and January 2013. Recruitment began immediately, and ran until July 2012. We initiated the clinical trial registration process before recruiting subjects, and registration was completed before we began to analyze the data. We confirm that all ongoing and related trials are registered. Please see S2 File. CONSORT Checklist.

Recruitment.

We recruited via local advertisements and direct mailing to veterans of the first Gulf War drawn from the Defense Manpower Data Center (http://www.virec.research.va.gov/Non-VADataSources/DMDC.htm). Because the demographics of GWI veterans are unpublished, we did not know if there was a sufficient population near our study offices from which to draw our sample. Thus we designed the study to include treatment sites within a 100-mile radius of our study offices, and incorporated a mechanism to add treatment sites within that radius in areas where GWI veterans were found clustered. Thirty treatment sites were utilized. This design has the added benefit of allowing veterans to attain treatments near where they live and work, a technique that may have improved adherence.

Eligibility.

All subjects met the illness definition of Gulf War Illness as determined by responses on the Gulf War Illness Symptom Checklist [5] and the inclusion/exclusion criteria set forth in the federal definition of Gulf War Illness as used for the Gulf War Registry (Please see Box 1). Subjects needed to pass through two eligibility screenings. First, a research assistant conducted a prescreening by phone, querying potential subjects about their illness and symptom experience. Second our study physician (MG) used the same criteria to complete an in-person medical screening. Our two-stage informed consent process included 1) verbal informed consent requested prior to the initial telephone screening, and 2) written informed consent administered in person, at the start of the screening visit. Please see Fig 1. An unblinded member of the study staff (LC) with no additional patient contact enrolled subjects. Study outcomes data were collected by electronic interface at our outpatient clinic. In fewer than 10% of cases, due to participants’ time constraints, participants were allowed to take their surveys home to fill out, and then mail back. During this screening visit, participants chose their future treatment practitioner from a list of practitioners convenient for them.

[Figure omitted. See PDF.]

Fig 1. PLOS CONSORT Flow Diagram: Diagram of Screening, Randomization, and Follow-up.

https://doi.org/10.1371/journal.pone.0149161.g001

Box 1: CDC Symptom Clusters for Gulf War Illness.

Inclusion:

1. deployed to the “Gulf Theater of operations, as defined by 38 CFR 3.317, includes Iraq, Kuwait, Saudi Arabia, Bahrain, Qatar, the United Arab Emirates, Oman, the Gulf of Aden, the Gulf of Oman, the Persian Gulf, the Arabian Sea, the Red Sea, and the airspace above all of these locations” in the years 1990–1,

2. have at least 2 of the following symptoms from the 3 CDC clusters of symptoms that have lasted for more than 6 months. Each symptom cluster must be characterized as “mild-moderate” or “severe” with at least one symptom in each cluster required to be severe.

1. A-Fatigability

* fatigue 24 hours or more after exertion

2. B-Mood and Cognition

* feeling depressed or

* feeling irritable or

* difficulty thinking or concentrating or

* feeling worried, tense, anxious or

* problems finding words or

* problems getting to sleep

3. C-Musculoskeletal

* joint pain or muscle pain

Exclusion:

Potential subjects will be excluded if they are:

* Currently enrolled in another clinical trial

* Have another disease that likely could account for the symptoms, as determined by our Medical Monitor

* Severe psychiatric illness (in the last 2 years psychiatric hospitalization, suicidal attempt, alcohol or substance abuse, use of antipsychotic medication) as measured by our primary screening instrument the Primary Care Evaluation of Mental Disorder (Prime MD).

* Unable to complete the protocol on based on the evaluation of the Medical Monitor

* Participants will not be excluded due to age, race, ethnicity, or gender limitations.

* Study is limited to United States veterans with Gulf War Syndrome, thus subjects could not be minors, illiterate, or unable to speak or understand English.

Group assignment and interventions.

Collaborator RD randomly assigned participants to the two study arms using permuted block randomization with variable block sizes and assignments provided in sequentially numbered opaque sealed envelopes. After baseline evaluation and consent, a member of the study staff without involvement in data collection with subjects (LC) opened the assignment envelopes and recorded the assignment of each participant in a confidential log. LC then reported to the subject’s chosen practitioner the subject’s contact information and study assignment. The chosen practitioner then took responsibility of contacting the subject to schedule and begin treatments. LC also called and/or emailed the subject to report if acupuncture treatments were to begin immediately (group 1), or in 2 months (group 2).

Licensed acupuncturists with at least 5 years of clinical experience, who received additional in-house training concerning GWI, provided the acupuncture treatments. Although there are many styles of acupuncture within Chinese Medicine, acupuncturists were chosen who self-reported use of the TCM model of diagnosis. During the first session, the acupuncturist conducted an interview reviewing the subject’s medical history, symptoms and aspects of diagnosis from the perspective of TCM, including condition of the tongue, pulse, meridians, and acupoints. Each subject received an individualized diagnosis and treatment protocol addressing his or her unique pattern of symptoms. Brief interviews began each subsequent session, allowing patient and practitioner to prioritize symptoms, and identify any questions or concerns. Individualized treatment protocols allowed the practitioners to alter the treatment plan based on how the patient presented at the moment; including varying the selection of acupoints across treatments and adding particular co-interventions commonly used as part of TCM therapy to supplement manual needling (such as electroacupuncture for its efficacy in reducing pain and inflammation [24], heat therapies (e.g. heat lamp), Chinese massage, and press balls, tacks or magnets applied to points after needling). A sample treatment protocol is offered as S3 File.

Each session lasted approximately one hour. Acupoints were stimulated manually until “obtaining de qi,” a technique characteristic of TCM to elicit a response felt by both the patient and the acupuncturist. This needling sensation, adjusted for the comfort and safety of each patient, may be experienced as a pinch that rapidly subsides, or a sense of spreading pressure, dull ache, or warmth. Needles were retained for 30–45 min (10–35 stainless steel, disposable needles per session). After needle insertion, subjects were left to rest or nap. The type of needle, including gauge [32–38] and length (15–50 mm) as well as the depth of insertion (subcutaneous to about 25 mm) varied according to the area of the body being treated (i.e. extremities vs. trunk). Choice of acupoints could vary during subsequent treatments to improve results. Herbs and supplements were not allowed. Subjects were encouraged (but not required) to remain with the same acupuncturist for the whole study period to allow for development of patient-practitioner rapport.

Outcomes.

A single measure of severity that addresses all possible presentations of GWI does not exist. Thus we chose to use the SF-36, a 36-item, well-validated and reliable general measure of health [25]. Given the importance of function on quality of life, we focused the main outcome to the SF-36 physical component scale score (SF-36P). Similarly, as pain is a common to most GWI presentations and very relevant in veteran health, our secondary outcome is the McGill Pain scale, a 15- item measure recording participants’ pain level and quality [26]. Outcomes were assessed with the assistance of a blinded study staff member at baseline, 2, 4, and 6 months. Raw data is offered as S4 File.

Subjects’ confidence in and usability of acupuncture were measured in a few ways. Participants were asked (1) their confidence in recommending acupuncture to a friend or family member using a five point scale from “Very Confident” to” Not Confident”, (2) the experience of the acupuncture using a five point scale from “Extremely Pleasant” to “Extremely Unpleasant”, (3) the experience of their relationship with their practitioner on a five point scale from “Extremely Pleasant” to “Extremely Unpleasant”, (4) how logical the acupuncture treatments were for them on a 6 point scale of “Very Logical” to “Not Logical at All”.

Sample size

Our sample size was calculated to allow detection of clinically meaningful differences between treatment groups. Previous acupuncture research using our main outcome, the SF-36, in pain conditions [27,28] show a consistent standard deviation of 20 points in the SF-36 P for both baseline values and change scores. Sixty individuals per group (total n = 120) would offer us a power of 80% to detect the difference between groups of 7 points. Using Cohen’s d estimation of effect size [29] a sample size of 60 would allow us to see a moderate effect. In further support of our main outcome, a 7.8-unit improvement has been estimated clinically relevant for patients with similarly serious conditions [30,31]. We estimated a dropout rate of 10%.

Statistical analyses

In our original proposal to the funder, to protect our main outcome from possibly large attrition, we proposed to initially test mean differences between groups following 2 months of treatment using Student’s t-tests at an alpha = 0.05. Using this strategy, we observed a mean reduction in SF-36 for Group 1 of 0.32 versus a reduction 4.53 for Group 2 (p = 0.22).

Of more interest to the study team is what changes might be seen after the clinically informed 6 month treatment window. This 6 month analysis is done by author TG and investigates whether those subjects assigned to biweekly acupuncture experienced differences in the SF-36P score over follow-up compared to those subjects who received weekly acupuncture following a 2-month delay. To assess potential differences, generalized estimating equation (GEE) models were fit in order to account for the correlation induced by repeated measurements on each subject. Under the assumption that baseline McGill pain is prognostic for SF-36P over time, model adjustment was made for baseline pain to increase precision in the estimated parameters.32 Eight subjects did not report this baseline measurement and, under the assumption that missingness was completely at random, these subjects were not included in the analysis set. In summary, we estimated the GEE modelwhere Itx denotes an indicator for biweekly acupuncture; It2,It4,and It6, are indicators for months 2, 4, and 6; ρ denotes baseline pain; and time was coded categorically to reflect suspected nonlinearities.

As a secondary analysis, the McGill pain score was similarly assessed for differences over time by treatment status. The time trend for the GEE fit was modeled categorically to account for nonlinear trends. The pain model was not adjusted for additional covariates, since no strongly prognostic variables were assumed to have been measured. All GEE models were fit using the software package geepack [33] in R version 3.1.1 under an exchangeable working correlation structure.

Results and Discussion

Study Population

Recruitment began in July of 2010 and the final follow-up visit was completed in January 2013. Please see Table 1 for baseline demographics and Fig 1. for study flow.

[Figure omitted. See PDF.]

Table 1. Baseline Characteristics of the Study Population.

Baseline Characteristics of the Study Population. Standard Deviations (SD) are offered for age, baseline pain, and baseline Sf-36(P).

https://doi.org/10.1371/journal.pone.0149161.t001

Outcomes

Of the 104 subjects who underwent randomization 103 completed at least one measurement timepoint, yielding 99.0% of data for analysis. However, 8 subjects are missing baseline pain data, yielding 95/104 = 91.3% in the analysis set. General Estimating Equations were used to compare the group 1 vs. group 2 acupuncture subjects’ experienced differences in the SF-36P, and McGill Pain scale.

Our analysis comparing baseline symptom levels to those at 6 months showed a significant average increase of 9.4 points in the SF-36 physical component scale score (SF-36P) for the biweekly acupuncture group at month 6 in comparison to the weekly waitlisted group, adjusting for baseline pain. Within the biweekly group, scores were generally stable at months 0, 2, and 4, with a mean increase of 6.6 points in SF-36P observed at month 6. A nonlinear pattern was observed for the waitlisted group, though a lower SF-36P relative to baseline was estimated for all subsequent months. Table 2 offers mean and empirical 95% confidence limits estimated for both SF-36P and McGill from 5000 simulations of the fitted GEE models, and Fig 2 provides a graphical summary of this information for SF-36P [34]. Though overlapping confidence bands are observed at time points determined significantly different in the GEE fit, we note that the simulation-based approach, while useful for visual interpretation, relies on slightly different calculations than the GEE model itself.

[Figure omitted. See PDF.]

Fig 2. Summary of model-based simulationsxxxiii of changes in mean SF-36P at 4 measurement timepoints.

Scores moving in the positive direction indicate improvement. Scores moving in the positive direction indicate improvement.

https://doi.org/10.1371/journal.pone.0149161.g002

[Figure omitted. See PDF.]

Table 2. Mean and empirical 95% confidence limits estimated for both SF-36P and McGill from 5000 simulations of the fitted GEE models.

https://doi.org/10.1371/journal.pone.0149161.t002

The secondary outcome of McGill pain index produced similar results. A decreasing trend in McGill score indicating symptom improvement was observed for the biweekly group, with a significantly lower score compared to the waitlisted group appearing at month 6. At the end of follow-up at month 6, the biweekly group was estimated to experience a reduction of approximately 3.8 points on the McGill scale compared to the weekly waitlisted group. Fig 3 illustrates the estimated value and 95% confidence intervals. Tables 3 and 4 offer estimates and accompanying statistical values from the GEE modeling of the two outcomes.

[Figure omitted. See PDF.]

Fig 3. Summary of model-based simulationsxxxiii of changes in mean McGill Pain Scale at 4 measurement timepoints.

Scores moving in the negative direction indicate improvement. Scores moving in the negative direction indicate improvement.

https://doi.org/10.1371/journal.pone.0149161.g003

[Figure omitted. See PDF.]

Table 3. Estimates and accompanying statistical values from the GEE modeling of the 6-month outcome SF-36P.

https://doi.org/10.1371/journal.pone.0149161.t003

[Figure omitted. See PDF.]

Table 4. Estimates and accompanying statistical values from the GEE modeling of the outcome McGill Pain scale.

https://doi.org/10.1371/journal.pone.0149161.t004

Participants reported high usability of acupuncture with 96% of the veterans (averaged across both groups and over all time points) reporting confidence in recommending acupuncture to a friend or family member (or at least a 3 on a five point scale from “Very Confident” to” Not Confident”), 98% reporting that the acupuncture experience was at least pleasant (or at least a 3 on a five point scale from “Extremely Pleasant” to “Extremely Unpleasant”), 97% reporting that their relationship with their practitioner was pleasant (or at least a 3 on a five point scale from “Extremely Pleasant” to “Extremely Unpleasant”) and 96% reported that the acupuncture treatments were logical for them (or at least a 3 on a 6 point scale of “Very Logical” to “Not Logical at All”). The trial had only two adverse events: (1) subject in biweekly treatment group reported pain on needling, (2) subject in weekly treatment group reported suicidal thoughts, which study staff followed up with additional medical oversight.

Discussion

This study supports the use of individualized acupuncture treatments for the management of GWI symptoms. Our results are in concordance with numerous other studies indicating that acupuncture is a widely available, safe, effective, and cost-effective option for the treatment of other diseases and syndromes with similar presentations to GWI [35] with high usability in veteran populations [36]. Given this research, it is likely that acupuncture treatment may be an effective, safe, low-cost treatment option for our returning military as well as civilian populations impacted by chronic multi-symptom illness and its co-morbidities.

The mechanisms of acupuncture in the treatment of GWI are unknown, which supports our choice of a low-constraint design. This naturalistic RCT includes individualized protocols, a clinically supported length and dose of treatment, and a wait list control. Data from our wait list arm (Table 2) indicates that symptoms are stable, as has been shown in published 5- and 10- year follow-ups [1,5]. The design aspect of the wait list group eventually receiving weekly acupuncture offers us data to begin to answer questions of minimal dose and satisfies ethical concerns allowing all subjects to receive treatment during the study. A sham acupuncture control arm was not used due to published indications that such sham interventions are effective and thus not appropriate controls; very high quality evidence of this is now available [37,38].

We chose a pragmatic design, and used practitioners in the community, to facilitate adherence and test the use of extant practitioners. Our positive results support the referral of GWI veterans to acupuncture treatments. Our low side effect rate mirrors that of the published literature that acupuncture is safe when provided by professionally trained practitioners [39]. Serious adverse events are extremely rare. In a systematic review of 12 prospective studies scrutinizing over a million treatments, the very low risk of serious adverse event, mostly trauma from needle puncture or infection, was estimated at 0.05 per 10,000 treatments, a risk below that of many common medical treatments [40]. Acupuncture is well tolerated, safe and effective in the management of Gulf War Syndrome. The inclusion of acupuncture in the routine management of this intractable condition is warranted.

Limitations and Future Directions

Our results suggest that 2 months of biweekly acupuncture is not sufficient to affect the outcome of physical function (as measured by the SF-36P), but pain scores (as measured by the McGill Pain Scale) did show group improvement as early as the first follow-up (2 months). These findings underscore the need for more dosage studies to determine the most therapeutic level of treatment for different illness presentations. Currently, we are conducting secondary data analyses exploring the effectiveness of acupuncture treatment on different subtypes of GWI to help treatment providers apply the best protocols for this complex illness. The team is also categorizing the most effective treatment protocols from a Traditional Chinese Medicine point of view and matching these with different biomedical symptom presentations. Our low-constraint/non-standardized design allowed for collection of naturalistic clinical data, which may increase data validity and make such communications across medical systems more useful.

Other items related to observation window are also of interest. For example, the obvious yet non-statistically significant decrease in Sf-36P component scores which happened while the veterans were waiting for treatment could be natural history but probably is not as observations over longer periods show symptom stability over time [41]. Most likely, this symptom change is due to veterans’ frustration in having to wait for treatment; a few of the veterans mentioned this frustration along with an acknowledgment that they were informed of the necessity for a wait list design and knowledge that they would receive treatment at 2 months. We will explore other changes which occurred during the wait list in later analyses as well as the symptom changes associated with different doses of acupuncture.

Although we did find support for our 6-month hypothesis we did not achieve the sample size determined to give us 80% power to see an effect. This leaves us more vulnerable to Type II (false negative) error. Future work should include larger sample sizes to protect against this. Larger sample sizes may also more easily support the use of standardized protocols, which become possible with the implementation of effectiveness trials such as ours that gather a range of clinically relevant treatment options

This work was supported by the Office of the Assistant Secretary of Defense for Health Affairs through the Gulf War Illness Research Program under Award No. W81XWH-09-2-0064. Opinions, interpretations, conclusions and recommendations are those of the author and are not necessarily endorsed by the Department of Defense. The sponsor was not involved in the study design; in the collection, analysis or interpretation of data; in the writing of the report; or in the decision to submit the paper for publication. Full trial protocol is available by request of the principle investigator (LC).

Supporting Information

[Figure omitted. See PDF.]

S1 File. Protocol.

https://doi.org/10.1371/journal.pone.0149161.s001

(DOC)

S2 File. CONSORT Checklist.

https://doi.org/10.1371/journal.pone.0149161.s002

(DOC)

S3 File. Example Treatment Protocol for Subject in Biweekly Treatment Condition.

https://doi.org/10.1371/journal.pone.0149161.s003

(DOCX)

S4 File. Data Used in Calculations.

https://doi.org/10.1371/journal.pone.0149161.s004

(XLSX)

Acknowledgments

Thank you to collaborator Roger Davis ScD (RD) of Beth Israel Deaconess Medical Center, Harvard Medical School, for the creation of the randomization materials.

Author Contributions

Conceived and designed the experiments: LC MS RS. Performed the experiments: MG KH LC. Analyzed the data: LC TG. Contributed reagents/materials/analysis tools: LC TG. Wrote the paper: LC TG RS MS KH.

Citation: Conboy L, Gerke T, Hsu K-Y, St John M, Goldstein M, Schnyer R (2016) The Effectiveness of Individualized Acupuncture Protocols in the Treatment of Gulf War Illness: A Pragmatic Randomized Clinical Trial. PLoS ONE 11(3): e0149161. https://doi.org/10.1371/journal.pone.0149161

References

1. Steele L, Sastre A, Gerkovich MM, Cook MR. Complex Factors in the Etiology of Gulf War Illness: Wartime Exposures and Risk Factors in Veteran Subgroups. Environ Health Perspect. 2012 Jan;120(1): 112–118. pmid:21930452

2. Kang HK, Li B, Mahan CM, Eisen SA, Engel CC. Health of US veterans of 1991 Gulf War: a follow-up survey in 10 years. J Occup Environ Med. 2009 Apr;51(4): 401–10. pmid:19322107

3. Centers for Disease Control and Prevention. Unexplained illness among Persian Gulf War veterans in an Air National Guard unit: preliminary report–August 1990-March 1995. MMWR Morb Mortal Wkly Rep.1995;44: 443–447. pmid:7770029

4. Blanchard MS, Eisen SA, Alpern R, Karlinsky J, Toomey R, Reda DJ, et al. Chronic multisymptom illness complex in Gulf War I veterans 10 years later. Am J Epidemiol. 2006 Jan 1;163 (1): 66–75. Epub 2005 Nov 17. pmid:16293719

5. Bullock ML, Pheley AM, Kiresuk TJ, Lenz SK, Culliton PD. Characteristics and complaints of patients seeking therapy at a hospital-based alternative medicine clinic. J Altern Complement Med. 1997;3(1): 31–7. pmid:9395692

6. Cassidy CM, Chinese medicine users in the United States. Part I. J Altern Complement Med. 1998;4(1): 17–27. pmid:9553832

7. Haake M, Muller HH, Schade-Brittinger C, Basler HD, Schäfer H, Maier C. et al. German acupuncture trials (GERAC) for chronic low back pain. Arch Intern Med. 2007;167(17): 1892–1898. pmid:17893311

8. Berman BM, Ezzo J, Hadhazy V, Swyers JP. Is acupuncture effective in the treatment of fibromyalgia? J Fam Pract. 1999 Mar;48(3): 213–8. pmid:10086765

9. Tukmachi E, Jubb R, Dempsey E, Jones P. The effect of acupuncture on the symptoms of knee osteoarthritis-an open randomized controlled study. Acupunct Med. 2004 Jun; 22 (1): 14–22. pmid:15077933

10. Niemtzow R, Gambel J, Helms J, Pock A, Burns S, Baxter J. Integrating ear and scalp acupuncture techniques into the care of blast-injured United States military service members with limb loss. J Altern Complement Med. 2006;12(7): 596–599. pmid:16970527

11. Goertz C, Niemtzow R, Burns S, Fritts M, Crawford C, Jonas W. Auricular acupuncture in the treatment of acute pain syndromes: a pilot study. Mil Med. 2006;171(10): 1010–1014. pmid:17076456

12. Vickers AJ, Rees RW, Zollman CE, McCarney R, Smith CM, Ellis N, et al. Acupuncture of chronic headache disorders in primary care: randomised controlled trial and economic analysis. Health Technol Assess. 2004 Nov;8(48):iii, 1–35.

13. Pilkington K, Kirkwood G, Rampes H, Cummings M, Richardson J. Acupuncture for anxiety and anxiety disorders–a systematic review. Acupunct Med 2007;12(1–2): 1–10.

14. Allen JB, Schnyer R, Hitt S. The efficacy of acupuncture in the treatment of major depression in women. Psychol Sci.1998;5: 397–401.

15. Allen JJ, Schnyer RN, Chambers AS, Hitt SK, Moreno FA, Manber R. Acupuncture for depression: a randomized controlled trial. J Clin Psychiatry. 2006 Nov;67(11): 1665–73. pmid:17196044

16. Manber R, Schnyer RN, Allen JJ, Rush AJ, Blasey CM. Acupuncture: a promising treatment of depression during pregnancy. J Affect Disord 2004;83: 89–95. pmid:15546651

17. Fireman Z, Segal A., Kopelman Y, Sternberg A, Carasso R. Acupuncture treatment for irritable bowel syndrome, Digestion. 2001;64: 100–103. pmid:11684823

18. Duncan B, White A, Rahman A. Acupuncture in the treatment of fibromyalgia in tertiary care—a case series, Acupunct Med. 2007 Dec;25(4): 137–47.

19. Hollifield M, Sinclair-Lian N, Warner T, Hammerschlag R. Acupuncture for posttraumatic stress disorder. J Nerv Ment Dis. 2007;195(6): 504–513. pmid:17568299

20. Lao L. Safety issues in acupuncture, J Altern Complement Med. 1996;2: 27–31. pmid:9395638

21. Patterson C and Britten N. Acupuncture as a complex intervention: a holistic model. J Altern Complement Med. 2004;5: 791–801.

22. Kaptchuk TJ, Kelley JM, Conboy LA, Davis RB, Kerr CE, Jacobson EE, et al. Components of placebo effect: randomised controlled trial in patients with irritable bowel syndrome. BMJ. 2008 May 3;336(7651): 999–1003. Epub 2008 Apr 3. pmid:18390493

23. Conboy L, St John M, Schnyer R. The effectiveness of acupuncture in the treatment of Gulf War Illness. Contemp Clin Trials. 2012 May;33(3): 557–62. Epub 2012 Feb 10. pmid:22349455

24. Napadow V, Ahn A, Longhurst J, Lao L, Stener-Victorin E, Harris R. et al. The status and future of acupuncture mechanism research. J Altern Complement Med. 2008;14(7): 861–869. pmid:18803495

25. McHorney CA, Ware JE Jr, Lu JF, Sherbourne CD. The MOS 36-item Short-Form Health Survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Med Care. 1994;32(1): 40–66. pmid:8277801

26. Melzak R. The short form McGill Pain Questionnaire. Pain. 1987;30: 191–7. pmid:3670870

27. Jena S, Witt CM, Brinkhaus B, Wegscheider K, Willich SN. Acupuncture in patients with headache. Cephalalgia. 2008 Sep;28(9): 969–79. Epub 2008 Jul 8.

28. Manheimer E, Lim B, Lao L, Berman B. Acupuncture for knee osteoarthritis—a randomised trial using a novel sham. Acupunct Med. 2006 Dec;24 Suppl: S7–14. pmid:17308513

29. Cohen J. Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum; 1988.

30. Wyrwich KW, Nienaber NA, Tierney WM, Wolinsky FD. Linking clinical relevance and statistical significance in evaluating intra-individual changes in health-related quality of life. Med Care. 1999;37(5): 469–78. pmid:10335749

31. Wyrwich KW, Tierney WM, Babu AN, Kroenke K, Wolinsky FD. A Comparison of Clinically Important Differences in Health-related Quality of Life for Patients with Chronic Lung Disease, Asthma, or Heart Disease. Health Serv Res. 2005;40(2): 577–592. pmid:15762908

32. Kahan BC, Jairath V,Doré CJ, Morris TP. The risks and rewards of covariate adjustment in randomized trials: an assessment of 12 outcomes from 8 studies. Trials. 2014;15: 139. pmid:24755011

33. Højsgaard S., Halekoh U. & Yan J. The R Package geepack for Generalized Estimating Equations. J Statistical Softw. 2006;15(2): 1–11.

34. King G, Tomz M, Wittenberg J. (2000). Making the Most of Statistical Analyses: Improving Interpretation and Presentation. Am J Pol Sci. 44(2): 341–355.

35. Burke A, Upchurch DM, Dye C, Chyu L. Acupuncture use in the United States: findings from the National Health Interview Survey. J Altern Complement Med. 2006 Sep;12(7): 639–48. pmid:16970534

36. Hull A, Reinhard M, McCoy K, Akhter J, Duncan A, Soltes K, et al. P02.138. Acupuncture and meditation for military veterans: patient satisfaction and self reported symptom reduction. BMC Complement Altern Med. 2012;12: 194.

37. Vickers AJ, Cronin AM, Maschino AC, Lewith G, MacPherson H, Foster NE, et al. Acupuncture for chronic pain: individual patient data meta-analysis. Arch Intern Med. 2012 Oct 22;172(19): 1444–53. pmid:22965186

38. Vickers AJ; Linde K. Acupuncture for Chronic Pain. JAMA.2014.311(9):955–6. pmid:24595780

39. Melchart D, Weidenhammer W, Streng A, Reitmayr S, Hoppe A, Ernst E, et al. Prospective investigation of adverse effects of acupuncture in 97733 patients. Arch Intern Med. 2004 Jan 12;164(1): 104–5. pmid:14718331

40. Witt CM, Pach D, Brinkhaus B, Wruck K, Tag B, Mank S, et. al. Safety of acupuncture: results of a prospective observational study with 229,230 patients and introduction of a medical information and consent form, Forsch Komplementmed 2009;16: 91–97. pmid:19420954

41. Ozakinci G, Hallman WK, Kipen HM. Persistence of symptoms in veterans of the First Gulf War: 5-year follow-up. Environ Health Perspect. 2006; Oct;114(10): 1553–7. pmid:17035142