Objective. This study aimed to investigate the methodological quality of clinical guidelines (CGs) for integrated Chinese and Western medicine (ICWM) to inform clinical practice and guideline development. Methods. We searched PubMed, EMBASE, Chinese Biomedical Literature Database, China National Knowledge Infrastructure, Wanfang Data, VIP, five guideline databases, and four online book malls to identify ICWM CGs published up to January 11, 2019. Four independent appraisers assessed the quality of CGs using the Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument and evaluated six specific concerns for ICWM. The standardized scores were calculated for the individual AGREE II domains. Results. Sixty-two ICWM CGs were included. The median standardized scores in the six domains of AGREE II were 65% in scope and purpose, 46% in clarity of presentation, 26% in applicability, 24% in stakeholder involvement, 15% in rigor of development, and 0% in editorial independence. The quality of ICWM CGs was significantly associated with the publication year (higher quality for CGs published after 2014) and the development method (higher quality for evidence-based CGs). Only one ICWM CG obtained a direct recommendation for use, and 14 could be recommended for use after modifications. The intra-appraiser consistency of the AGREE II appraisal was good (mean intraclass correlation coefficient range, 0.813–0.998). ICWM CGs also lacked a systematic search of ancient traditional Chinese medicine (TCM) classics (40.3%), conversion of TCM recommendations from ancient Chinese to the vernacular (14.5%), a discussion of interactions between TCM and Western medicine (27.4%), and rankings of different ICWM choices (0%). Conclusions. Although an improvement after 2014 occurred, the current 64 ICWM CGs are generally of poor methodological quality. Only 15 ICWM CGs can be recommended for use directly or with modifications. As the key distinctions from Western/Chinese medicine CGs, the ICWM-specific recommendations are also insufficient for the ICWM CGs, especially for interactions between TCM and Western medicine and rankings of different ICWM choices. Study Registration. This study has been registered at PROSPERO (no. CRD42018095767).
1. Introduction
Although Western medicine has become the mainstay of the health care system in China, traditional Chinese medicine (TCM), an ancient medical approach with 3000 years of history, is still widely practiced [1, 2]. Integrated Chinese and Western medicine (ICWM) has proven to be more effective than either treatment style alone for many conditions, especially some chronic refractory conditions such as osteoarthritis and functional gastrointestinal disorders, as well as side effects from chemotherapy [3, 4]. In China, more than 90% of Western medicine practitioners and almost all TCM practitioners are simultaneously prescribing Chinese and Western medicine [5]. Moreover, ICWM has received specific support from the Chinese government as a basic health development strategy [6].
ICWM practice, however, is difficult. TCM is fundamentally different from Western medicine in both theory and practice. Western medicine is based on modern scientific methods and evidence, whereas TCM uses philosophical perspectives to explain physiological and pathological changes in the human body, as manifested in its unique diagnostic method of syndrome differentiation and use of Chinese herbs and acupuncture to treat diseases [7]. Due to the limitation of professional education, few healthcare practitioners master both Western medicine and TCM, and this is associated with ineffective application of ICWM and may increase costs and safety concerns [8]. In fact, most Western medicine practitioners do not understand TCM theory, such as syndrome differentiation, and the properties, compatibility, and contraindications of Chinese herbs [5]. Indeed, 21% of adverse events caused by TCM were associated with inappropriate combinations of Chinese and Western medicine [9]. TCM practitioners also lack evidence-based concepts and knowledge to make clinical decisions in Western medicine practice. They may be afraid that Western medicine is not sufficiently effective or safe and thus abandon this treatment approach, which also creates a risk of delaying treatment and missing the optimal time to treat a condition.
An essential approach to improving ICWM practice is to develop and utilize clinical guidelines (CGs), which are recommendation documents systematically developed to standardize clinical decisions in specific clinical settings [10]. To date, dozens of ICWM CGs have been released and have influenced decision-making and patient outcomes. The actual efficacy of ICWM CGs is, of course, closely related to their quality.
Many efforts have been made by methodologists to improve the quality of CGs. The Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument, developed in 2009, is a representative achievement of these efforts to assess the quality of CGs [11]. Owing to its advantages in structure and feasibility, AGREE II has a myriad of uses worldwide [12]. Overall, AGREE II is applicable to ICWM CGs, but many issues regarding the particular characteristics of ICWM, such as the vernacularization of ancient Chinese in TCM recommendations and the interactions between Chinese and Western medicine, should be further considered [13].
A previous study explored the quality of ICWM CGs published before 2014 [14]. However, this assessment was out of date and did not consider ICWM-specific concerns. With the publication of an increasing number of ICWM CGs since that previous study, we aimed to conduct an updated survey to inform clinical practice and guideline development by thoroughly investigating the methodological quality and ICWM-specific concerns of ICWM CGs.
2. Methods
2.1. Study Design
This is a survey of currently available ICWM CGs published before January 11, 2019, using the AGREE II instrument. The study protocol is registered at PROSPERO (no. CRD42018095767). We report this study following the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) criteria where applicable [15].
2.2. Searches
We searched PubMed, EMBASE, Chinese Biomedical Literature Database, China National Knowledge Infrastructure, Wanfang Data, VIP, five guideline databases (the National Guideline Clearinghouse, the Guidelines International Network, the National Institute for Health and Clinical Excellence, the Scottish Intercollegiate Guidelines Network, and MedLive), and four online book malls (http://www.amazon.com, http://www.jd.com, http://www.amazon.cn, and http://www.dangdang.com). The keywords used for the search included “integrated Chinese and Western,” “guideline,” “consensus,” and “recommendation,” among others (see details in Table S1 in the supplementary file).
2.3. Inclusion Criteria
We defined ICWM CGs as CGs that simultaneously focused on TCM and Western medicine for diagnosis or treatment. We used the World Health Organization’s definition of CGs, i.e., systematic recommendations developed to assist any healthcare provider in making optimal clinical decisions [10]. The CGs were further divided into two types: (1) evidence-based CGs developed according to systematic searches and a summary of research evidence and (2) consensus-based guidelines developed according to scant research evidence or only expert opinions and consensus [16]. If a guideline had multiple versions, the final version was included. There were no restrictions regarding language.
2.4. Exclusion Criteria
The following CGs were excluded: (1) CGs developed for TCM or Western medicine alone; (2) no full text available; (3) translations; (4) CGs developed by a single author; (5) guideline-like textbooks; and (6) systematic reviews, narrative reviews, clinical pathways, and clinician training manuals.
2.5. Study Screening and Data Extraction
Two appraisers, independently and in duplicate, screened the bibliographies to identify potentially eligible CGs and then read the full text to determine the final eligibility. Discrepancies were resolved by discussion.
The following information was extracted from the CGs included using a standardized, pilot-tested form: authors, publication year, publication form, type of CG (evidence-based or consensus-based), type of developer, scope, condition, numbers of pages and references, consensus method, evidence grading system, composition of development team, and funding sources.
2.6. Appraisal of Guidelines
We used the AGREE II instrument to assess the quality of ICWM CGs across the following six domains: scope and purpose (3 items), stakeholder involvement (3 items), rigor of development (8 items), clarity of presentation (3 items), applicability (4 items), and editorial independence (2 items). The items in AGREE II are scored from strongly disagree (1 point) to strongly agree (7 points). A standardized score for each domain was calculated using the following formula: ((actual score − minimum score)/(maximum score − minimum score)) × 100%. Finally, we performed an overall assessment of “whether you recommend this guideline for use” with the following standards proposed by previous studies [17, 18]: (1) recommend, all domains scored ≥60%; (2) recommend with modifications, less than three domains scored <30% but one or more domains scored <60%; and (3) not recommend: three or more domains scored <30%.
Four appraisers who had experience in the development and appraisal of CGs independently assessed the quality of each CG. To enhance the consistency between the appraisers, they were trained in both English and Chinese versions of the AGREE II user’s manual [19, 20], a paper of detailed explanations of the AGREE II assessment for the CGs of Chinese medicine [21], and an online AGREE II training tutorial (https://www.agreetrust.org/resource-centre/agree-ii/agree-ii-training-tools/). Before the formal appraisal, the appraisers performed a pilot exercise for two ICWM guidelines with different methodological qualities and discussed and addressed discrepancies. The formal appraisal was performed using an official online platform “My AGREE PLUS (https://http://www.agreetrust.org/my-agree/),” which is widely used in AGREE II appraisals and may enhance consistency between appraisers [22, 23].
To assess whether the CGs appropriately specified key concerns for ICWM, we consulted five clinicians each in Western medicine and Chinese medicine to collect the information they were most interested in and the gaps they most frequently faced when using ICWM guidelines. We determined the following six ICWM-specific concerns by a consensus meeting: (1) whether the CG included evidence from ancient TCM classics; (2) whether ancient Chinese in the TCM recommendations had been converted to the vernacular for the comprehension of non-TCM practitioners; (3) whether the CG provided the principles of the addition and subtraction of TCM interventions based on syndrome differentiation; (4) whether the CG specified the interactions between TCM and Western medicine; (5) whether the CG ranked the efficacy and safety of different ICWM interventions; and (6) whether the CG provided monitoring criteria for both diseases (Western medicine concept) and syndromes (TCM concept). All these items received an answer of “yes” or “no.” The assessment was also performed by four appraisers, and they achieved consistency through discussion.
2.7. Statistical Analysis
We calculated the mean, standard deviation, median, interquartile range (IQR), range, or proportion to describe the characteristics of the CGs, standardized AGREE II scores, and additional items. We performed stratified comparisons of CGs with different characteristics using the Mann–Whitney U test or the Kruskal–Wallis H test, and the difference was significant if the value was less than 0.05. The intraclass correlation coefficient (ICC) and 95% confidence interval (CI) were calculated to assess consistency among the appraisers using a two-way mixed model; an ICC <0.50, 0.50–0.74, 0.75–0.89, and 0.90–1.00 indicated poor, fair, good, and excellent consistency, respectively [24].
3. Results
3.1. Epidemiological Characteristics
As shown in Figure 1, the search yielded 5,571 results, and 62 ICWM CGs [25–86] were included in the quality appraisal after screening. The number of ICWM CGs was low between 2003 and 2015 but increased rapidly since 2016 (Figure 2). Most CGs were published as journal articles, except for one published as a conference paper. Two CGs were published in English, and the others were published in Chinese. There were 15 evidence-based and 47 consensus-based CGs. Most (87.1%) of the developers were academic associations, two (3.2%) were government departments, and six (9.7%) were nonofficial organizations. Recommendations for diagnosis and treatment were included in 47 CGs (75.8%), for only treatment in 13 (21.0%), and for only diagnosis in two (3.2%). Seventeen CGs (27.4%) focused on digestive diseases, 12 (19.4%) on cardiovascular diseases, 8 (12.9%) on skin diseases, 8 (12.9%) on urogenital diseases, 7 (11.3%) on respiratory diseases, and 10 (16.1%) on other diseases (e.g., bone, kidney, and nerve). The average number of pages and references in the included CGs was 6.2 and 32.3, respectively. Only five CGs were longer than 10 pages, and 13 had no references. The detailed characteristics of the included ICWM CGs are presented in Table S2 in the supplementary file.