• rheumatoid arthritis
• cardiovascular disease
• osteoarthritis

With great interest, we read the paper recently published in the Annals of the Rheumatic Diseases titled ‘Metabolic and cardiovascular benefits of hydroxychloroquine in patients with rheumatoid arthritis(RA): a systematic review and meta-analysis’.1 In the study, the authors revealed that hydroxychloroquine may benefit the metabolic profile and to a lesser extent cardiovascular events in patients with RA. We congratulate and applaud their significant work for clinical practice, but there are some worthwhile issues that need to be further explored.

First, the current meta-analysis was conducted according to the guidelines of the Preferred Reporting Items for Systematic Review and Meta-Analysis, but the protocol, which should have been predefined, was not registered in any platforms, such as the Cochrane Library, PROSPERO (https://www.crd.york.ac.uk/prospero/) and BMJ Open, which may affect its transparency and reproducibility.2 Medline, Embase, the Cochrane Library, and the databases for the American College of Rheumatology and the European League Against Rheumatism annual meetings were searched in this study, but these databases seemed to be not enough to retrieve all the eligible studies. Additionally, NLM Gateway, BIOSIS previews and some grey literatures should be included, which may contribute to a more comprehensive collection of eligible studies.3

Second, Jadad score was employed to assess the quality of eligible randomised controlled trials (RCTs) in the current meta-analysis. However, owing to the lack of allocation concealment assessment, Jadad score may overestimate the quality of eligible RCTs, thus overstating authentic intervention effect of hydroxychloroquine. Alternatively, the Cochrane risk-of-bias tool including random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective reporting and other bias may be a better choice to comprehensively evaluate the quality of eligible RCTs.4

Third, RCTs and observational studies were inappropriately pooled together in the current meta-analysis. Part of reasons may be that limited studies were involved in the outcomes of interest, such as low-density lipoprotein, high-density lipoprotein and triglycerides. However, it is general that RCTs and observational studies cannot be pooled when considering different evidence levels in the hierarchy of evidence pyramid according to the Cochrane Handbook. We wonder whether further subgroup analyses based on study design should be performed to clarify the potential advantages of hydroxychloroquine, which may increase the robustness and credibility of the conclusions. Furthermore, authors stated in the Methods section that all the meta-analyses were performed using a random-effects model in case of significant heterogeneity. Nevertheless, we found that fixed-effects model was chosen to estimate all the pooled effects in the following analyses in the Results section. As with us, some readers may be confused about the inconsistency.

Fourth, the meta-analysis in total included only five studies for publication bias assessment. Funnel plots and Egger’s tests indicated no significant statistical difference, and then the authors stated that publication bias did not exist. However, this conclusion may be inaccurate. Generally, owing to the poor statistic weight from ‘small-study effects’ and bad performance of funnel plots’ visualisation in detecting publication bias, publication bias can be identified though the asymmetry of funnel plot and subsequent tests only when more than 10 studies were included in the meta-analysis.5 Thus, considering limited studies included in the meta-analysis, the possibility of publication bias was hardly excluded.

To sum up, we respect the great work done by the authors, but the study should be interpreted with the aforementioned limitations, and further high-quality multicentre RCTs with larger sample size should be warranted to clarify the potential benefits of hydroxychloroquine in patients with RA.