A warning to readers about the term metanalysis in non-systematic reviews about diabetic retinopathy: documental study

| BACKGROUND: Meta-analysis is a very seductive term in scientific papers because it summarizes a compilation of quantitative results; however, its interpretation must be made carefully. OBJECTIVE: Search the diabetic retinopathy intervention literature for the occurrence of the term meta-analysis in non-systematic reviews, showing the reader the frequency that they occur and alert to the dangers that inadequate interpretation can cause. METHODS AND MATERIALS: An extensive search in Pubmed was performed for works with the term meta-analysis in the title, abstract, and keywords (ti, abs, kw) without date or language restriction. The selected papers were read fully in search of characteristics of systematic review (SR) or not. The results are presented objectively with a critical analysis of each analyzed term. RESULTS: We found 39 papers with the term metaanalysis on (ti, abs, kw). 12 (30.8%) of them did not write the study design in the abstract. 14 (35.9%) did not present the search strategy, 15(38.5%) did not mention having a language or date restriction. 23 (59%) did not mention the SR model followed, and only five (12.8%) registered the protocol. Only two articles fulfilled the RS requirements. CONCLUSION: We found in the literature reviews of selected studies with meta-analysis, not configuring an SR that can induce the reader to interpret it as an SR. The term meta-analysis cannot be interpreted as the panacea for the solution of all doubts.

kw) without date or language restriction. The selected papers were read fully in search of characteristics of systematic review (SR) or not.
The results are presented objectively with a critical analysis of each analyzed term. RESULTS: We found 39 papers with the term metaanalysis on (ti, abs, kw). 12 (30.8%) of them did not write the study design in the abstract. 14 (35.9%) did not present the search strategy, 15(38.5%) did not mention having a language or date restriction. 23 (59%) did not mention the SR model followed, and only five (12.8%) registered the protocol. Only two articles fulfilled the RS requirements.

Introduction
The incidence of diabetic retinopathy (DR) in the United Kingdom dropped from 45.3% between 1991 and 1999 1 to 23.9% between 2000 and 2007 2 . That was mainly due to advances in the treatment and diagnosis of diabetes. However, the incidence of DR remains high, requiring high demand for more effective and cheaper therapies. The answer to the search for new therapies is found in clinical trials. The annual number of registered clinical trials increased from 3294 in 2004 to 23 384 in 2013 3 because it is the appropriate study design to answer real-world questions, and only 2.15% of all publications in ophthalmology are randomized clinical trials (RCT) 4 .
RCTs are long and expensive studies that sometimes limit the samples' size in the face of technical difficulties. One of the purposes of systematic reviews is to enhance the sample size by compiling quantitative data.
The Systematic Review (SR) of RCT with meta-analysis occupies the decision-making pyramid's peak at evidencebased medicine 5 . It is the most robust evidence synthesis method for intervention clinical trials and brings a more reliable response to real-world questions.
Meta-analysis is a mathematical analysis that compiles several qualitative results 6 . It is a seductive term in scientific documents but must be read carefully. As a mathematical analysis, it is dependent on the data feed given to it. One of the most significant criticisms of meta-analysis is that it focuses on the summary and ignores that the effect of treatment may vary from study to study 6 . Besides this, the meta-analysis of some selected studies according to the author's interest will show results according to the presented sample and may not reflect the real world.
PRISMA 7 is a tool used to guide publications in SR. It helps authors to report the text and meta-analyses and improves the quality of the publication. Application of PRISMA 7 results in a detailed presentation of the quality of selected clinical trials and, if the search strategy is adequately executed and the papers rigorously selected, this sample will be representative of everything presented in the literature closer to the real world and providing greater security in decision making.
Thus, this study aims to alert the reader to the potential risks of reading a meta-analysis without an SR by identifying publications with meta-analysis in the title, abstract, or keyword (ti, abs, kw).

Methods
This study was developed at the Department of Ophthalmology from Escola Paulista de Medicina-UNIFESP. A literature review was carried out on DR interventions papers without date or language restriction in PubMed published until 2020, 19 June. All reviews and SR with the term meta-analysis on the title, abstract, or keyword (ti, abs, kw) were considered. The search strategy is presented in Table 1. The articles' selection process was carried out by two authors independently using Rayyan Platform 8 , selecting all papers in DR interventions that presented the term meta-analysis (ti, abs, kw). We excluded network meta-analysis, empty SR, strictly summary presentation, epidemiologic papers, and protocol studies. Disagreements were solved by consensus.
Two independent authors read the selected studies in full to capture the following data. The selected articles were confirmed with the term meta-analysis (ti, abs, kw) and read in full to assess the presence of the term systematic review in (ti-abs-kw); whether they presented a search strategy and whether it was adequate; how many databases the search strategy has run; whether studies were restricted or studies were selected; if GRADE 9 was presented; which SR model is oriented; whether the PRISMA flow 7 was presented; whether there was any suggestion for the use of the intervention or not, whether the bias was classified and whether the protocol record was presented.
The data were taken from selected studies and will be presented narratively and quantitatively below.

Results
The search strategy found 386 titles and 39 papers were selected according to the inclusion criteria, 38 with the term meta-analysis in the title and one in the keyword. Twenty-two presented the term systematic review in the title; two presented the term review in title, 12 studies said nothing about the study design, and 12 six studies said that was from randomized clinical trials only. Eighteen papers identified the study as RS in the abstract. 13 papers with the term SR in the title confirmed it in the description of the study design in the abstract. Two papers presented the term meta-analysis and SR in the keyword, nine papers presented the term meta-analysis strictly in the keyword, and no one writes SR only in the keyword. The remaining 28 did not use any term of the study design in the keywords.
Only two papers presented the search strategy properly, 22 presented the search strategy partially, 14 did not present the search strategy, and one paper described a meta-analysis of selected studies according to the author's interest. One paper reports having searched in only one database, one does not report, one claims that the study was made up of selected studies, four studies used two databases, 15 used three databases, seven papers used four databases, four papers used five, three used six, one reports having used seven, and one study used 12 databases.
Ten papers used RCT or not, one does not say, and the other 28 studies used RCT. Four studies report having restricted the date of study selection, five studies restricted language, one study restricted date and language, 15 did not speak if there was or was not restriction, 13 made it clear that there was no form of restriction and one study and one study said nothing because it is a meta-analysis of clinical trials selected by the author.
Thirty-five did not present the GRADE 9 and four did it. 17 presented risk assessment of bias in the selected clinical trials, the other 22 studies did not present. Seven studies refer to having followed PRISMA 7 , three refer to having followed the Cochrane handbook model, three refer to having followed a PRISMA 7 and Cochrane 10 model, and 23 do not mention which RS publication model they followed. 31 presented PRISMA flow 7 and the others eight did not. Four studies registered the protocol in PROSPERO 11 , one published the protocol as an article, and the others did not present any form of protocol register.
Twenty-six studies presented strongly positive conclusions regarding the use or not of the intervention, 13 ended up suggesting little evidence or the need for further studies for future conclusions.

Discussion
The need for formal language in scientific papers is a determining factor for a high standard publication to guarantee the basic principle of all science which is the reproducibility. SR is a well-designed study developed to determine decision making and in order to fulfil its purpose it must be executed according to the tool that guides the specifications to prevent the use of the results inappropriately.
The systematic review is the compilation of similar studies in order to enhance the sample size. It begins with the clinical question design, PICO (Population, Intervention, Comparison and Outcome) and public protocol registration. It outlines the search strategy as broad as possible without restriction of date or language to select the largest number of studies, thus making the sample of the population the most reproducible for the most diverse peoples. Data relevant to the search are captured and gathered in a meta-analytical study if possible. Some SR does not find studies for compilation because it does not present studies to answer it. In these cases, they are called empty SR, which, although empty, denounces the lack of an answer to a real-world question.
PRISMA 7 is the tool that guides the design of the SR and the Cochrane handbook 10 . According to PRISMA 7 , the title must come with a clear provision for systematic review with meta-analysis or not. In this regard, our sample found 22 (56.4%) papers entitled as SR. PRISMA 7 preaches the need for publication of the protocol. PROSPERO 11 is a base for the registration of SR protocols at the University of York, it has no fees, so it does not justify the lack of protocol publication. The protocol is the security that there were no selective studies after capturing the data according to the author's interest 12 . It confers credibility and reproducibility of the results. In our sample we found 87.1% without protocol published or registered.
Still in the methodology, it is planned to present the complete electronic search strategy in at least one database ensuring reproducibility 13 . Only two (5.1%) studies presented a search strategy complete in our sample, and all of two presented the protocol register and adequate title.
The classification of the risk of bias in clinical trials is also provided for in the methodology. This study found 17 (43.6%) papers that presented the assessment of the risk of bias, and among them are the two that present the registration of the protocol and the search strategy. The risk of bias assessment, also called quality assessment or critical assessment, helps establish the transparency of the results and findings of the synthesis of evidence.
Thus, in our sample, we found two (5.1%) works entitled SR presenting meta-analysis that complies with the requirements set out in PRISMA 7 , and only one (2.6%) of them presented the GRADE 9 evaluation. The placement of the term "CERTAINTY OF" is foreseen in the PRISMA 2020 EVIDENCE 14 , suggesting that it could be the GRADE 9 table; it is a new requirement that was contemplated in four (10.2%) studies of our work. Our sample and analysis were before the publication of PRISMA 2020 14 ; that is, there was no forecast for the quality of evidence by GRADE 9 .
As stated in the introduction, meta-analysis is a mathematical tool that quantitatively compiles a sample of various data, but it cannot be seen as the answer to all doubts. It will always give an adequate response to the data with which it was fed. In this work sample, we found a meta-analytical study of a sample of clinical trials selected by the author. When reading meta-analysis in the title, an inattentive reader may consider this study as the reproduction of a large sample; however, as it is a compilation of selected studies, it will bring an adequate response to this special sample project a bias in the selection of studies. This work's objective was to alert the reader to the reading of the meta-analysis 12 . This care should also be applied to epidemiological meta-analyses and other areas of interest. Remember that the metaanalysis, as it is a mathematical calculation, will always have an answer; it depends on the power source of the data and, as such, the more accurate the data, the more correct the answer will be. For the critics of the meta-analysis, a number cannot summarize a field of research. The meta-analysis allows for the mixing of inconsistent data; essential studies can be ignored if the data does not come from an SR; the meta-analysis may disagree with randomized studies; the meta-analysis can be poorly performed, leading to the question of whether a narrative description could not be better 15 .
Every study is important when designed and executed properly and intended for the purpose for which it is proposed and the best study is the one that is well designed and well executed fulfils the intended role. Remember that SR results will serve as a basis for the decision-making of the healthcare provider, the health administrator and guidelines 16 , it expands and diversifies the sample and the meta-analysis cannot be considered a panacea to compensate for the methodological bias of the research, such as the lack of a systematic review.
This work aimed to warn the reader not to be seduced by the term meta-analysis in the title of a work and to believe that it will be running a broad data compilation, it is necessary to evaluate the qualities of data collection fed that sample. This work did not assess the impact factor of the journals that published the meta-analyses, as our objective was not to assess the quality of the publications, but rather to assess what readers have had access regardless of language and date of publication.

Conclusion
Meta-analyses are resulting from a series of studies selected according to the author's interest and which convey results specific to that sample, which may be different from results from SR meta-analysis.