Study publication date should not dictate suitability of methodological and reporting quality assessment

1Corresponding author. Universidade Federal de São Paulo (São Paulo), Centro Universitário São Camilo (São Paulo). São Paulo, Brazil. rleitepacheco@hotmail.com 2Faculdade de Medicina de Petrópolis (Petrópolis). Rio de Janeiro, Brasil. luis.fontes@ceo2.com.br 3University of Oxford (Oxford). United Kingdom. enderson.miranda@kellogg.ox.ac.uk 4Centro Universitários São Camilo (Sâo Paulo), Universidade Metropolitana de Santos (Santos). São Paulo, Brazil. analuizacabrera@hotmail.com 5Núcleo de Avaliação de Tecnologias em Saúde, Hospital Sírio-Libanês (São Paulo), Universidade Federal de São Paulo (São Paulo). São Paulo, Brazil. rachelriera@hotmail.com


Main text
"Research-on-research" studies, also known as methodological studies, have the main purpose of evaluating certain methodological aspects of published research, which includes methodological and reporting quality characteristics from previous studies 1-3 . Research-on-research studies have an important implication on helping the conception, design, and conduction of future studies 4,5 . Therefore, assessing previous research is essential in avoiding unnecessary duplication and waste 6,7 .
The critical appraisal process of healthcare literature is typically based on three core criteria: methodological quality assessment, completeness, and transparency of reporting.
Tools and guidelines have been developed over the years in the attempt to standardize the methodological quality assessment. The AMSTAR-2 (A Measurement Tool to Assess Systematic Reviews) 8 and the QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies) 9 are two of the most used instruments for assessing the methodological quality of systematic reviews and diagnostic accuracy studies worldwide.
Quality assessment related to the transparency of reporting and completeness of publications is usually performed by checking its adherence to a pre-determined checklist designed to systematically guide reporting 10,11 . The EQUATORnetwork database (http://www.equator-network. org/) has published 422 reporting guidelines to date with 83 additional reporting guidelines under development. This reflects the increased popularity of such guidelines, whose dissemination and adherence are also encouraged by the International Committee of Medical Journal Editors 12 .
Evaluators often question the validity of utilizing tools during the critical appraisal process if such tools were created after the study was conducted and/or published. The common argument is that it would be unfair to assess the adherence to a recommendation that did not exist at that time. Based on this argument, some authors have restricted the evaluation to only include studies published after the tool has become available [13][14][15] . Furthermore, certain critical appraisal studies have published in their discussion sessions that the utilization of a quality assessment tool in studies published before the tool's development should be considered as a limitation of the studies [16][17][18] .
We do not agree with this argument as we believe that the basic principles of well-conducted and properly reported studies should be known and fulfilled before the development of any appraisal tool. Furthermore, transparency and reproducibility have always been a key best practice within the scientific process. Thus, the recommendation that studies should be conducted with a minimum methodological rigor should be followed regardless of the existence of a valid tool to systematically evaluate their quality. It is reasonable to assume that risk of bias and uncertainty in the results derived from a poor quality study would occur regardless of the existence of a specific tool when the study was conducted and/or published. The results from studies that were conducted prior to the existence of a valid methodological evaluation tool should not be shielded from biases associated to its low internal validity.
The proper evaluation of healthcare literature published prior to the existence of a relevant quality assessment tool creates an important opportunity to benchmark study quality over time by comparing recent reports with previous ones. This is critical to confirm if the quality assessment tool was actually able to promote an enhancement in the methodological and/or reporting process 10,19,20 .
In conclusion, all published studies must be appraised for its methodological and reporting quality, regardless of their publication date. Removing appraisal restrictions based on date thresholds would allow us to learn from our past mistakes, which can be a great way to avoid waste in research and significantly improve the overall quality of future studies.