Covid-19 research evidence : An international survey exploring views on useful sources , preferred formats , and accessibility

1Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK. eve.tomlinson@bristol.ac.uk 2The Evidence Centre. debra@evidencecentre.com 3Universidad de Valparaíso. Valparaíso, Chile. jana.stojanova@uv.cl 4Oxford University Hospitals NHS Foundation Trust, Oxford, UK. roses.parker@ouh.nhs.uk 5Knowledge Translation Department, Cochrane Central Executive. Ontario, Canada. mumoquit@cochrane.org 6Knowledge Translation Department, Cochrane Central Executive. Berlin, Germany. slagosky@gmail.com 7University of the Western Cape, Cape Town, South Africa. bschmidt@uwc.ac.za 8Corresponding author. Quality Division, Surrey and Borders Partnership Trust, National Health Service, UK. karenshead@hotmail.co.uk


Introduction
In a pandemic, policy makers, clinicians, patients, and the public need to make decisions quickly. If decisions are to be informed by research, decisionmakers need access to high-quality, timely, and relevant research evidence in a format that is easy to understand and apply.
The research community mobilised quickly to develop and share information about COVID-19. In 2020, over 86,000 articles were indexed in PubMed with 'COVID-19' in the title (MeSH unique ID: D000086382). Additionally, the spread of COVID-19 evidence took place across other sources including mass media and social media. 1 Due to wide variation in the quality and focus of the research available, misinformation and misconceptions about the pandemic were reported amongst the public 2,3 This is an issue as exposure to misinformation regarding COVID-19 promotes distrust in public health experts 4 and increases belief in 'conspiracy theory', found to negatively associate with the use of health protective behaviours. 5 By reducing adherence to evidence-based recommendations such as social distancing 6 , and in some cases increasing engagement in harmful activities, misinformation can have deleterious consequences.
Cochrane is an international collaborative that produces synthesised research evidence to inform decisions about health and healthcare. 7 We wanted to know where and how people were accessing research about COVID-19 so that we and others could optimise how we provide information. At the time we did this work, there was limited published international research about how people find COVID-19 evidence. Some studies suggested government bodies 2 and internet sources including news websites 8,9 may be popular sources to access for COVID-19 information.
We conducted a cross-sectional survey of people who accessed Cochrane websites and social media in September 2020. Our aims were to: 1. test whether it was feasible to get feedback promptly from a range of people of diverse roles, interested in research evidence during a pandemic, 2. understand ways people want to interact with research evidence to inform their decisions about COVID-19. This is relevant for organisations and individuals who produce evidence and want to ensure their evidence is available to inform policy and practice.

Survey Development and pre-testing
We collected feedback using an online crosssectional survey in September 2020, available in English and Spanish. We developed survey questions in partnership with people who prepared and used research evidence, including healthcare professionals, patients, and policy influencers. We piloted the survey with over 100 people, up to five times for various sections. Testers were selected by convenience from our target audiences and countries, with the same roles as our target population. We conducted four focus groups and additional interviews to check the validity, usability, and clarity of questions.
The survey was split into three sections: participant characteristics, including people's role and region; experience of finding and using COVID-19 evidence in general; and feedback about Cochrane's COVID-19 evidence when people had used it (this element is not covered in this paper). There were 14 questions, with a mix of multiple choice, Likert scale, and open-ended questions. A copy of the final version of the survey is available in the data supplement.

Sampling
We did not aim to be representative of everyone interested in research evidence. We wanted feedback from anyone interested in research about COVID-19. We used convenience sampling to invite anyone who visited Cochrane websites or social media over a 2 to 4-week period to take part (various links were open for two, three or four weeks).
Cochrane websites, newsletters, and social media were used as proxy for reaching people interested in research evidence. Around 5 million people visit Cochrane.org a month and Cochrane has over 150,000 social media followers globally.
We promoted the survey using: • a pop-up survey that appeared when people visited one of our websites • a pop-up link to an online survey that appeared when people visited two of our websites (one in English and one in Spanish) • invitations on our social media accounts • emails sent directly to stakeholders The online survey platform limited responses to one per IP address and website pop-ups appeared only once per IP address during the survey period.
No payments were offered. Participation was voluntary and anonymous. The process was reviewed by Cochrane's Central Executive Team for ethical implications and followed the Declaration of Helsinki.

Analytical approach
We collated data into a spreadsheet and translated Spanish responses into English for analysis. No identifying information was collected. All raw data was kept password protected and shared only with authorized analysts.
We used the Statistical Package for the Social Sciences (IBM SPSS Statistics 20) to analyse data. We used frequencies and percentages to summarise categorical variables, and analysed crosstabulations using Chi-Squared tests. P-values ≤0.05 were considered statistically significant. We calculated percentages out of the people who answered questions, reporting the number of responses and omitting missing responses.
We analysed open-ended questions by coding responses using the constant comparative method. All analyses were undertaken and checked by two researchers independently. Responses were voluntary and missing responses were accepted; we report total number of responses received and present responses as a proportion of responses received.
Most people provided feedback via surveys advertised on our websites rather than through social media or email (Table 1). About one quarter of participants provided feedback using the Spanish language survey.

Access to research about COVID-19
Ninety percent of respondents said research evidence had influenced their decisions related to COVID-19 ( Figure  1) (808 respondents, 97.2%). This was the same regardless of their primary role (P>0.05). About half thought it was easy to keep up to date with the latest COVID-19 research (Figure 1). Healthcare professionals and researchers were more likely than others to say this (P<0.05). Of the 478 respondents who indicated their preferred language was not English, 65% agreed research evidence about COVID-19 has been available in that language, although people from Central/South America were more likely to disagree with this statement (P<0.05). Note: The question asked 'Thinking about the COVID-19 pandemic so far, to what extent do you agree with the following?' The four statements above were listed with a four-point response scale ranging from Strongly Disagree to Strongly Agree. We asked people to leave the statement about preferred language blank if their preferred language was English.

Preferred formats for research evidence
We asked about preferred formats for evidence about COVID-19 (Table 2). People could choose as many options as they wanted from a list and add options. Preferred formats were short summaries in plain language (60%), systematic reviews available online (60%), short summaries with commentaries about implications (51%), visual summaries e.g., diagrams and infographics (48%), tables listing benefits and harms of different options (48%), and short journal articles (48%). Fewer than one in five participants prioritised podcasts (17%) or blogs (9%). Healthcare professionals and researchers were more likely than others to prefer online systematic reviews whereas policy makers, members of the public, and other audiences prioritised short summaries in layperson language, visual summaries, and videos (all P <0.05).
We asked people the best way to make them aware of COVID-19 research evidence; 522 responded (63%). Around one third preferred email updates or newsletters, tailored to their areas of interest (34%). Around one in ten preferred traditional media, social media, and a regularly updated website with searchable repositories (Table 3).

Discussion
Our findings have implications for researchers, evidence producers, and others who want to help people access and use research evidence about COVID-19 in a timely manner.
It may be challenging to keep up to date with evidence about COVID-19 due to the large number of publications and pre-print articles about COVID-19 and rapid developments in prevention and treatments. 10 There is limited research exploring perceptions about keeping updated with COVID-19 evidence, but some suggest professionals have felt stressed about the amount of information. 11 In our survey, only half of respondents agreed that it was easy to keep up to date with research evidence.
People in our sample said they gained useful research evidence from websites and newsletters of government departments, international organisations, and professional societies. Individuals wanting to disseminate COVID-19 research evidence could consider partnering with organisations to showcase research rather than relying primarily on the circulation of journal articles. Furthermore, despite not being rated as the most useful sources, media (e.g., newspapers, TV) and social media (e.g., Facebook) were still used by 37% and 27% of the sample, respectively. The media was a common source of information accessed by members of the public. These sources are accessible to many and when properly used, can provide useful platforms to disseminate health information widely. Promotion of high-quality COVID-19 research evidence through skilful use of these channels, for example by scientists and governmental organisations, may therefore be an effective dissemination strategy, and may help to increase trust in this form of media by counteracting misinformation. 1 The people we surveyed wanted to be made aware of COVID-19 research evidence through email updates, traditional media, social media, and an up-to-date searchable online repository. This suggests a desire for continuous timely updates of reliable COVID-19 evidence, by having it provided directly rather than needing to actively seek it. It also suggests a desire for information to be organised and curated (email updates, searchable online repository) or in a synthesised, easy to understand format (traditional media, social media).
There is unlikely to be a single best way to format and present research evidence across all audiences. Listening to their needs and involving end-users in developing and testing formats are likely to be important in identifying optimal approaches.

Comparisons with prior work
Others have similarly found high levels of trust in COVID-19 information from Government websites and scientists [12][13][14][15][16][17][18] and lower levels of trust in social media. 14 Recent systematic reviews found that the type of healthcare decision-maker, context, area of interest, individual skills, and competencies affect preferred means for people to access information. 19,20 These reviews focus on health research evidence in general but suggest more work may be needed to understand whether people from different regions and roles may differ in preferences for accessing COVID-19 evidence.

Strengths and limitations
A strength of our approach is that we sought feedback from a range of people rapidly, and this feedback informed how Cochrane produced and disseminated COVID-19 evidence. It was feasible to collect useful information about access to evidence during a pandemic using a simple online survey, despite not reimbursing participants. We did not aim to be representative, and we did not extensively promote the survey through our networks or partners. Even so, in just 4 weeks we gained international feedback, from professionals, patients, and policy influencers.
There are limitations to this cross-sectional survey using a convenience sample. Promoting the survey solely through Cochrane channels may have resulted in sampling participants who are knowledgeable about evidence-based healthcare and supportive of incorporating research evidence into health decisionmaking, limiting generalizability. Even within this population, one quarter (27%) said they had not used any Cochrane resources before the pandemic and a further 42% were aware of Cochrane but were not regular users.
We only offered the survey in English and Spanish as the Cochrane Library is available in these languages in its entirety (other languages are partially covered). More than half of participants stated that English was not their preferred language. Language restrictions may have prevented others from responding and most responses were from Europe and the Americas, limiting generalisability.
We used the term 'research evidence' to mean information from research studies, but participants could have interpreted this to mean any information about COVID-19, not solely informed by research.

Conclusions
Our survey provides a starting point to help researchers and evidence producers consider the most effective way for disseminating research evidence. Journal articles and systematic reviews remain important formats. However, to reach a wide range of people, tailored emails, promotion via traditional and social media, and links on the websites of government departments and professional societies, may also have a role. Visual abstracts, infographics, and short summaries in plain language may likewise be helpful.

Role of the funding source
This research was funded by Cochrane as part of their evaluation into their initial response to the COVID-19 pandemic. This research did not receive any specific grant from funding agencies in the public or commercial, or not-for-profit sector.

Authors' contribution
Tomlinson, E drafted manuscript, searched for literature, reviewed

Conflicts of interest
No financial, legal or political conflicts involving third parties (government, corporations and private foundations, etc.) have been declared for any aspect of the submitted work (including, but not limited to grants and funding, advisory board participation, study design, preparation of manuscript, statistical analysis, etc.).