The five studies identified through the additional searches had an important impact on the review findings; indeed, one study had a central role in the development of one of the review’s descriptive themes. The other four studies were less influential, but added detail and strength to the review’s findings. If this group of additional studies had not been found, the syntheses would have been a less complete story of children’s views of body size. These five studies were also of a disproportionally high quality compared with the studies in the review as a whole and so added to the robustness of the reviews findings. This runs counter to previous findings on sourcing health effectiveness studies where additional searches designed to increase a review’s comprehensiveness have been limited in use because of the low quality of the research found [25
We have shown that a small number of studies located from searching databases in addition to those specified in the original review protocol had a significant impact on the outcome of the final review. Although this case study is based upon 28 studies included in one review, the studies were from different subject disciplines, dispersed across a range of databases, and over a fifth were located from sources other than databases or search engines. In this case, a small number of studies were found in several places across large international bibliographic databases, and for other studies, the research records were scattered across a range of search sources over the disciplines of medicine, social science and education. At the outset of the systematic review it was anticipated that the literature would be widely dispersed. The original search protocol aimed to draw on a large sample of studies from a range of sources. It contained sources judged to potentially contain good sources of UK studies, notably: Applied Social Sciences Index and Abstracts (ASSIA), Bibliomap (EPPI-Centre's register of health promotion research), and HealthPromis (a UK Health Development Agency database that is no longer updated). ASSIA is an international social science database comprising of largely of journals published in the UK (46%) and in the USA (43%)[26
]. The later searches of seven additional sources provided a greater UK focus to the search strategy. These contained UK studies not found by other means, rather than containing exclusively UK studies. For example, the International Bibliography of Social Sciences (IBSS) covers a large range of countries, and a majority (18%) of its source titles are published in the UK [27
]. We cannot provide an accurate assessment of the time taken to undertake the additional searches. It involved a visit to another library to access one database; for some databases multiple simplified searches were undertaken owing to limited search functionality; and some databases were scanned at source, owing to restrictions in downloading the research records.
Although this case study is based on the results of searches used in the review, rather than what is actually present within the search source, we consider this justified as it is representative of the process of searching for a systematic review. The search strategies used for each database source were conceptually the same with variations depending on the functionality and volume of records. The database searches were intended to be sensitive, using a range of free text and controlled vocabulary terms, and this is demonstrated in Table
by the low precision values. Publication type has only been given a cursory consideration here, and when searching for specific types of research, the nature of the publication type may also have an impact in deciding where to search.
Searching for ‘views’ studies can involve considerable investment of resources. The low precision found from database sources is consistent with published findings, such as Shaw et al. [2
] who report a precision of under four percent in database searching for qualitative studies. For such a small number of studies, it would not be sensible to use this case study for selecting one search source over another, but it is useful in increasing awareness of the types and range of sources that might be useful in limiting database selection bias. Search engines such as Google and Scirus are difficult to search comprehensively, but they can be used for supplementary searching to locate potentially useful studies. Response from contacting authors can be a good source, although this method has unpredictable outcomes. Forward citation chasing was not effective in yielding any new studies in this case; however, it was limited to Web of Science Cited Reference Search and in retrospect, additionally utilising a citation search database that covers non-journal reports and dissertations may have improved this. Our finding that a high proportion of studies (21% (n
6)) were obtained only from non-database sources (as shown in Table
) is similar to other studies of searching for qualitative research [12
Identifying search sources for systematic reviews can be challenging, but could be improved with more knowledge-sharing between information specialists and researchers. There will always be limitations on how wide one is able to search, the number of strategies that can be employed to improve comprehensiveness and reduce database selection bias, and relevant studies could be missed. Making more informed choices of where to search can mitigate the effects of this. The EPPI-Centre is building on this case study and comparing search sources across a number of systematic views of people’s views [5
Locating studies from a particular region is not just relevant to ‘views’ studies. A greater awareness of the value of searching a wider range of sources and the geographical slant of sources has contributed to the search protocol for other public health reviews within the EPPI-Centre. In undertaking other reviews we have identified other small databases and websites that could be of potential use. It can be challenging for the user to discover what is covered by a database or how publication sources are selected for inclusion, particularly where journals are partially indexed. In this case study, two included studies were from the Health Education Journal, and both were not picked up by database searches. Further investigation revealed partial indexing of this journal in electronic database sources and has provided a case for hand-searching of this journal in later reviews.
The impact of new technologies on the choice of database is another consideration. Some databases have had substantial investment in their IT infrastructure. PubMed, in particular has spawned a small ecosystem of supporting services which all build on and publish the PubMed dataset in different forms. GoPubMed offers semantic and other analyses of PubMed documents in an interface that few social science databases can match. HubMed offers an alternative interface and a highly useful ‘citation finder’ which enables users to copy and paste bibliographies into a text box that processes this into a list of citations with links to their PubMed records. PubMed itself offers an open application programming interface (API) that enables programmers to integrate searching PubMed into their applications. These new services are innovative and demonstrate the potential of modern IT technologies. However, they are becoming a source of potential bias, as the very availability and accessibility of these services promotes the PubMed dataset above smaller, regional databases.