Building on the same concepts our group has used to create novel high performance search filters for general nephrology and renal transplantation [15
], we have succeeded in developing and validating search filters for glomerular disease that are highly sensitive and specific. All filters achieved a balance of at least 93.8% sensitivity and specificity. Our best performing high-sensitivity filter was in Embase, achieving 99.0% sensitivity and 95.3% specificity. The best performing high-specificity filter was also in Embase, which reached 95.7% sensitivity and 98.6% specificity. Without changing their PubMed search terms, in an illustrative example physicians were able to retrieve articles with a higher degree of precision (less non-relevant articles) with use of these filters.
These filters are complex, often combining in excess of 50 terms with Boolean operators. Coding these filters into the PubMed and Ovid search engine interfaces will permit their easy use by anyone doing a search. In the meantime, we provide these filters at the following link: http://hiru.mcmaster.ca/hiru/hiru_hedges_nephrology_filters.aspx
. As of September 2011, use of the high-sensitivity glomerular disease filter reduced the PubMed database from 21 million to 195,374 articles, and the high specificity filter reduced this to 107,658 articles.
Depending on the search terms entered by the user, these filters may serve many purposes, which are best understood in the context of our illustrative proof of concept searches (Table ). First, without changing the original search term(s), selecting a filter applies the search only to a subset of articles that are richer in glomerular disease content. The result is an increase in precision of the search, similar to the increase in positive predictive value of a screening test when applied to a high-risk population. This was demonstrated by the use of search terms ‘minimal change treatment’ in Table . Fewer non-relevant articles were retrieved with use of the filter (1236 versus 4662 articles), without impacting relevant article retrieval. Second, the filter acts as an optimized substitute for glomerular disease specific terms and synonyms allowing users to simplify the search query. This avoids unnecessarily limiting the search due to indexing inconsistencies inherent with the terminology used to define glomerular disease. For example, if a user was searching for dietary recommendations in diabetic nephropathy, the search terms may be simplified to ‘low protein diet diabetes’, instead of searching for ‘low protein diet diabetes’ with selected terms such as ‘nephropathy’, ‘kidney disease’, or ‘glomerulosclerosis’ that may negatively impact relevant article retrieval. In this case, even without use of search terms pertaining to glomerular disease, precision of the results was enhanced (Table ). Third, users may opt to exclude disease specific terms entirely and use the filter to address questions that potentially relate to all glomerular disease equally. An example of this may include entering ‘immunization’ when addressing the impact of vaccinations in patients with glomerular disease.
Our results also highlight that even with high performance validated search filters, a single search will rarely retrieve everything of relevance on a particular topic. There is simply too much variation in the quality of accompanying search terms entered by the user, completeness of the database, and quality and consistency of indexing. This explains why in some proof of concept searches, retrieval of relevant articles was incomplete both with and without use of the filter (Table ). Also, the extent to which the search filter is generalizable depends upon the sample of journals selected for study and the method by which articles were defined as relevant. Our selection of journals was deliberately enriched with leading clinical nephrology journals. Although it also included a random sampling of other journals, this set of journals may not adequately represent the complete set of multi-disciplinary journals that feature glomerular disease content in PubMed. This may explain the significant drop in precision when the filter was applied to the validation set, which was a smaller database by design with a lower proportion of relevant articles. Our choice to divide articles into the development and validation sets at the journal level may have also contributed to the lower proportion of articles with glomerular disease content in the validation set. However, this approach provided insight as to what would occur if the search database were expanded to include the over 5000 journals indexed in PubMed.
Proof of concept searches were used to illustrate the functionality of our best performing filters with real physician searches. In each case, the clinical questions formulated from recent systematic reviews were relevant to glomerular disease and physician searches appear typical for the average user. These examples show a gain in search strategy precision with use of the high-sensitivity and high-specificity filter through a dramatic reduction in non-relevant articles. This occurs without sacrificing retrieval of relevant articles in most cases. However, the methods for defining the reference standard based on articles used in systematic reviews of variable quality is indirect and has not been compared with one derived from hand searching [34
]. For this reason the proof of concept searches should be viewed as illustrative examples, not as evidence of further filter validation.
These search filters for glomerular disease were designed to offer physicians and researchers a strategy to optimize results by sensitivity or specificity, depending on the level of article retrieval they deem manageable on a practical level. Filters that maximize sensitivity involve a compromise on the level of precision achieved, though this still may appeal to a researcher conducting a systematic review. For busy physicians at the point of care, we recommend starting with the high-specificity filter. To narrow results even further, physicians may prefer use these search filters in conjunction with previously developed methods filters, such as the therapy filter for randomized controlled trials available via PubMed’s Clinical Queries section [5
]. This approach has not been formally tested with the glomerular disease filters, but in a recent study has been shown to increase the efficiency of retrieval of articles relevant to renal care [35