The field of computational chemistry, particularly as applied to drug design, has become increasingly important in terms of the practical application of predictive modeling to pharmaceutical research and development. Tools for exploiting protein structures or sets of ligands known to bind particular targets can be used for binding-mode prediction, virtual screening, and quantitative prediction of activity. A serious weakness within the field is a lack of standards with respect to statistical evaluation of methods, data set preparation, and data set sharing. Our goal should be to report new methods or comparative evaluations of methods in a manner that supports decision making for practical applications. In this editorial, we propose a modest beginning, with recommendations for requirements on statistical reporting, requirements for data sharing, and best practices for benchmark preparation and usage.
There are two fundamental premises in making such a proposal. First, we must believe that the goal of reporting new methods or evaluations of existing methods is to communicate the likely real-world performance of the methods in practical application to the problems they are intended to solve. Ideally, the specific relationship between methodological advances and performance benefits will be clear in such reports. Second, we must understand that the utility of the methods of broad utility in pharmaceutical research application are predicting things that are not known at the time that the methods are applied. While this seems elementary, a substantial proportion of recent reports within the field run afoul of this observation in both subtle and unsubtle ways. Rejection of the first premise can reduce scientific reports to advertisements. Rejection (or just misunderstanding) the second premise can distort any conclusions as to practical utility.
This special issue of the Journal of Computer-Aided Molecular Design includes eleven papers, each of which makes a detailed study of at least one aspect of methodological evaluation [1
]. The papers collected within this issue make the detailed case for the recommendations that follow; the recommendations are intended to provide guidance to editorial boards and reviewers of work submitted for publication in our field. In surveying the eleven papers, we feel there are three main areas of concern: data sharing, preparation of datasets, and reporting of results. Concerns within each area relate to three main subfields of molecule modeling, i.e. virtual screening, pose prediction, and affinity estimation, and to whether protein structural information is used or not. We describe the issues in each area and then present recommendations drawn from the papers herein.