Inferring high-confidence human protein-protein interactions

doi:10.1186/1471-2105-13-79

BMC Bioinformatics. 2012; 13: 79.

Published online 2012 May 4. doi: 10.1186/1471-2105-13-79

PMCID: PMC3416704

Inferring high-confidence human protein-protein interactions

Xueping Yu,¹ Anders Wallqvist,¹ and Jaques Reifman¹

¹Biotechnology High-Performance Computing Software Applications Institute, Telemedicine and Advanced Technology Research Center, U.S. Army Medical Research and Materiel Command, Ft. Detrick, MD, 21702, USA

Corresponding author.

Xueping Yu: xyu/at/bioanalysis.org; Anders Wallqvist: awallqvist/at/bioanalysis.org; Jaques Reifman: jaques.reifman/at/us.army.mil

Received October 5, 2011; Accepted May 4, 2012.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

As numerous experimental factors drive the acquisition, identification, and interpretation of protein-protein interactions (PPIs), aggregated assemblies of human PPI data invariably contain experiment-dependent noise. Ascertaining the reliability of PPIs collected from these diverse studies and scoring them to infer high-confidence networks is a non-trivial task. Moreover, a large number of PPIs share the same number of reported occurrences, making it impossible to distinguish the reliability of these PPIs and rank-order them. For example, for the data analyzed here, we found that the majority (>83%) of currently available human PPIs have been reported only once.

Results

In this work, we proposed an unsupervised statistical approach to score a set of diverse, experimentally identified PPIs from nine primary databases to create subsets of high-confidence human PPI networks. We evaluated this ranking method by comparing it with other methods and assessing their ability to retrieve protein associations from a number of diverse and independent reference sets. These reference sets contain known biological data that are either directly or indirectly linked to interactions between proteins. We quantified the average effect of using ranked protein interaction data to retrieve this information and showed that, when compared to randomly ranked interaction data sets, the proposed method created a larger enrichment (~134%) than either ranking based on the hypergeometric test (~109%) or occurrence ranking (~46%).

Conclusions

From our evaluations, it was clear that ranked interactions were always of value because higher-ranked PPIs had a higher likelihood of retrieving high-confidence experimental data. Reducing the noise inherent in aggregated experimental PPIs via our ranking scheme further increased the accuracy and enrichment of PPIs derived from a number of biologically relevant data sets. These results suggest that using our high-confidence protein interactions at different levels of confidence will help clarify the topological and biological properties associated with human protein networks.

Keywords: High confidence, Human protein interaction network, Protein-protein interactions

Background

The development of high-throughput techniques during the last decade has led to an unprecedented increase in the volume of identified human protein-protein interactions (PPIs). The currently available individual PPI data sets can be roughly categorized into three sets: 1) proteome-wide, large-scale screenings aimed at investigating all possible PPIs [1-3], 2) semi-large-scale screenings aimed at investigating the interactions between a specific group of proteins (typically in a pathway) and all other proteins [4,5], and 3) small-scale, traditional studies aimed at detecting specific PPIs among biologically interesting proteins, e.g., oncogenes and their regulators. Although this latter set is still numerically dominant (~80% of all PPIs belong to this set), examples of the first two types of investigations are expanding rapidly.

Given this extensive resource of known human PPIs and their continuous accelerated growth, how to globally analyze and aggregate the data remain a challenge. Statistical methods for inferring confidence of protein interactions can be broadly divided into two groups [6-8]: scoring schemes that rely on the interaction data themselves (e.g., affinity purification/mass-spectrometry [AP/MS] data or yeast two-hybrid [Y2H] data) and scoring schemes that require additional data sources not directly related to the interactions per se (e.g., functional annotation or gene expression data). Herein, we address the question of how to extract high-confidence PPIs while relying only on the aggregated interaction data themselves.

The most intuitive approach to infer high-confidence PPIs is to score PPIs based on the number of times an interaction has been reported [9-11]. However, using the number of times a PPI has been reported (occurrence) across different studies as the metric of reliability could be influenced by numerous unknowable experimental factors, e.g., recent studies have demonstrated that such factors may result in decreased reliability of PPIs containing frequently studied proteins [12]. Moreover, a large number of PPIs share the same number of reported occurrences, making it impossible to use occurrence alone to establish the reliability of these PPIs and rank-order them. For example, for the data analyzed here, we found that the majority (>83%) of currently available human PPIs have been reported only once.

Herein, we propose an unsupervised statistical approach to score and rank a set of diverse, experimentally identified PPIs. We applied this methodology to human PPIs (non-physical associations excluded) aggregated from nine publicly available primary databases that exclusively contain experimental data (Additional file 1): the Biomolecular Interaction Network Database (BIND) [13], the Biological General Repository for Interaction Datasets (BioGRID) [14], the Database of Interacting Proteins (DIP) [15], the Human Protein Reference Database (HPRD) [16], IntAct [17], the Molecular INTeraction database (MINT) [18], the mammalian PPI database of the Munich Information Center on Protein Sequences (MIPS) [19], PDZBase (a PPI database for PDZ-domains) [20], and Reactome [21]. Our method re-normalizes the importance of frequently occurring proteins among PPIs to avoid giving added (and potentially artificial) weight to those interactions. We estimated the importance of a PPI by comparing the actual observed occurrence of a PPI with its occurrence in a randomized sample. This calculation gauges the likelihood that the interaction occurs by chance in the set of all observed PPIs. Using these estimates, we rank-ordered the aggregated input PPI data set, allowing us to create high-confidence subsets based on a given rank threshold. At the lowest ranked threshold, all interactions are included and there is no difference between the ranked data and the original set of PPIs.

The presented scoring and ranking procedure can be seen as an extension of our previous effort to infer high-confidence interactions from the affinity purification raw data, termed interaction detection based on shuffling (IDBOS) [22], and, in the following, we will also refer to our scoring and ranking scheme as IDBOS. Our proposed procedure shares similarities to estimating probabilities of observed interactions above a random background based on the hypergeometric distribution [23], with the distinction that the IDBOS-generated probability density distribution functions correct for biases toward self-interaction among frequently studied proteins. Although other methods exist for assigning confidence scores to PPIs, these generally require additional data or reference sets [24,25], or a priori assumptions of network topology [26]. To the best of our knowledge, this is the first application of an unsupervised probabilistic scoring and ranking scheme to create subsets of unbiased high-confidence human PPI networks.

We evaluated the improvement in using IDBOS-ranked PPI data by comparing it with other methods and assessing their ability to retrieve biological associations from a number of diverse and independent reference sets. These reference sets contain known biological data that are either directly (e.g., crystallographically determined protein complexes) or indirectly (e.g., co-expressed genes) linked to interactions between proteins. The hypothesis we tested was that sets of highly ranked PPIs are enriched in biological associations as determined from the diverse reference sets. We quantified the average effect of using ranked protein interaction data to retrieve this information and showed that, when compared to randomly ranked interaction data sets, IDBOS created a larger enrichment (~134%) than either ranking based on the hypergeometric test (~109%) or occurrence ranking (~46%).

From our evaluations, it was clear that ranked interactions were always of value because higher-ranked PPIs had a higher likelihood of retrieving biologically relevant data. Statistically removing the biasing factors inherent in aggregated PPI data via the IDBOS-ranking scheme further increased the accuracy and enrichment of biological information associated with PPIs.

Results

Statistical scoring of human PPIs from literature data

We used the collection of human experimental and physical PPIs to create a set of 116,134 reported interactions, containing 80,980 unique physical associations between 13,369 distinct proteins (see Materials and methods). Out of the unique PPIs, 13,554 interactions were observed in more than one experiment. The number of times a PPI has been reported in the literature is an important metric for inferring high-confidence interactions [9,10,27], but it could also be dependent on other factors [12,28]. The observed number of interactions of a protein is partly reflective of how often it has been studied (popularity). For example, the top five connected proteins in the PPI data are G-protein beta subunit (GNB1), G-protein gamma subunit (GNGT1), G-protein alpha subunit (GNAL), ubiquitin C (UBC), and tumor protein p53 (TP53), having 2,280, 2,248, 2,243, 1,899, and 1,097 reported interactions, respectively. To normalize this popularity bias, we compared the observed number of protein interactions with statistics derived from the corresponding probability density distribution functions generated from randomized data. In the generation of the random interaction sets, we kept each protein’s reported number of interactions fixed and, thus, we expect the corresponding interaction probabilities of proteins with a high (low) number of reported interactions also to be high (low).

Note that the reported number of interactions involving a protein refers to the total number of observed PPIs in the literature involving that specific protein. This is different from a protein’s degree, which is defined as the number of unique interacting protein partners. Thus, while TP53 is associated with 1,097 observed PPIs, its degree is reduced to 478 due to the multiple observations of many of the involved interactions.

Our calculations followed our previously described procedures for generating Z-scores from sets of interacting protein pairs [22]. Briefly, the aggregated PPI data tabulates all pairs and the number of their occurrence in the literature. From this list we counted, for each unique pair between proteins i and j, how many times it occurred O_ij. Randomized versions of the original PPI list were then created under the conditions that: 1) the protein identifiers and the number of times they occur are preserved and 2) no interactions are allowed between proteins of the same identifier, i.e., self-interactions are not allowed. We generated M

10⁶ randomized PPI lists and calculated the average number of times each PPI from the original list occurred

R_ij

and its standard deviation σ_R. The corresponding Z-score was then calculated as,

(1)

We further used the randomized data to estimate the p-value p_ij of observing an interaction between proteins i and j in the original data set. Briefly, for each unique pair in the original PPI list, we calculated the number of times it occurred in each of the n

1, … ,M random realizations and created the normalized probability density function PDF_ij, i.e., the probability of finding equation M2

occurrences by random chance. We estimated the p-value for each actual interaction as,

(2)

Z-scores and p-values (Z_ij and p_ij) are legitimate metrics for ranking an observed PPI, although they are associated with different numerical properties and uncertainties because of the limitations imposed on using a finite number of randomizations.

Assessing p-values and Z-scores as PPI metrics

To assess the p-values and Z-scores as metrics in identifying and ranking high-confidence protein interactions in biological data sets, we compared the scored interactions with an independent PPI data set that was not part of the aggregated human data set. We used the human PPIs derived from experimentally determined structural protein complexes deposited in the Protein Data Bank (PDB) as outlined in the Materials and methods (Additional file 2). To assess our scored data using this data set as a benchmark, we selected only those interacting protein pairs from the scored human PPI data where both proteins appeared in the PDB-derived PPI data set. We termed these pairs as “judgeable,” i.e., the proteins appear in the benchmark data set, and we can judge whether the interaction occurred or not. Thus, for a given score, we determined how many of our judgeable PPIs were interacting and non-interacting within the benchmark set. Figure

1A-B shows the frequency of interacting and non-interacting pairs as a function of Z-scores and p-values. Interactions associated with high Z-scores and small p-values were enriched with benchmark interactions, indicating that both metrics were effective in extracting high-confidence PPIs from the aggregated literature data.

Figure 1

IDBOS scoring schemes. The method presented is an extension to the interaction detection based on shuffling (IDBOS) method used for mass spectrometry co-purification data [22]. We compared the set of known protein-protein interactions (PPIs) with randomized (more ...)

The procedure for shuffling the data allowed us to compare the frequency of each observed interaction O_ij to that interaction’s probability density distribution function PDF_ij. This distribution is different for each interaction and depends on the number of distinct proteins and interactions present in the entire network. Lacking an analytical expression to generate the exact distribution functions, the procedure outlined in Equations 1 and 2 allowed us to generate estimates for both Z-scores and p-values for each interacting pair. We found that using either only p-values or only Z-scores to be inferior to using the combined information (data not shown), and, hence, we aggregated these two metrics by converting the value of each metric into a rank and generating the average rank from both metrics for each interaction. Interactions with the same p-value (or Z-score) were assigned the same rank. If interactions had the same rank in the final averaged-rank list, these interactions were considered equivalent and analyzed together. The ranked data are provided in the Supplementary materials (Additional file 3). Instead of assigning post-priori probabilities to already observed interactions, we only used ranks and comparisons between ranked data to gauge the biological information contained in these subsets.

Top-ranked IDBOS PPIs are different from the most frequently reported PPIs

Table

1 shows the 20 top-ranked PPIs that were identified using the IDBOS system (sorted by average rank), and Table

2 shows the most frequently reported PPIs (sorted by PPI occurrence). As expected, the most frequently reported PPIs involved ubiquitously studied proteins, such as those mentioned above (UBC and TP53) as well as growth factor receptor bound 2 (GRB2), which had 916 reported interactions. These interactions, however, were not observed in the top PPIs scored by the IDBOS procedure. Instead, the top PPIs of the IDBOS set were enriched with infrequently observed interactions. Although we could not independently assess whether these interactions are more “real” than the frequently observed ones, they are partly supported by independent literature citations. In fact, the proteins in these interactions could be classified into two groups. The first group comprised proteins that had the same specific function or were subunits of the same protein complex, such as branched chain keto acid E1 alpha (BCKDHA) and branched chain keto acid E1 beta (BCKDHB) [3,29], and dynein cytoplasmic 2 intermediate chain 1 (D2LIC) and dynein cytoplasmic 2 heavy chain 1 (DNCH2) [30,31]. The second group comprised proteins that had a ligand/receptor relationship, such as inducible T cell co-stimulator (ICOS) and inducible T cell co-stimulator ligand (ICOSLG) [32-34], or gastric inhibitory polypeptide (GIP) and gastric inhibitory polypeptide receptor (GIPR) [35-38]. Among the top 20 PPIs of the IDBOS set, two seemingly unrelated protein pairs were actually closely related. The proteins in the LTC4S/MGST1 interacting pair are actually two of the six members of the membrane-associated proteins in the eicosanoid and glutathione metabolism (MAPEG) family. Similarly, the top-ranked interaction between L-threonine dehydrogenase (TDH) and aminoacetone synthetase (alias of GCAT) catalyzes the conversion of L-threonine to glycine [39].

Table 1

Top 20 protein-protein interactions from IDBOS ranking

Table 2

Top 20 protein-protein interactions from occurrence ranking

Not only were the top-ranked sets different, but the remaining bulk of the interactions also showed considerable changes in rank when the IDBOS p-values were non-zero (See Additional files 3 and 4). To evaluate the effect of this re-ranking of interactions, we then asked whether these rankings have any impact in retrieving biological information. We addressed this question by comparing the ability of the two schemes to identify known and inferred biological relationships based on rank. The hypothesis we tested was that higher-ranked subsets of the data sets are better at retrieving biological information, and that IDBOS ranking provides a more efficient way of retrieving this information than ranking solely based on frequency of occurrence.

Evaluation of ranking schemes as measures of identifying interacting proteins

To assess different PPI ranking schemes, we constructed six benchmark reference sets derived from high-quality experimental studies as detailed in Materials and methods. These independent data sets comprise PPIs detected using 1) far-Western blotting, 2) isothermal titration calorimetry, 3) nuclear magnetic resonance, 4) surface plasmon resonance, 5) direct interactions from protein complex structures from the PDB, and 6) homologous human PPIs derived from actual mouse PPI data. These benchmark sets tested the ability of the ranked data to retrieve known interactions derived from a variety of experimentally determined PPI data sets.

Using these six benchmark reference sets, we compared the IDBOS-ranked data set with a set ranked by PPI frequency of occurrence, a set ranked using the hypergeometric test [23] (see Materials and methods, Additional file 5), and two random data sets. The random data sets included a set in which the PPIs themselves were retained, but their ranks were assigned randomly, and a set where both the interactions themselves were randomized and were assigned a random rank. In addition, we included the results obtained by directly using the nine individual data sources that formed the aggregated collection of human PPIs. The objective here was to assess each ranking method’s capability to retrieve true interactions in each of the reference sets, as a function of rank, i.e., higher-ranked data should contain a larger fraction of true interactions than lower-ranked data.

To make a fair comparison within each benchmark reference set, for each set we defined a protein pair from the aggregated data set as a judgeable interaction if both proteins appeared in the reference set; otherwise, we assumed that the pair could not be judged by the reference set. If a judgeable interaction was observed in the reference set, we termed it a true interaction. A good scheme would rank true interactions before other judgeable interactions. To quantify the ability to retrieve true interactions in each scored set, we extracted the judgeable subset and, using each corresponding rank as a threshold, we counted the numbers of judgeable and true interactions with scores above the threshold. We defined the number of true interactions as coverage and the fraction of true interactions among those judgeable interactions as accuracy.

Figure

2 shows the results of this analysis for the six interaction benchmark reference sets. As expected, the completely random PPI data set had no capability to retrieve direct protein interaction data. For the network created by assigning random ranks to existing PPIs, the accuracy was almost uniform at any given coverage.

Figure 2

Retrieving protein-protein interactions. The proposed interaction detection based on shuffling (IDBOS) ranking scheme was compared with hypergeometric test ranking, frequency of occurrence ranking, one randomly ranked set (where the observed protein-protein (more ...)

IDBOS, hypergeometric, and frequency of occurrence ranking were each associated with enhanced accuracy at higher ranks, thus indicating that true interactions ranked before other judgeable interactions. We assessed the overall performance of the ranking methods by calculating relative improvement compared to using no ranking, which is equivalent to assigning random ranks to the existing data. For the accuracy as a function of coverage plots shown in Figure

2, we calculated the average accuracy equation M4

over all ranks r as,

(3)

where n(r) is the number of true PPIs at rank r and N is the sum of all n(r)’s. The gain of using IDBOS, hypergeometric, or occurrence as a ranking method was estimated by comparing the average accuracy equation M6

to the randomly ranked data equation M7

as,

(4)

Table

3 lists the gains in average accuracies in each reference set for the different ranking schemes. The IDBOS ranking scheme shows the greatest increase compared to using randomly assigned ranks, achieves a two-fold increase in accuracy compared with using frequency of occurrence ranking, and consistently outperforms the hypergeometric ranking. In practice, one should also note that selecting any high-accuracy subset of PPIs based on ranked interaction data creates a smaller data set and lowers the overall PPI coverage.

Table 3

Evaluation of different ranking schemes

The independent scores (Z-scores or p-values) were typically better than, or equivalent to, the hypergeometric method: Z-score ranking was close to the IDBOS, and better than the hypergeometric, whereas the p-value ranking was not as good as the hypergeometric. For example, using the PDB benchmarking set, the gains of the IDBOS, Z-score, hypergeometric, and p-value methods relative to the average accuracy of the randomly ranked data were 258%, 253%, 217%, and 216%, respectively; using the mouse homologous benchmarking set, the gains were 222%, 211%, 198%, and 155%, respectively. Thus, the IDBOS combined rankings consistently outperformed all attempted scoring schemes.

IDBOS versus hypergeometric test rankings

The observed differences in Table

3 and Figure

2 between the IDBOS and hypergeometric test rankings warrants further comment. The fundamental difference between the IDBOS and hypergeometric test rankings lies in how they account for interactions between self-interacting proteins. For the IDBOS randomized data, each pair of proteins was re-assigned an interaction partner such that no self-interacting protein pairs remained in the final set used to construct the probability density distribution functions. In the hypergeometric test, self-interacting protein pairs were assigned finite probabilities of occurrence based on a background distribution for each protein pair, which was different from IDBOS. Conceptually, the sum of the probability of observing all interactions with a given protein A among all other proteins p_AA, p_AB, p_AC, etc., was the same in both schemes. However, the constraint that p_AA was zero in IDBOS and p_AA was non-zero in the hypergeometric test, re-distributed the probabilities such that any interaction probability p_AB between protein A and another protein B could be different in the two schemes. This strongly influenced the probability of detecting proteins that occurred with high frequency in the data set.

The effect of including or excluding self-interacting protein pairs was magnified in the evaluation of interactions involving popular proteins. In the hypergeometric test, the likelihood of randomly generating self-interacting protein pairs is roughly proportional to the square of the number of times the protein appears in the data set. In practice, this leads to an underestimate of the occurrence of non-identity pairs in the random data, and this increases the significance attached to the observed non-identity PPIs involving popular proteins. The effect on ranking PPIs was considerable, e.g., among the top 20 occurrence-ranked interactions listed in Table

2, 11 appeared in the top 20 hypergeometric test ranking scheme. In fact, the first occurrence-ranked interaction between MDM2 and TP53 was still the second ranked interaction based on the hypergeometric test. Figure

3 shows the overlap of interactions among top-ranked PPIs based on IDBOS, the hypergeometric test, and frequency of occurrence rankings. As exemplified above, there was a noteworthy overlap (>0.40) between the ranking results in the frequency of occurrence and the hypergeometric test rankings for all ranks tested. In contrast, the first 10³ top-ranked IDBOS PPIs showed low overlap (~0.10) with the frequency of occurrence ranked PPIs, but higher similarity (~0.36) to the ranked PPIs identified using the hypergeometric test. Thus, the high overlap between occurrence and hypergeometric rankings suggested that the inadvertent biases introduced by frequent investigations of popular proteins could not be completely disentangled by the hypergeometric-ranking scheme. Furthermore, we verified that consensus ranking schemes that included the average rank of all three ranking methods, or combining IDBOS and hypergeometric test rankings only, did not increase the accuracy beyond using IDBOS rankings (data not shown). Instead, we contend that the corresponding probability density distribution functions generated by IDBOS are the most appropriate to gauge unbiased PPIs in this data set.

Figure 3

Overlap of top-ranked protein interactions. The overlap of top-ranked protein-protein interactions (PPIs), based on a pairwise comparison of three interaction evaluation schemes: interaction detection based on shuffling (IDBOS), hypergeometric test ranking, (more ...)

Enrichment of known domain-domain interactions

The hypothesis that PPIs are mediated by a smaller set of specific domain-domain interactions (DDIs) that are repeatedly used, can be exploited by inferring DDIs from known PPIs and then predicting novel PPIs from the inferred DDI set [40,41]. A high-confidence PPI set, such as the collection of highly ranked PPIs, is more likely to contain reliable DDIs. To confirm this, we investigated the fraction of known DDIs among all candidate domain pairs as a function of the rank threshold. As before, we only considered judgeable protein pairs whose domains appear in the set of known DDIs. The known DDI set was inferred from the PDB crystal structures [42]. As two random controls, we also randomly ranked and randomly rewired the judgeable protein pairs. Figure

4 shows that more stringent ranking thresholds (represented by fewer “Top judgeable PPIs” in Figure

4) resulted in higher enrichments of DDIs among all domain pairs between interacting proteins for both the IDBOS and frequency of occurrence rankings. This behavior was in contrast with the two random controls, which did not show such enrichments. Table

4 quantifies the improvement gained by ranking the data relative to no rankings. Within the same top number of PPIs, the IDBOS set had a higher enrichment of known DDIs than sets ranked either by the hypergeometric test or by frequency of occurrence, indicating a potential application of the IDBOS-ranked PPI sets for the inference of accurate DDIs.

Figure 4

Recovery of known domain interactions from protein interactions. The proposed interaction detection based on shuffling (IDBOS) ranking scheme was compared with the frequency of occurrence ranking, one randomly ranked set (where the observed protein-protein (more ...)

Table 4

Domain-domain interaction enrichment

Evaluation of ranking PPIs as a means of retrieving biological information

Up until now, the different scoring schemes were used to evaluate reference data sets that can be considered to be directly related to interacting proteins. We next evaluated the improvement that could be gained by using the differently ranked data sets to rank-order the interactions in reference data sets that are presumed to be enriched with interacting proteins, i.e., in sets of proteins that share the same Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway, are implicated in the same disease, share Gene Ontology (GO) function, or tissue mRNA expression levels.

Case I: Enrichment of KEGG co-pathway gene pairs

Genes encoding interacting proteins are more likely to be part of the same pathway. Hence, the fraction of PPIs that are annotated to belong to the same pathway should be larger in sets containing higher-ranked PPIs. We used KEGG pathway data [43] to define co-pathway gene pairs, as outlined in the Materials and methods. For each investigated PPI set, we excluded protein pairs involving proteins that did not participate in any KEGG pathway (these were deemed to be non-judgeable). Figure

5A shows the results of this analysis. We confirmed the correlation between co-pathway membership of a protein pair and its rank threshold in both the IDBOS and the frequency of occurrence ranking schemes, as well as the absence of correlation in the two random controls. Overall, IDBOS ranking outperformed frequency of occurrence ranking by a factor of four for enrichment of co-pathway gene pairs among selected top-ranking PPIs (Table

5), and the gain seen in hypergeometric ranking was substantially smaller than that of IDBOS ranking. There is a discontinuity in the curve for the frequency of occurrence ranking scheme, which was caused by hundreds of interactions between three subunits of G protein (GNAL, GNB1, and GNGT1) and olfactory proteins which were supported by seven publications (Additional file 1). We reviewed these publications and found that this literature support was weak, with some based solely on gene expression evidence of those olfactory genes [44]. Among these hundreds of PPIs, those involved with GNB1 (beta subunit) or GNGT1 (gamma subunit) were not supported by KEGG pathway data, resulting in a sharp drop in accuracy.

Figure 5

Recovery of biological co-annotations from protein interactions. The proposed interaction detection based on shuffling (IDBOS) ranking scheme was compared with the frequency of occurrence ranking, one randomly ranked set (where the observed protein-protein (more ...)

Table 5

Evaluation of ranked interactions to detect biological relationships

Case II: Enrichment of co-disease gene pairs

Genomic variations may underlie different susceptibilities to disease, e.g., individuals with specific single nucleotide polymorphisms (SNPs) can develop, or be pre-disposed to, a particular disease phenotype. Furthermore, genes encoding interacting proteins are more likely to occur within the same disease classification [45]. Using the gene co-disease data extracted by Goh et al. [45] from the Online Mendelian Inheritance in Man (OMIM) dataset [46], we investigated the enrichment of co-disease gene pairs as a function of rank threshold (Figure

5B, Table

5). A gene pair was termed a co-disease gene pair if their SNPs led to susceptibility to the same disease. With this co-disease gene pair set, we also extracted judgeable protein pairs whose encoding genes appeared in this co-disease set (i.e., genes that have an actual disease annotation). Similar to the previous analyses, the IDBOS-scored set showed an enhanced probability of identifying existing disease-susceptible genes as compared with the frequency of occurrence ranking scheme, hypergeometric-based rankings, or random controls.

Case III: Enrichment of GO co-function gene pairs

Figure

5C shows the ability of the IDBOS and frequency of occurrence ranking schemes to identify PPIs whose proteins share the same gene ontology (GO) function. Both ranking schemes displayed a similar rank-dependent enrichment of co-function proteins compared to the randomly ranked (all ranks have the same importance) and randomly rewired (interactions are random) data set. Table

5 indicates that the improvements of the ranking schemes compared to randomly ranked data are relatively modest in retrieving functionally related gene pairs. As pointed out by Gillis and Pavlidis [28], the ability of PPI networks to distinguish co-functionality is largely influenced by the multifunctional nature of the gene annotations schemes themselves. In this case, the IDBOS, hypergeometric, and occurrence ranking schemes themselves will also have less influence on the retrieval of functionally related protein pairs compared to the annotation scheme itself.

Case IV: Enrichment of tissue co-expression gene pairs

Finally, we examined the occurrence of interacting protein pairs in a large data set that maps out the global mRNA expression levels for ~20,000 human genes in 32 normal tissues [47]. For detail on the construction of this reference set, see Materials and methods. The biological hypothesis tested here examined co-expression of two mRNAs in the same tissue as indicative that the corresponding proteins had higher probability of interaction. Figure

5D shows the fraction of PPIs retrieved as a function of ranking scheme. Overall, the fraction of PPIs contained in the co-expression data set was lower than that of the other biological reference sets examined above. However, the investigated ranking schemes were able to produce ranked data sets that contain a slightly larger fraction of co-expressed PPIs than using no ranking (Table

5).

Topological differences of ranked PPI networks

Data sets of protein interactions are commonly represented as graphs or networks of protein interactions, and applying network topology metrics to characterize the biological role of PPIs has attracted wide attention [9,10,22,48]. The presence of confounding factors, e.g., protein abundance [49] or popularity [12], can strongly influence the topological properties of the network. Here, we used subsets of highly ranked PPIs derived from using either IDBOS or occurrence ranking to select smaller high-confidence networks. Figure

6A shows the overall networks constructed from the top-ranking PPIs corresponding to 4,425, 6,561, and 13,554 interactions selected based on the number of PPIs that have more than or equal to four, three, and two reported occurrences, respectively. Figure

6B shows the corresponding degree distribution, i.e., the distribution of the number of unique interacting partners each protein has (Degree), as well as the overall degree distribution for the entire network (All).

Figure 6

Network topology of ranked interactions. (A) We reconstructed the corresponding top-ranked protein-protein interactions (PPIs) networks using both interaction detection based on shuffling (IDBOS) ranking and frequency of occurrence ranking. The columns (more ...)

Selecting highly ranked subsets of PPIs, using either IDBOS or occurrence, had the effect of reducing the number of high-degree proteins (hubs) in the overall network. This effect was most pronounced in the top-ranked IDBOS networks, which effectively down-weights interactions of popular proteins. The resulting top-ranked IDBOS networks were not dominated by hub interactions, but show a more distributed interaction network, with distinct topological and scaling properties.

Discussion

Single experiments designed for large-scale detection of PPIs judge multiple observations of the same PPI as evidence for the confidence of the interaction. Multiple observations, across many different experiments, of a particular PPI also lend confidence that this interaction occurs in nature. One would then conclude that if one aggregated all known PPI experiments, the more times a particular PPI occurred, the more confident one would be of the interaction. We found a somewhat counterintuitive result in that ranking PPIs in order of the number of times a PPI has been observed was actually not the optimal way of assessing the importance of frequently reported interactions. Instead, we used the IDBOS method, which ranks interacting protein pairs in the observed PPI data sets as compared to random occurrences derived from an empirically reconstructed probability density distribution function specific to each interaction. We then used these ranks to order the PPI data and showed that this approach consistently identified more known protein associations in numerous benchmark reference sets than using either hypergeometric test rankings or simply frequency of occurrence. The improvement of IDBOS over the hypergeometric test results stems from the differences in the underlying random distributions in the two schemes. While the hypergeometric test does not impose any limitations on self-interacting proteins, by construction, the IDBOS procedure sets their probability of occurrence to zero even for the random case. The input data contain no self-interacting proteins; therefore, the corresponding random probability density distribution functions generated by IDBOS seem to be the most appropriate to gauge unbiased PPIs in this data set.

We evaluated the improvement in using ranked PPI data to identify known protein relationships in eleven independent reference sets, including direct and indirect readouts of protein interactions. We used a large number of different reference sets to perform as comprehensive an analysis as possible, as each reference set only covers a specific subset of protein interactions. It was clear that ranking interactions was of value, because higher-ranked PPIs had better success at retrieving protein associations. Across all sets investigated, occurrence ranking improved the performance by 46% compared to using non-ranked data, hypergeometric ranking achieved a 109% increase, and the IDBOS ranking scheme achieved a 134% increase. This conformed to the assumption that aggregating and creating high-confidence subsets of data creates added value, albeit at a cost of reducing the overall number of PPIs that can be considered.

Conclusions

We have developed a statistical approach to infer subsets of high-confidence human PPIs, and showed that ranked data can consistently enrich the accuracy of the retrieved PPI data in these sets. Our IDBOS method was more successful in ranking interactions than using either the number of times an interaction has been observed across experiments or rankings based on the hypergeometric test. Furthermore, using either IDBOS or hypergeometric scoring schemes generates unique ranks for almost all interactions, as opposed to the frequency of occurrence method, in which many interactions have the same integer score corresponding to the number of observed occurrences. The IDBOS ranking increased accuracy and enrichment of protein interaction data associated with PPIs by more than two-fold compared to simply ranking interactions based on observed occurrences. We achieved this improvement by comparing the observed interaction data with a probability density distribution function that does not inflate the statistical importance of interactions associated with frequently studied proteins. These results suggest that using our high-confidence protein interactions at different levels of confidence could help clarify the dependence on confidence on topological and biological properties associated with human protein networks.

Materials and methods

Statistical scoring of PPIs from aggregated experimental data

We downloaded the collection of PPIs in October 2011 from the nine databases that covers the bulk of all known experimentally determined PPIs. Databases of non-primary nature, i.e., containing aggregated data and/or predicted and inferred interactions, were excluded. From this collection, we extracted the subset of human physical PPIs, consisting of PPIs from both large-, semi-large-, and small-scale experimental studies. Because Reactome has no standardized annotations describing physical associations or direct interactions, instead we extracted PPIs annotated as “direct complexes.” We treated each interaction reported by each study (identified by a unique publication ID) as a unique record by deleting redundant copies arising from the overlaps among the nine databases. In total, we analyzed 13,369 proteins, 80,980 PPIs, and 116,134 records (Additional file 1). The computational procedure to generate 10⁶ random realizations of the PPI data set and compute the corresponding p-values and Z-scores took ~2,000 minutes on a dual core Xeon Irwindale 3.6

GHz 64-bit Linux server equipped with 4

GB of RAM. We used fractional rankings, i.e., PPIs that had the same score received the same ranking number, which is the mean of what they would have received under ordinal rankings. The Supplementary material provides the scored and ranked PPI data set, with equivalently ranked interactions tabulated in arbitrary order.

To rank interactions based on an alternative statistical method compared to IDBOS and frequency of occurrence, we included ranks based on a hypergeometric test [23]. For two proteins, i and j, given that the interaction between i and j occurred N_ij times, the probability (p_ij) for these two proteins to have this or a larger number of interactions by chance was approximated using a hypergeometric distribution as follows:

(5)

where N_i (N_j) is the number of times protein i (j) was observed in the aggregated data set, and N is the total number of interactions in the aggregated data set of PPIs. This formulation is correct in the limit of N >

N_iN_j, which was satisfied in this data set. Finally, we ranked the PPI data based on the calculated probabilities.

PDB interactions

We mapped all PDB sequences (downloaded in October 2010) to the human protein sequences stored in the UniProtKB/Swiss-Prot database (downloaded in June 2008) and obtained 649 protein complexes that contained at least two different human proteins. We then calculated the contact area of each intra-complex human protein pair using the program EMPIRE [50], to determine which protein pairs interact. For protein pairs occurring in multiple complexes, we selected the pair with the largest contact area as an interacting pair. We collected 281 direct protein interactions between 563 proteins, with contact areas ranging from 0.1 to 183

nm², as a reference set for known human PPIs derived from structural data (Additional file 2).

Reference sets from the HIPPIE database

We extracted four reference sets from the Human Integrated Protein-Protein Interaction rEferenc (HIPPIE) data set (http://cbdm.mdc-berlin.de/tools/hippie/ information.php, January 2012), each of which consists of PPIs detected by a top-scored experiment technique (score

10, specified with European Bioinformatics Institute (EBI) term ID, and number of PPIs >100). The extracted data set contained PPIs derived from 1) far-Western blotting, 2) isothermal titration calorimetry, 3) nuclear magnetic resonance, and 4) surface plasmon resonance experiments.

Homologous human PPIs from mouse data

We extracted the subset of mouse physical PPIs from the above collection and constructed a homologous human PPI set according to the sequence homology between mouse and human proteins defined by the National Center for Biotechnology Information (NCBI) homology mapping scheme (ftp://ftp.ncbi.nih.gov/pub/HomoloGene/, October 2009). We used this homologous human PPI set (containing 3,148 interactions between 5,632 proteins) as a reference data set reflective of direct protein interactions.

Domain annotation and interaction data

We used the Pfam-A families of the Pfam21.0 database as the source for domain annotation [51]. We also downloaded the DDI set inferred from PDB crystal structures from the iPfam database [42]. This DDI set contains 4,030 interactions between 2,837 Pfam-A domains.

Co-disease susceptibility gene pairs

We used the disease susceptibility gene data of Goh et al. [45], which was constructed by processing OMIM raw data [46]. There were 43,249 pairs between 3,670 genes whose two genes were susceptible to at least one common disease. We used these pairs to represent the co-disease susceptibility gene pairs.

Co-function gene pairs

We downloaded the GO annotation data (http://www.geneontology.org, June 2010) to identify a set of functionally related gene pairs as a reference set. For each gene, we expanded its GO annotation list by including all ancestors of each member in the list. For two genes, g and h, given that they had n_gh GO annotations in common, the probability (p_gh) for these two genes to have this or larger annotation overlap by chance was estimated from the hypergeometric distribution as follows:

(6)

where N_g (N_h) is the number of GO annotations of gene g (h) and T is the total number of unique GO annotations. We used –log(p_gh) as the score and chose the top 1% of all gene pairs as a reference set, resulting in 1,502,420 co-function pairs for this reference set.

Co-pathway gene pairs

The KEGG pathway data file (hsa_gene_map.tab) was downloaded from its Web site (ftp://ftp.genome.jp/pub/kegg/pathway/, June 2010), which lists the pathways that each annotated gene participates in. Similar to the co-function score described above, the co-pathway score was also computed using the hypergeometric distribution method. We chose the top 1% as the reference set of significant co-pathway gene pairs (139,841 pairs).

Tissue co-expression of gene pairs

We used a global expression data set containing mRNA expression levels for ~20,000 human genes in 32 normal tissues derived from massively parallel signature sequencing [47]. The raw data set documents the abundance values of each short sequence signature tag (with a total of 182,727 tags) in 32 tissues. We mapped these tags to the regions of the human genome (hg18) that encode genes, in both orientations. We obtained 105,512 hits in the gene regions in both orientations and, among them, 68,855 hits in the gene orientation (p-value

10^−2,000), indicating that the tags were able to distinguish the transcribed orientation from the non-transcribed orientation in the genome. Assigning the tags that hit a gene region and orientation to the corresponding gene, we obtained a set of tags for each gene, resulting in a total of 14,516 genes having non-empty tag sets. We summed up the abundance tissue profiles of the tags of a gene to create its raw expression profile.

Furthermore, we computed the statistical significance of a gene being preferentially expressed in a tissue (termed “tissue specificity”) using an approach similar to that of Yu et al.[52], which identified tissue-specific genes from the NCBI Expressed Sequence Tag database. Let e_k(g) be the expression level of gene g in tissue k. The total expression of gene g in all 32 tissues is,

(7)

and the expected total expression of all genes in tissue k is,

(8)

If we randomly throw E(g) darts into 32 areas of sizes E_k, k

1, 2, …32, and each dart has a probability,

(9)

of hitting area k, we would expect to see E(g) q_k darts in this area with variance E(g)q_k(1-q_k). Similarly, if gene g were equally expressed across all tissues, the expected expression level in tissue i would be E(g)q_k with variance E(g)q_k(1-q_k). Thus, we used the corresponding Z-score,

(10)

as the tissue specificity of gene g in tissue k. Accordingly, we defined the tissue co-expression score of genes g and h as,

(11)

We chose the top 1% of these scored gene pairs as the reference set of co-expressed gene pairs to evaluate the PPI scoring approaches. There were 1,053,529 co-expression pairs in this set.

Competing Interests

The authors declare that they have no competing interests.

Authors’ contributions

XY, AW, and JR conceived of the study and participated in its design and coordination. XY collected the data, developed the algorithms, and performed the calculations. AW and XY drafted the original manuscript, which was edited by JR. All authors read and approved the final manuscript.

Supplementary Material

Additional file 1

Aggregated human physical PPI data. This tab-delimited text file contains the aggregated PPI data based on publications collected from nine databases: BIND, BioGRID, DIP, HPRD, IntAct, MINT, MIPS, PDZBase, and Reactome.

Click here for file^{(2.4M, txt)}

Additional file 2

Direct interactions supported by known complexes in PDB. This tab-delimited file contains the protein pairs that have direct contact in at least one protein complex in PDB.

Click here for file^{(6.7K, txt)}

Additional file 3

IDBOS Ranked PPI data. This tab-delimited file contains the aggregated PPI data set with the corresponding IDBOS ranking.

Click here for file^{(4.5M, txt)}

Additional file 4

Occurrence Ranked PPI data. This tab-delimited file contains the aggregated PPI data set with the corresponding occurrence ranking.

Click here for file^{(1.6M, txt)}

Additional file 5

Hypergeometric Ranked PPI data. This tab-delimited file contains the aggregated PPI data set with the corresponding hypergeometric ranking.

Click here for file^{(2.2M, txt)}

Acknowledgements

We thank Drs. Jacob Feala, Vesna Memišević, and Xin Hu for helpful discussions. The authors were supported by the Military Operational Medicine Research Program of the U.S. Army Medical Research and Materiel Command, Ft. Detrick, Maryland, as part of the U.S. Army's Network Science Initiative. The opinions and assertions contained herein are the private views of the authors and are not to be construed as official or as reflecting the views of the U.S. Army or the U.S. Department of Defense. This paper has been approved for public release with unlimited distribution.

References

Rual JF, Venkatesan K, Hao T, Hirozane-Kishikawa T, Dricot A, Li N, Berriz GF, Gibbons FD, Dreze M, Ayivi-Guedehoussou N. et al. Towards a proteome-scale map of the human protein-protein interaction network. Nature. 2005;437(7062):1173–1178. doi: 10.1038/nature04209. [PubMed] [Cross Ref]
Stelzl U, Worm U, Lalowski M, Haenig C, Brembeck FH, Goehler H, Stroedicke M, Zenkner M, Schoenherr A, Koeppen S. et al. A human protein-protein interaction network: a resource for annotating the proteome. Cell. 2005;122(6):957–968. doi: 10.1016/j.cell.2005.08.029. [PubMed] [Cross Ref]
Ewing RM, Chu P, Elisma F, Li H, Taylor P, Climie S, McBroom-Cerajewski L, Robinson MD, O'Connor L, Li M. et al. Large-scale mapping of human protein-protein interactions by mass spectrometry. Mol Syst Biol. 2007;3:89. [PMC free article] [PubMed]
Jeronimo C, Forget D, Bouchard A, Li Q, Chua G, Poitras C, Therien C, Bergeron D, Bourassa S, Greenblatt J. et al. Systematic analysis of the protein interaction network for the human transcription machinery reveals the identity of the 7SK capping enzyme. Mol Cell. 2007;27(2):262–274. doi: 10.1016/j.molcel.2007.06.027. [PubMed] [Cross Ref]
Sowa ME, Bennett EJ, Gygi SP, Harper JW. Defining the human deubiquitinating enzyme interaction landscape. Cell. 2009;138(2):389–403. doi: 10.1016/j.cell.2009.04.042. [PMC free article] [PubMed] [Cross Ref]
Suthram S, Shlomi T, Ruppin E, Sharan R, Ideker T. A direct comparison of protein interaction confidence assignment schemes. BMC Bioinforma. 2006;7:360. doi: 10.1186/1471-2105-7-360. [PMC free article] [PubMed] [Cross Ref]
Schelhorn SE, Mestre J, Albrecht M, Zotenko E. Inferring physical protein contacts from large-scale purification data of protein complexes. Mol Cell Proteomics. 2011;10(6):M110 004929. [PMC free article] [PubMed]
Yu X, Ivanic J, Memisevic V, Wallqvist A, Reifman J. Categorizing biases in high-confidence high-throughput protein-protein interaction data sets. Mol Cell Proteomics. 2011;11:M111 012500. in press. [PubMed]
Wodak SJ, Pu S, Vlasblom J, Seraphin B. Challenges and rewards of interaction proteomics. Mol Cell Proteomics. 2009;8(1):3–18. doi: 10.1074/mcp.R800014-MCP200. [PubMed] [Cross Ref]
Yu H, Braun P, Yildirim MA, Lemmens I, Venkatesan K, Sahalie J, Hirozane-Kishikawa T, Gebreab F, Li N, Simonis N. et al. High-quality binary protein interaction map of the yeast interactome network. Science. 2008;322(5898):104–110. doi: 10.1126/science.1158684. [PMC free article] [PubMed] [Cross Ref]
Hakes L, Pinney JW, Robertson DL, Lovell SC. Protein-protein interaction networks and biology–what's the connection? Nat Biotechnol. 2008;26(1):69–72. doi: 10.1038/nbt0108-69. [PubMed] [Cross Ref]
Pfeiffer T, Hoffmann R. Large-scale assessment of the effect of popularity on the reliability of research. PLoS One. 2009;4(6):e5996. doi: 10.1371/journal.pone.0005996. [PMC free article] [PubMed] [Cross Ref]
Bader GD, Betel D, Hogue CW. BIND: the Biomolecular Interaction Network Database. Nucleic Acids Res. 2003;31(1):248–250. doi: 10.1093/nar/gkg056. [PMC free article] [PubMed] [Cross Ref]
Stark C, Breitkreutz BJ, Reguly T, Boucher L, Breitkreutz A, Tyers M. BioGRID: a general repository for interaction datasets. Nucleic Acids Res. 2006;34(Database issue):D535–539. [PMC free article] [PubMed]
Salwinski L, Miller CS, Smith AJ, Pettit FK, Bowie JU, Eisenberg D. The Database of Interacting Proteins: 2004 update. Nucleic Acids Res. 2004;32(Database issue):D449–451. [PMC free article] [PubMed]
Peri S, Navarro JD, Amanchy R, Kristiansen TZ, Jonnalagadda CK, Surendranath V, Niranjan V, Muthusamy B, Gandhi TK, Gronborg M. et al. Development of human protein reference database as an initial platform for approaching systems biology in humans. Genome Res. 2003;13(10):2363–2371. doi: 10.1101/gr.1680803. [PubMed] [Cross Ref]
Aranda B, Achuthan P, Alam-Faruque Y, Armean I, Bridge A, Derow C, Feuermann M, Ghanbarian AT, Kerrien S, Khadake J. et al. The IntAct molecular interaction database in 2010. Nucleic Acids Res. 2009;38(Database issue):D525–531. [PMC free article] [PubMed]
Chatr-aryamontri A, Ceol A, Palazzi LM, Nardelli G, Schneider MV, Castagnoli L, Cesareni G. MINT: the Molecular INTeraction database. Nucleic Acids Res. 2007;35(Database issue):D572–574. [PubMed]
Pagel P, Kovac S, Oesterheld M, Brauner B, Dunger-Kaltenbach I, Frishman G, Montrone C, Mark P, Stumpflen V, Mewes HW. et al. The MIPS mammalian protein-protein interaction database. Bioinformatics. 2005;21(6):832–834. doi: 10.1093/bioinformatics/bti115. [PubMed] [Cross Ref]
Beuming T, Skrabanek L, Niv MY, Mukherjee P, Weinstein H. PDZBase: a protein-protein interaction database for PDZ-domains. Bioinformatics. 2005;21(6):827–828. doi: 10.1093/bioinformatics/bti098. [PubMed] [Cross Ref]
Vastrik I, D'Eustachio P, Schmidt E, Gopinath G, Croft D, de Bono B, Gillespie M, Jassal B, Lewis S, Matthews L. et al. Reactome: a knowledge base of biologic pathways and processes. Genome Biol. 2007;8(3):R39. doi: 10.1186/gb-2007-8-3-r39. [PMC free article] [PubMed] [Cross Ref]
Yu X, Ivanic J, Wallqvist A, Reifman J. A novel scoring approach for protein co-purification data reveals high interaction specificity. PLoS Comput Biol. 2009;5(9):e1000515. doi: 10.1371/journal.pcbi.1000515. [PMC free article] [PubMed] [Cross Ref]
Hart GT, Lee I, Marcotte ER. A high-accuracy consensus map of yeast protein complexes reveals modular nature of gene essentiality. BMC Bioinforma. 2007;8:236. doi: 10.1186/1471-2105-8-236. [PMC free article] [PubMed] [Cross Ref]
Deane CM, Salwinski L, Xenarios I, Eisenberg D. Protein interactions: two methods for assessment of the reliability of high throughput observations. Mol Cell Proteomics. 2002;1(5):349–356. doi: 10.1074/mcp.M100037-MCP200. [PubMed] [Cross Ref]
Deng M, F Sun, T Chen. Assessment of the reliability of protein-protein interactions and protein function prediction. Pacific Symposium on Biocomputing Pacific Symposium on Biocomputing. 2003;8(140):151–4376. [PubMed]
Goldberg DS, Roth FP. Assessing experimentally derived interactions in a small world. Proc Natl Acad Sci U S A. 2003;100(8):4372–4376. doi: 10.1073/pnas.0735871100. [PubMed] [Cross Ref]
Bossi A, Lehner B. Tissue specificity and the human protein interaction network. Mol Syst Biol. 2009;5:260. [PMC free article] [PubMed]
Gillis J, Pavlidis P. The impact of multifunctional genes on "guilt by association" analysis. PLoS One. 2011;6(2):e17258. doi: 10.1371/journal.pone.0017258. [PMC free article] [PubMed] [Cross Ref]
Wynn RM, Kato M, Machius M, Chuang JL, Li J, Tomchick DR, Chuang DT. Molecular mechanism for regulation of the human mitochondrial branched-chain alpha-ketoacid dehydrogenase complex by phosphorylation. Structure. 2004;12(12):2185–2196. doi: 10.1016/j.str.2004.09.013. [PubMed] [Cross Ref]
Grissom PM, Vaisberg EA, McIntosh JR. Identification of a novel light intermediate chain (D2LIC) for mammalian cytoplasmic dynein 2. Mol Biol Cell. 2002;13(3):817–829. doi: 10.1091/mbc.01-08-0402. [PMC free article] [PubMed] [Cross Ref]
Mikami A, Tynan SH, Hama T, Luby-Phelps K, Saito T, Crandall JE, Besharse JC, Vallee RB. Molecular structure of cytoplasmic dynein 2 and its distribution in neuronal and ciliated cells. J Cell Sci. 2002;115(Pt 24):4801–4808. [PubMed]
Cabello OA, Eliseeva E, He WG, Youssoufian H, Plon SE, Brinkley BR, Belmont JW. Cell cycle-dependent expression and nucleolar localization of hCAP-H. Mol Biol Cell. 2001;12(11):3527–3537. [PMC free article] [PubMed]
Wang S, Zhu G, Chapoval AI, Dong H, Tamada K, Ni J, Chen L. Costimulation of T cells by B7-H2, a B7-like molecule that binds ICOS. Blood. 2000;96(8):2808–2813. [PubMed]
Wang S, Zhu G, Tamada K, Chen L, Bajorath J. Ligand binding sites of inducible costimulator and high avidity mutants with improved function. J Exp Med. 2002;195(8):1033–1041. doi: 10.1084/jem.20011607. [PMC free article] [PubMed] [Cross Ref]
Volz A, Goke R, Lankat-Buttgereit B, Fehmann HC, Bode HP, Goke B. Molecular cloning, functional expression, and signal transduction of the GIP-receptor cloned from a human insulinoma. FEBS Lett. 1995;373(1):23–29. doi: 10.1016/0014-5793(95)01006-Z. [PubMed] [Cross Ref]
Gallwitz B, Witt M, Morys-Wortmann C, Folsch UR, Schmidt WE. GLP-1/GIP chimeric peptides define the structural requirements for specific ligand-receptor interaction of GLP-1. Regul Pept. 1996;63(1):17–22. doi: 10.1016/0167-0115(96)00019-5. [PubMed] [Cross Ref]
Manhart S, Hinke SA, McIntosh CH, Pederson RA, Demuth HU. Structure-function analysis of a series of novel GIP analogues containing different helical length linkers. Biochemistry. 2003;42(10):3081–3088. doi: 10.1021/bi026868e. [PubMed] [Cross Ref]
Yamada Y, Seino Y. Physiology of GIP–a lesson from GIP receptor knockout mice. Horm Metab Res. 2004;36(11–12):771–774. [PubMed]
Tressel T, Thompson R, Zieske LR, Menendez MI, Davis L. Interaction between L-threonine dehydrogenase and aminoacetone synthetase and mechanism of aminoacetone production. J Biol Chem. 1986;261(35):16428–16437. [PubMed]
Ta HX, Holm L. Evaluation of different domain-based methods in protein interaction prediction. Biochem Biophys Res Commun. 2009;390(3):357–362. doi: 10.1016/j.bbrc.2009.09.130. [PubMed] [Cross Ref]
Gupta S, Wallqvist A, Bondugula R, Ivanic J, Reifman J. Unraveling the conundrum of seemingly discordant protein-protein interaction datasets. Conf Proc IEEE Eng Med Biol Soc. 2010;2010:783–786. [PubMed]
Finn RD, Marshall M, Bateman A. iPfam: visualization of protein-protein interactions in PDB at domain and amino acid resolutions. Bioinformatics. 2005;21(3):410–412. doi: 10.1093/bioinformatics/bti011. [PubMed] [Cross Ref]
Kanehisa M, Goto S. KEGG: kyoto encyclopedia of genes and genomes. Nucleic Acids Res. 2000;28(1):27–30. doi: 10.1093/nar/28.1.27. [PMC free article] [PubMed] [Cross Ref]
Zhang X, De la Cruz O, Pinto JM, Nicolae D, Firestein S, Gilad Y. Characterizing the expression of the human olfactory receptor gene family using a novel DNA microarray. Genome Biol. 2007;8(5):R86. doi: 10.1186/gb-2007-8-5-r86. [PMC free article] [PubMed] [Cross Ref]
Goh KI, Cusick ME, Valle D, Childs B, Vidal M, Barabasi AL. The human disease network. Proc Natl Acad Sci U S A. 2007;104(21):8685–8690. doi: 10.1073/pnas.0701361104. [PubMed] [Cross Ref]
Hamosh A, Scott AF, Amberger JS, Bocchini CA, McKusick VA. Online Mendelian Inheritance in Man (OMIM), a knowledgebase of human genes and genetic disorders. Nucleic Acids Res. 2005;33(Database issue):D514–517. [PMC free article] [PubMed]
Jongeneel CV, Delorenzi M, Iseli C, Zhou D, Haudenschild CD, Khrebtukova I, Kuznetsov D, Stevenson BJ, Strausberg RL, Simpson AJ. et al. An atlas of human gene expression from massively parallel signature sequencing (MPSS) Genome Res. 2005;15(7):1007–1014. doi: 10.1101/gr.4041005. [PubMed] [Cross Ref]
Pierre S, Scholich K. Toponomics: studying protein-protein interactions and protein networks in intact tissue. Mol Biosyst. 2010;6(4):641–647. doi: 10.1039/b910653g. [PubMed] [Cross Ref]
Ivanic J, Yu X, Wallqvist A, Reifman J. Influence of protein abundance on high-throughput protein-protein interaction detection. PLoS One. 2009;4(6):e5815. doi: 10.1371/journal.pone.0005815. [PMC free article] [PubMed] [Cross Ref]
Liang S, Liu S, Zhang C, Zhou Y. A simple reference state makes a significant improvement in near-native selections from structurally refined docking decoys. Proteins. 2007;69(2):244–253. doi: 10.1002/prot.21498. [PMC free article] [PubMed] [Cross Ref]
Bateman A, Coin L, Durbin R, Finn RD, Hollich V, Griffiths-Jones S, Khanna A, Marshall M, Moxon S, Sonnhammer EL. et al. The Pfam protein families database. Nucleic Acids Res. 2004;32(Database issue):D138–141. [PMC free article] [PubMed]
Yu X, Lin J, Zack DJ, Qian J. Computational analysis of tissue-specific combinatorial gene regulation: predicting interaction between transcription factors in human tissues. Nucleic Acids Res. 2006;34(17):4925–4936. doi: 10.1093/nar/gkl595. [PubMed] [Cross Ref]

Articles from BMC Bioinformatics are provided here courtesy of
BioMed Central