Initial classification of loci
The HAVANA group divides gene features into different categories of which only the first two (known and novel coding sequence (CDS)) are confidently predicted to be protein-coding genes. The common factor between all annotated gene structures is that they must be supported by transcriptional evidence, through homology to cDNA, expressed sequence tags (ESTs) and/or protein sequences. The following are the gene types first applied to the human chromosome 20 annotation [8
] and later expanded to fully classify the annotation produced for the ENCODE project.
Known genes are identical to human cDNA or protein sequences and identified by a GeneID in Entrez Gene [9
Novel coding sequence
Novel coding sequences have an open reading frame (ORF) and are identical, or have homology, to cDNAs or proteins but do not fall into the above category; these mRNA sequences are submitted to public databases, but they are not yet represented in Entrez Gene or have not yet received an official gene name from the nomenclature committee [10
]. They can also be novel in the sense that they are not yet represented by an mRNA sequence in the species concerned.
Novel transcripts are as above but no ORF can be unambiguously assigned; these can be genuine non-coding genes or they may be partial protein-coding genes supported by limited evidence. They should be supported by at least three ESTs from independent sources (not originating from the same clone identifier).
Putative genes are identical, or have homology, to spliced ESTs but lack a significant ORF and polyA features; these are generally short two or three exon genes or gene fragments.
Pseudogenes (assumes no expressed evidence) have homology to proteins but generally suffer from a disrupted CDS and an active homologous gene can be found at another locus. This category can be further subdivided into processed or unprocessed pseudogenes. Sometimes these entries have an intact CDS or an open but truncated ORF, in which case there is other evidence used (for example genomic polyA stretches at the 3' end) to classify them as a pseudogene.
Transcribed pseudogenes are not currently given a separate tag within GENCODE and are handled by creating a pseudogene object and an overlapping transcript object with the same locus name.
TEC (To be experimentally confirmed)
To be experimentally confirmed (TEC) is used for non-spliced EST clusters that have polyA features. This category has been specifically created for the ENCODE project to highlight regions that could indicate the presence of novel protein coding genes that require experimental validation, either by 5' rapid amplification of cDNA ends (RACE) or RT-PCR to extend the transcripts or by confirming expression of the putatively encoded peptide with specific antibodies.
Artefact gene is used to tag mistakes in the public databases (Ensembl/SwissProt/Trembl). Usually, these arise from high-throughput cDNA sequencing projects, which submit automatic annotation sometimes resulting in erroneous CDSs that are, for example, 3' untranslated regions (UTRs).
GENCODE annotation of the ENCODE regions
The first release of the annotation of the 44 ENCODE regions was frozen on 29 April 2005 and was used in the E-GASP workshop. It contained 416 known loci, 26 novel CDS loci, 82 novel transcript loci, 78 putative loci, 104 processed pseudogenes and 66 unprocessed pseudogenes. The current version (release 02) was frozen on 14 October 2005. It contains 411 known loci, 30 novel CDS loci, 81 novel transcript loci, 83 putative loci, 104 processed pseudogenes and 66 unprocessed pseudogenes. The gene content has changed as a result of the experimental validation (see next section). In total, 2.9% of the nucleotides in the ENCODE regions (both strands considered separately) are covered by annotated exons (1.2% by coding and 1.7% by UTRs and non-coding), and 31% are transcribed (covered by annotated exons or introns).
Multiple transcripts are annotated at any locus where supporting evidence is available. Thus, the 487 compiled GENCODE reference loci set (compiled from coding and experimentally verified loci) corresponds to 2,608 transcripts, of which 1,097 are coding. Of the coding loci (known and novel CDS), 78% have alternative splice forms (86% of the multi-exon gene loci), with an average of 5.7 variants per locus. Of the coding variants, approximately 70% have a complete CDS (the remainder are partial); 54% of the coding loci have alternative CDS, indicating that diversity is lower at the protein level than at the transcript level as a substantial proportion of the alternative splice forms affect only the UTRs. The RNPC2 (RNA-binding region (RNP1, RRM) containing 2) gene has 37 variants, which is the highest number in the ENCODE regions, of which only 6 are annotated as coding.
Experimental verification of GENCODE annotation
The initial HAVANA annotation was submitted for experimental verification (Figure ). First, 5' RACE in 12 different tissues was employed to confirm that annotated coding genes (within both known and novel CDS locus categories) had been extended as far as possible towards the transcriptional start site, to exclude the possibility of additional exons in their 5' UTR and identify a representative full-length transcript for each locus. Of the 420 coding loci tested, 229 RACE products could be sequenced. They supported 5' extensions of 30 loci (extension of the first exon in two-thirds of the cases, new 5' exons in one-third of the cases) and new splice variants (not extending the 5' end) in 50 loci.
Second, RT-PCR in 24 tissues was used for verifying transcript (novel and putative) structures by checking the splice junctions. All 360 splice junctions in the 161 novel and putative transcript loci were tested. Of those tested, 47 loci were validated, consisting of 31 novel and 15 putative transcripts. As expected, the success rate of RT-PCR was higher for the 'novel transcripts' (37%) than for the putative transcripts (19%). Bidirectional RACE was carried out for transcript loci with successfully validated splice junctions. This supported seven loci over their full length but did not extend them.
Third, all annotated non-canonical sites (that is, all introns not conforming to the AG-GT or AG-GC rule) were tested by RT-PCR on 24 tissues. Of the annotated splice sites, 98% are canonical GT-AG and an additional 0.9% are GC-AG. There are 0.2% of AT-AC splice sites, most of them corresponding to canonical U12 introns [11
]. Other non-canonical splice sites occur in the remaining 0.9% of the introns. Among 90 non-canonical splice sites tested by RT-PCR in 24 tissues, 78 reactions were negative, 11 provided other canonical junctions (most of them already annotated in other splice forms), and only 1 was confirmed (CT-TG). The very low level of success of the RT-PCRs on non-canonical splice sites in 24 tissues suggests that these events may be artifactual. As a control, we performed RT-PCR on 24 tissues (see Materials and methods) on 96 randomly selected exon pairs from within the GENCODE annotation. After sequencing of the amplimer, the annotated exon pair was confirmed in 84 cases (87%) in at least one tissue. This is essentially the expected result, given the fact that many alternative splice forms in GENCODE are likely to have a restricted expression pattern, and may not be represented in the 24 tissues tested.
Figure summarizes the process of annotation, experimental validation and reannotation that has occurred since the original release of the GENCODE annotation in April and its current update in October 2005.
Figure 2 Experimental validation of HAVANA annotation. 'Known' and 'Novel_CDS' were submitted to 5' RACE, and 'Novel transcript' and 'Putative' loci were submitted to RT-PCR on all their exon junctions, followed by bi-directional RACE. Several steps of reannotation (more ...)
Assessing completeness of the GENCODE annotation
To examine whether the manual annotation had missed any coding loci, RT-PCR reactions in 24 tissues were also carried out for splice junctions from all those gene objects predicted by a panel of automated gene prediction algorithms before the E-GASP workshop (Geneid [12
], Genescan [13
], Twinscan [14
], SGP [15
], Fgenesh [16
], Exonify [17
], Acembly [18
] Ecgene [19
], Ensembl EST [20
]) that lie outside a HAVANA annotated gene in 13 of the 44 ENCODE regions (corresponding to the training regions for which the annotations were released before the E-GASP predictions submission deadline). Of the 1,215 exon pairs tested, only 14 (1.2%) produced a positive result, 9 of which perfectly predicted exon boundaries and 5 with displaced exon boundaries (8 other positive RT-PCRs were falling in 2 pseudogene loci). Among the 14 positive validated junctions, 8 were new splice forms internal to annotated loci, 4 were new splice forms extending annotated loci, and only 2 were completely inter-genic to any annotation. These results suggest that the GENCODE gene set was relatively complete. It was then updated to include the new splice forms/loci suggested by these experiments.
To further assess the completeness of the GENCODE annotation, we have compared it with other publicly available and widely used human gene sets: RefSeq [21
] and ENSEMBL [22
]. These gene sets were downloaded from the UCSC genome browser in November 2005. Table shows the overlap between these sets and GENCODE by at least one bp: 99% of RefSeq, and 94% of ENSEMBL exons overlap GENCODE exons. In contrast, only 80% and 84% of the GENCODE exons overlap RefSeq and ENSEMBL exons, respectively.
Analysis of RefSeq and ENSEMBL ENCODE annotation compared with GENCODE
Figure illustrates the comparisons at exact exon/intron level. Although the exact agreement between GENCODE on the one hand, and RefSeq and ENSEMBL on the other, is lower than when considering one base overlap, the same trend is observed: 84% (3,361/3,984) of RefSeq and 76% (3,584/4,734) of ENSEMBL exons are included in the GENCODE set, but only about 40% of the GENCODE exons are included in RefSeq or ENSEMBL.
Figure 3 Comparison of GENCODE transcript annotation with RefSeq and ENSEMBL. The exact agreement between GENCODE and RefSeq and GENCODE and ENSEMBL exons, introns, and nucleotides (NT) for the full transcripts or only the coding parts of the transcripts (CDS) (more ...)
As illustrated by Figure , the exact agreement is larger for exons than for introns, which suggests that the disagreements are mostly found at the terminal exons, which is also reflected in the fact that the agreement is also larger for the subset of coding than for the set of all exons. In summary, the comparison shows that GENCODE contains most of the features from RefSeq and ENSEMBL but has more unique exons than the two sets, which is reflected by its high number of alternative splice forms.
Investigation of ENCODE regions that are problematic for automatic annotation
The gene prediction algorithms that performed most successfully in the E-GASP evaluation workshop when compared to the manual annotation were the ones that used alignments of expressed sequences to produce their gene predictions (see [7
] in this issue). However, even the most successful methods of automated gene prediction achieved a maximum sensitivity of 70% at the gene level (where at least one coding transcript exon/intron structure was correctly predicted) and 45% at the transcript level (where all alternatively spliced variants were correctly predicted). There are several reasons for this. Some incidences of missed genes could be explained by the lack of high identity transcript evidence; for example, many of the olfactory receptor genes in ENm009 (Figure ) lack good transcript and protein support [23
]. Another example is the ANKRD43 locus in ENr221, where partial coverage of the gene with human mRNA produces truncated automated predictions. However, cross-species evidence supports an extended protein-coding gene (Figure ). In other cases, predictors fail to make a correct prediction even though a full length transcript with perfect sequence identity is present in the databases (for example, Pairagon at the TRIM22 locus in ENm009; Figure ). There are also examples where the predictions differ from the manual annotation gene structure, even though they use the same supporting evidence, because of problems with automated alignment (for example, Ensembl and Pairagon at the MAP3K1 locus in ENr221; Figure ). A problem that appears to be associated with tandem duplicated gene clusters is the linking together of adjacent loci. The predicted transcript uses consecutive exons from more than one locus, for example for a six exon gene taking exons 1 and 2 from locus A, 3, 4 and 5 from locus B and 6 from locus C. Because the equivalent exons of the different copies of the gene are very similar (often identical), the resulting predicted transcript is an elongated structure usually covering multiple loci (for example, AceView at the HBG1/HBG2 loci in ENm009).
Figure 4 Comparison of GENCODE annotation with automated gene prediction methods. Viewed in Fmap of Acedb. Panel A shows the MAPK1 gene in ENr221. The GENCODE annotated gene structure is represented in green and red, the circled region highlights the different (more ...)
Another observation is that there are predictions that have an identical intron/exon structure to the manual annotation but have a different CDS. In such cases, the CDS has either a 5' extension, that is, completely matches the GENCODE CDS but uses an upstream translation initiation codon (most often non-ATG; for example, AceView at the SEPT8 locus in ENr221 and approximately 41% of AceView have a non-ATG start), or has an entirely different CDS in a different frame. The latter often results in unusual structures, with multi-exon 5' and/or 3' UTRs that are at odds with rules governing re-initiation [24
] and nonsense mediated decay (NMD) [25
] (for example, Pairagon at the AC008937.5 locus in ENr221 and AceView at the IFNAR2 locus in ENm005; Figure ). Many of the predictors suffer from reduced specificity as a result of over-prediction of CDSs at loci where manual annotation does not identify any CDS that can be confidently assigned. These fall into two types; the first includes CDS predicted at pseudogene loci, often where the pseudogene suffers from small but significant disablements (for example, Ensembl at the AC08730.14 locus in ENm009; Figure ); and the second includes the 'rule-breaking' types of CDSs described above (AceView at the AC008937.2 in ENr221). Almost all the predictors (with AceView the notable exception) under-predict coding (and non-coding) splice variants, most predicting one transcript per gene.
GENCODE annotation uses only primary evidence; no predictions or RefSeq entries are used to support gene structures. This has the effect of reducing the risk of propagating any errors that may be present in the databases. The gene set annotated by GENCODE is supported using evidence from all available sources, human and non-human mRNAs, ESTs and proteins. The use of non-human evidence is supported by our analysis of four exons not present in our first pass annotation identified by the UNCOVER algorithm [26
], two of which are only supported by non-human EST evidence. The identification of a rare splice variant in the C16orf35 gene at the alpha globin locus is also facilitated using mouse EST evidence (J Hughes, personal communication). Importantly, manual annotation allows context to be taken into account when making a decision about difficult gene regions, which includes consulting literature and various web resources.