|Home | About | Journals | Submit | Contact Us | Français|
To understand how a cancer precision medicine tumor board (CPM-TB) made choices about return of results.
Observed CPM-TB deliberations and completed in-depth interviews with committee members.
Responding to complex evidence of ambiguous significance, deliberations of the CPM-TB were predicated on analytic validity and clinical utility. Members had concerns both about potential harms due to returning results based on weak evidence and about withholding potentially meaningful results. Group dynamics and the clinical experiences of individual committee members shaped their work.
Both scientific evidence and the social context surrounding deliberations of a CPM-TB influenced decisions about return of results. Subjective elements, while present in any scientific endeavor, may carry more weight in the face of ambiguous findings.
Precision medicine aims to use information about a patient's disease biology, including genomic data, to select individuals’ therapies . While precision medicine shows increasing promise, particularly in oncology, substantial uncertainty surrounds how to interpret large-scale genomic data and determine their clinical value. Interpretation of sequencing data is a complex process, and clinicians who aim to apply this information to treatment decisions need efficient procedures that achieve reliable and accurate interpretations. Review and deliberation by a multidisciplinary group may represent one mechanism to ensure rigorous consideration of genomic data . Toward this end, some institutions convene so-called precision medicine or molecular tumor boards [3–5].
This study describes how one group of scientists and clinicians working with large-scale cancer genomic data reviewed tumor and germline sequencing information, obtained by analyzing DNA from tumor and blood specimens from advanced cancer patients, and deliberated about its significance. The group's objective was to determine which results should be returned to the treating oncologist and potentially to the patient. We examine how this group reviewed evidence and determined when findings should be passed from the laboratory into the clinical setting. We identified a series of uncertainties the group faced as well as the frameworks it employed to address them.
Understanding how somatic mutations develop and function is changing how clinicians diagnose and treat cancers . This knowledge can inform which mutations might be clinically ‘actionable’ or ‘druggable’ and which treatments might be futile or harmful [2,7]. For example, non-small-cell lung cancers with particular alterations in the EGFR gene are responsive to treatment with EGFR tyrosine kinase inhibitors such as erlotinib and gefitinib  and those with specific alterations in the ALK gene are responsive to ALK inhibitors such as crizotinib . On the other hand, some KRAS gene alterations found in colon cancer confer resistance to EGFR antibodies such as cetuximab . Such knowledge is used to tailor treatments, often improving the efficacy of therapy or decreasing the toxicity.
Although sequencing-related care is making rapid progress, challenges remain. Analyzing and interpreting the resulting information for any one patient involves a multistep process, with uncertainties about the validity and reliability of many pieces of information at each step .
First, because hundreds or even thousands of genes (in the case of whole-exome sequencing) are assessed at once, dozens or hundreds of plausibly relevant somatic or germline alterations can be observed for an individual patient. The massive volumes of information generated must be filtered and compared with large clinical cohorts to determine what is useful or even understandable . Second, confounding variables that affect the technical performance of the sequencing assay and its computational bioinformatics, such as insufficient reading or ‘coverage’ of the DNA, ambiguity in repetitive or homologous regions of the genome, or heterogeneity of cells within a tumor, can produce false-positive or false-negative findings. These issues may pose difficulties in assessing the extent to which sequencing results are analytically valid . Third, evidence-based determination of the biological function of an alteration may prove challenging, because some mutations arise infrequently and the molecular pathways affected remain incompletely characterized . This may make it difficult to ascertain if a given mutation is a pathogenic ‘driver’, or merely a ‘passenger’ with no clinical or biological significance. In particular, large-scale clinical sequencing often reveals alterations that have not been documented in the literature, frequently leading to uncertainty in interpreting their biological significance .
Determining the clinical significance of a variant requires gathering information by searching large databases and the scientific literature. With respect to somatic variants, the information uncovered in these searches is used to classify alterations according to how well they: aid in the diagnosis of the cancer type; are prognostic of disease trajectory; and predict which treatments might be useful or ineffective . Germline alterations in cancer risk genes are classified according to the degree to which they are likely to be pathogenic or predict a patient's risk of developing cancer . In instances where the pathogenicity is unknown or the evidence is uncertain, the mutation is classified as a ‘variant of uncertain significance’. The bioinformatics processes and libraries of information necessary to interpret and classify findings may be laborious to maintain; moreover, since knowledge about variants is continually evolving, at any given time these resources may contain errors and gaps .
To interpret somatic sequencing, the results should ideally be compared with the patient's own germline DNA to distinguish somatic from germline alterations [12,17]. Although this approach reduces the incidence of false-positive somatic mutation calls, it also magnifies the complexity of the endeavor as two sets of genomic data must be obtained, analyzed, compared and interpreted. In addition, sequencing germline DNA can reveal alterations related to the risk of a patient or family members developing either cancer (either related or unrelated to the current cancer diagnosis) or other noncancerous conditions. This aspect may raise ethical dilemmas about whether such results, particularly ambiguous results and those that are not clearly actionable, should be returned to the patient or family .
Despite these challenges, the prospect of targeting therapies based on an individual's tumor or germline genomic information continues to push this approach more deeply into clinical settings . As a result, numerous ongoing studies seek to understand how the clinical integration of genomic sequencing is currently proceeding and how it can be improved in the future [20–23]. In this report we look at an early component of one such study (termed CanSeq) conducted at the Dana-Farber Cancer Institute (DFCI), Brigham and Women's Hospital (BWH) and the Broad Institute .
The CanSeq project had three overarching aims: to implement and determine the feasibility of using whole-exome sequencing (WES) in the clinical care of patients with advanced lung and colorectal adenocarcinoma; to develop a strategy and framework for interpreting WES results and returning them to treating oncologists and patients; and to understand the impact of WES data on patients, providers and institutional procedures and practices. To review and evaluate WES findings for use in clinical care, CanSeq established a cancer precision medicine tumor board (CPM-TB).
The function of CPM-TB was to identify, through WES, specific alterations in the DNA of cancer cells that would yield clinical insights into a patient's colorectal or lung cancer diagnosis, prognosis or treatment. In addition, it sought to determine which germline alterations were biologically significant or clinically actionable. The primary indication for sequencing was to identify somatic alterations that might aid in the selection of targeted anticancer therapies. Typically these treatments involved drugs already approved by the US FDA or therapies available in the context of a clinical research study. The CPM-TB advised the study investigators about which somatic and germline results should be transmitted to the treating oncologist, but as specified in the study protocol, the CPM-TB did not communicate results directly to the patient. They also advised on the nature of the communication to the oncologist. In this regard the work of CPM-TB was unique because many genomic tumor boards consider treatment choices in light of the results of the DNA sequencing after they have been reported to the treating clinician .
For every patient enrolled in CanSeq, bioinformaticians assembled the sequencing data and a curation team of geneticists, oncologists, pathologists and genetic counselors prepared the results for review by CPM-TB. CPM-TB had 28 members including pathologists, geneticists, genetic counselors, oncologists, genomic scientists, ethicists and bioinformaticians. Individuals were eligible for CPM-TB participation if they were on the staff of one of the two hospitals involved in this project, if they had experience in one of the key areas required (e.g., pathology, bioinformatics) and if they were willing to participate in study activities. In contrast to the typical tumor board, the treating oncologist did not attend CPM-TB meetings because the genomic data presented at CPM-TB had not yet been confirmed in a Clinical Laboratory Improvement Amendments-approved laboratory and therefore could not yet be used for clinical management. For the majority of cases reviewed, the CPM-TB members did not know the identity of patient or the treating oncologist; they did know, however, that the cases were being treated by a clinician on staff at the DFCI. Figure 1 illustrates the steps in the curation and review process. The DFCI institutional review board approved both the study presented here as well as the parent CanSeq study, and all CPM-TB members provided written informed consent for this sub-study.
CPM-TB met twice a month and at each meeting reviewed results for a set of patient cases. Prior to each CPM-TB meeting, four committee members (two germline and two somatic experts) were assigned to perform secondary reviews of the curation team's evidence with a focus on the annotation of the literature and the interpretation of the diagnostic, prognostic and predictive significance of an alteration. The secondary reviewers then reported their assessments back to CPM-TB with recommendations for return of both the somatic and germline findings to a patient's treating oncologist. Each meeting included the curation team, the secondary reviewers and other CPM-TB members. A case discussion began with a presentation of the WES findings, the quality of the sequencing results and the curation, and preliminary recommendations from the curation team and the secondary reviewers about the clinical significance of the findings. Over the course of the first year of meetings, CPM-TB reviewed sequencing results for 40 patients who collectively represented 204 somatic and 166 germline findings (143 and 15 returned, respectively).
The main source of data for this study was observation of 21 CPM-TB meetings held over 18 months. The primary researcher (SMcG) attended six meetings in-person and the remainder through conference call but was not a member of the CPM-TB; one or more of the co-authors participated in all of the 21 meeting as members. The primary researcher also conducted semi-structured telephone interviews with seven CPM-TB members who represented the professional make-up of the committee including one pathologist, three oncologists, two geneticists and a genetic counselor.
The purpose of these interviews was to elicit members’ views about CPM-TB. Questions covered their reactions to the deliberations about the cases, the value of CPM-TB for cancer care, challenges they encountered doing secondary reviews and their assessment of how CPM-TB weighed evidence. The meetings and interviews were audiotaped and transcribed verbatim. A third source of data was the case materials provided to CPM-TB members at each meeting.
Since the overarching CanSeq study was designed as a pilot implementation of WES in oncology practice, we used an iterative process in order to reflect upon and improve WES implementation. The observation study was part of that process. Although the iterative process and presentation of interim findings to board member participants may have influenced their behavior, we acknowledge this possibility and believe that the benefit of reflection and the opportunity for process improvement was essential given the clinical context. Interim findings were reported to the CPM-TB at two meetings – one at the mid-point to describe observations about the CPM-TB review of somatic alterations and the second toward the end of the study period describing observations about review of germline findings and the process overall. Members responded to the observations and discussed their interpretations and implications for their work going forward. This process of reporting back to the CPM-TB is consistent with the qualitative technique of member checking by allowing the group under study to comment on the validity of the findings . The transcripts of these discussions provided a fourth source of data and were included in the analysis of meeting transcripts with a focus on the CPM-TB members’ reactions to the observations.
The analysis for this report examined the content of meeting deliberations and members’ responses to interview questions. The analysis of the meeting transcripts focused on how the CPM-TB: appraised the sequencing information; determined the meaning of the information and assessed the clinical value of the sequencing results; and shaped its deliberations and recommendations for returning WES results to the treating oncologist. Of particular interest were the explicit and implicit values that informed the CPM-TB's decisions, including members’ presumptions about what information should be shared or not, and how they addressed the uncertainties inherent in the endeavor.
The analysis of the meeting transcripts employed a deductive approach by coding for predefined topics of interest and an inductive approach by identifying emergent topics . Coding began with a list of predefined codes (e.g., the genetic variants discussed at each meeting, the predictive, prognostic or diagnostic implications of the findings). This list was supplemented with codes that best reflected key concepts that emerged from a reading of the transcripts. Two coders (SM and a research assistant) independently coded the each transcript following defined codes using Atlas ti. They compared their coding, revised the code list and then recoded the transcripts. In the final stage, the codes were summarized by grouping repeating concepts to identify themes or ideas. SM coded the interview transcripts separately following a similar iterative process. Themes identified from the interviews were compared with those that emerged from the meeting transcripts. Here we present the major themes identified in both the meeting and interview transcripts. Quotations below are identified by a letter indicating the transcript source (meeting [M] and interview [I]) and the number of each transcript.
Four overarching themes characterized the CPM-TB's deliberations. Firstly, the CPM-TB grappled with making sense of complex and ambiguous evidence in as rigorous a manner as possible. Secondly, given that its purpose was to determine which results should be returned to the treating oncologist, the committee sought a framework of principles to guide its choices in the face of uncertain evidence. Thirdly, members’ expectations about the consequences of the returned results for the patient, family and treating oncologist influenced their decisions. Fourthly, despite efforts to maintain scientific rigor throughout this process, social context shaped the committee's decisions. We describe each of these themes below and provide text from meeting transcripts and interviews with members to illustrate our points.
Given the complexities inherent in interpreting large-scale sequencing data, the CPM-TB confronted the full spectrum of challenges summarized above. Members sometimes raised concerns about whether the results revealed real alterations or were technical artifacts. They frequently struggled to interpret the biological significance of identified alterations. Ambiguous evidence made many decisions difficult and required the committee to come to terms with uncertainty. For example, after a discussion of a germline finding, one committee member noted:
“I think we're living in that middle gray space.” – (M1)
“The reality is that it's highly uncertain information.” – (M20)
One way the committee sought to deal with ambiguity was by seeking reassurances about the analytical rigor behind these research results. Meetings often began with technical questions, in part to screen out false-positive findings and to assess the accuracy and reliability of the results. For example, members asked if evidence about a particular alteration was distorted by admixed benign cells within the tumor sample. The answer to this question, however, was never a simple one; tumor samples can be heterogeneous or have sub-clones and, further, can have concurrent copy number alterations in parallel with sequence variants, thereby complicating the analysis and interpretation of the DNA sequence and creating difficulties in determining if the results are, in one member's words, a ‘real event’. In looking back over the committee's early work, one member recalled the challenge:
“…(CPM-TB) spent quite a bit of time on … what kind of thing was real – ‘how real was the real’ – and then what does ‘the as real as we can tell it's real’ mean, both of which had uncertainty built into them … we spent a lot of time, especially in the beginning, talking about that.” – (M20)
Following discussion about the analytic validity of an alteration, CPM-TB considered the evidence about the clinical utility of the identified alteration as summarized by the Curation Team and the secondary reviewers. During these discussions, the reviewers sometimes remarked that a scientific report about an alteration or a database cataloging alterations could paradoxically complicate the process of determining if an alteration was clinically significant. In some instances they had difficulty assessing the quality of the science in a published report – or indeed found the science to be flawed. In other instances they worried about publishing bias inherent in most scientific literature. For example, a CPM-TB member voiced his concern that published reports may ‘skew’ findings to make an alteration look more important than it might be because “our literature is tilted toward positive findings”. (I6)
Finally, because sequencing can reveal novel alterations, the committee sometimes found it difficult to define the function of the mutation. Specifically, challenges arose regarding categorizing alterations as ‘activating’ (i.e., potentially driving tumorigenesis), inactivating or passenger events with no effect on protein function. Despite the substantial expertise on the board, there were instances when members simply had to acknowledge that they did not know the answer, as one member commented in the course of discussing a somatic alteration:
“So, there may be splice experts who know more about this kind of stuff, but because of that, I could not figure out one way or the other whether this thing would actually have an activating or inactivating or no function at all.” – (M7)
Adding to the challenge of determining the significance of a finding was a question about the scope of the committee's focus. At the outset, the CPM-TB considered each alteration as a singular event. However, the reviewers understood that the significance of any genetic alteration could be a function of other alterations and a patient's broader clinical context. Over time, CPM-TB members raised questions about the best point of focus. For instance, a committee member asked during a discussion about a somatic finding:
“…so, when we decide these things, are you deciding it in the context of the whole case or each genomic abnormality, so it could be decided as a separate entity? In other words, are we supposed to make a decision about PIK3R1 in the absence of what else is going on in the case?” – (M7)
Technical questions became less prevalent over the course of the project. Eventually, CPM-TB members placed greater emphasis on interpreting the significance of the results and deliberations about the pros and cons of return. There were three reasons for this shift. First, questions about the sequencing process from members who did not have in-depth knowledge about WES at the outset diminished in frequency as they acquired familiarity. In addition, members quickly gained confidence in the skills of the Curation team and the analytic validity of the findings. Those with less experience in genomics recognized that they did not fully grasp the nuances of interpreting the complex data, but trusted the Curation team and others with expertise in bioinformatics and genetics:
“I think probably two or three people in the room – maybe four people in the room – have the statistical sophistication to really understand where things could be falling apart, but that's where the trust comes in.” – (I6)
Also, the large quantity of alterations to be reviewed led the committee to seek ways to streamline the review process, which freed them to focus on the interpretation of the evidence.
At each meeting, members wrestled with choices about the return of results in light of the complexity and ambiguity of the evidence they confronted. Some members found certain choices uncomfortable and challenging. As one observed about the review process at meetings:
“…what I had difficulty with was really knowing how I was supposed to decide on what to report or what not to report.” – (I1)
They also understood that they might incorrectly choose to return or withhold a finding, and as a result, some reviews sparked extensive debate. A CPM-TB member commented that it was difficult to know if results should be returned because many of the findings they reviewed were of questionable significance.
“…the vast majority of things that we are reviewing and discussing are things where the reliability of the significance of any of these findings is highly in question to begin with … and that's why they sometimes spur pretty large discussions.” – (I7)
In response CPM-TB sought to create a rubric or define a set of principles to guide their decisions; members often referred to this as their ‘philosophy’. Given the large number of alterations identified, the committee hoped that a decision framework would expedite the review process by curtailing deliberations. During a discussion about germline findings, a committee member observed:
“We are going to have to come to some decision about a philosophy that we can live with as we go forward…we are going to have to come to some statement so that we can not only expedite this but that we all can be sure we're comfortable with the way we choose to report things out.” – (M3)
A question that emerged early in the process was whether the CPM-TB should simply return all results (somatic and germline), regardless of how confident the members were about the quality of the evidence for an alteration. For example, one member listed out the reporting options for somatic findings:
“…but there is everything from only reporting the most restrictive, where we're clear that there is already an existing drug. There is the possibility of reporting where we think there are drugs coming. There is a possibility of reporting everything.” – (M3)
The committee sought to define criteria for determining what should be returned. In the case of somatic alterations, criteria were based on scientific evidence about clinical actionability. For example, some members proposed that the committee should only return alterations for which they could put together a ‘legit story’ such as alterations known to be responsive to an FDA-approved therapy, specified as an eligibility criterion for a drug trial or anticipated to be responsive to drugs in the ‘development pipeline’. For germline findings, most believed they should only return those that were understood to be, at a minimum, likely pathogenic.
Over the course of the year, members commented that it appeared that the committee applied different thresholds of certainty for determining the return of somatic and germline alterations:
“So, we're using…a much higher bar, a threshold for germline returns compared with somatic things.” – (M3)
Similarly, another remarked that the committee had different thresholds for germline findings of cancer-related genes than for other genes:
“…it speaks to the overall idea that we have a slightly lower threshold for any cancer gene in the germline than for any noncancer gene.” – (M17)
Committee members ventured explanations for the different thresholds for somatic and germline findings. For somatic findings, some said that they did not want to miss reporting a somatic alteration that would be potentially important for determining treatment choices:
“Who knows what's going to come up in the family and what's going to happen in the future.” – (M4)
By contrast, some members believed the decision to return germline findings should be more conservative because they were incidental findings and not the principal focus of the analysis. They wanted a higher burden of affirmative evidence that a germline alteration was pathogenic:
“…philosophically when something is an incidental pickup and it's of uncertain significance, it seems to me that the priors that it's meaningful in the germline are pretty darn low. So, in general, my philosophy has become don't return unless there's pretty good evidence that it's actually pathogenic mutation in a gene that we should care about.” – (M4)
Others noted that they wanted to be sure that the evidence was strong before returning incidental findings that could have potentially serious implications for patients and/or family members:
“I think the germline is more complex just because it ripples out to family members, to risk for family members, to carrier status, to the 30,000 genes that we're not so interested in looking at. The germline is just such a morass … So, to return something that we're saying this might predispose you and/or family members to cancer, and even more so when we're dealing with genes that are outside the cancer realm entirely, we better be really sure that those variants are meaningful.” – (I3)
A decision framework emerged as a result of their first year of reviews. While allowing for determinations on a ‘gene by gene’ basis (M20), the guidelines were to return germline findings only if they were pathogenic or likely pathogenic. Return of somatic alterations was broader, returning many of the somatic findings, even many of uncertain significance even if they had a plausible causal link based on preclinical or theoretical evidence.
Beyond arguments about the quality of the evidence, CPM-TB members were concerned about potential harms resulting from either erring on the side of ‘over-disclosure’ (what one committee member referred to as ‘commission’) or erring on the side of withholding of results that should have been returned (‘omission’) (Box 1).
The committee members had four worries related to commission. Firstly, members worried that returning an alteration based on uncertain evidence might lead a clinician to make a faulty decision to change a patient's treatment, potentially harming a patient by substituting a less effective or poorly understood treatment in place of one already in use, or in the case of a somatic alteration:
“…actually cause harm by putting people on something that has so little chance of working when you could have put them on something else.” – (M3)
Another member worried, during a discussion about a germline alteration, that a particular pharmacogenomics finding, based on what he called ‘lousy data’, might lead the clinician to lower the dose of a patient's regular medication:
“We could actually hurt a patient because if somebody looks at that lousy data and says I'm now going to give (a lower dose) … we might have hurt the patient because there might not have been a reason to give lower than the standard dose.” – (M2)
Secondly, some worried that returning certain alterations might cause unnecessary psychological harm to the patient or family. For example, returning a somatic alteration in BRCA2 might be provocative and lead the patient or family to incorrectly conclude that they had a heritable increased risk of breast cancer (which might be true of a germline but not a somatic alteration). They worried that just hearing about this gene, which had been covered so widely in the lay media, would lead to unnecessary worry for this patient or his/her family, despite being told that it was a somatic finding and did not indicate a heritable risk of breast cancer.
“I was very reluctant to return it because I think BRCA2 is kind of this charged gene and if you tell people that they have a BRCA2 alteration, they're likely to misunderstand. (They will) think they've got a hereditary cancer risk syndrome and, as I wrote in my note, I know I'm being paternalistic about it, but I wasn't sure if we were ready to go down that pathway on this one.” – (M4)
A related concern came up when the committee discussed a somatic alteration that suggested a poor prognosis for a patient but the evidence about the prognostic implications of this alteration was limited. In this case, a reviewer argued against returning this information to the treating oncologist because it was “based on softer, more theoretical kind of evidence”. He added, “That seems to me like a pretty harsh thing to tell somebody on the basis of soft evidence.” – (M5)
Some committee members countered these fears by arguing that most of the clinicians who received reports will be able to interpret the findings accurately and communicate them appropriately to the patient and family. During a review of a somatic finding, one added that he would personally want this information if he were the clinician:
“I should … trust that the staff here are capable of sorting out the sort of things that we know are genuinely important events molecularly and those that aren’t…” – (M4)
A third concern was returning information that a clinician, patient or family could not use. For example, the committee reviewed a germline finding in a patient with colorectal cancer that was potentially linked to risk of hereditary neuroblastoma, but the evidence suggested that in this case it was likely a variant of unknown significance. The ambiguous result led a committee member to wonder:
“So, let's say you find out that you do have a risk of having children with neuroblastoma, the question's still what's actionable about that. So, should they not get pregnant? Dad's dying now of colon cancer and now their kid isn't supposed to have a kid? I just don't know what – the hard part is what is actionable about it?” – (M1)
Fourthly, during a review meeting, several members were concerned that releasing a lot of information of uncertain value might overwhelm the treating clinicians:
“I can understand you're going to get lots of people saying my God; this is way too much.” – (M11)
Or some worried that receiving uncertain information would lead patients to ignore or even ‘oppose’ future genetic information:
“I think if every patient who walks through the door finds out they might be at risk for four or five genomic syndromes, there's going to be quite a bit of opposition to the projects going forward in society.” – (M1)
Concerns about omission or failure to return results fell into three categories. Among the more common worries was missing the chance to inform family members about a serious condition. One reviewer described her worry about failing to tell a family about a germline alteration that might be linked to a rare neurodegenerative disease:
“I don't think it's the strongest evidence but if we know it, I feel it would be too bad not to disclose it.” – (M3)
Another concern about withholding results was causing a patient to miss the opportunity to enroll in trial for a drug in development that would target the somatic alteration found in the patient's tumor. An oncologist explained:
“Yeah, down the road this particular mutation could predict for therapeutics that are currently in development…” – (M17)
Related to concerns about omission was the argument to return all somatic findings because knowledge was changing so rapidly that an alteration identified as uncertain today might be important for treatment choices in the near future:
“…we're at this point where information changes rapidly so if you are going to tell people that somebody has had their whole genome analyzed but only report back to them two things, then next year something might maybe be revealed to be important for that patient. And we haven't disclosed it, so that just sort of seems – that concerns me more than not giving them back.” – (I1)
Most of the CPM-TB discussions focused on evidentiary concerns like analytic validity and clinical utility, but when it came time to make decisions about return of a result other questions emerged. The committee worked hard to maintain a thoughtful and rigorous approach but often there was no obviously correct answer. In these instances members turned to their personal experiences and beliefs to guide their choices. We describe four examples of these instances.
Firstly, as they discussed return of uncertain findings of a somatic alteration, some committee members drew on their own experiences as clinicians, stating “if I were the doc taking care of this patient, that's what I'd want” – (M4). In fact another committee member said that he found it frustrating to receive what he believed were reports that were too limited in scope:
“…If you don't return all of the data that you have available to you, you're really kind of putting people who want to really understand the tumor in a difficult position. That's just my personal opinion.” – (M11)
Secondly, the professional background of committee members shaped committee discussions and decisions. While several interview respondents commented that the professional make-up of the committee was ‘well-rounded’ (I3 and I5) one observed that predominance of individuals focused on genomics made for an overly enthusiastic emphasis on the potential benefits of this information.
Two respondents suggested that:
“…I think we need a bunch of guys who are not really all that interested in genomics, sitting on the judge panel, hearing the evidence, and we need an impartial jury.” – (I2)
In a similar vein, another noted:
“Again, I think there's a bit of hyper-enthusiasm about everything's going to connect to cancer somehow here and everything is going to be mechanistically important as opposed to this is likely random noise.” – (I6)
Some committee members remarked on differences in the predilection to return results. For example, during the review of a somatic alteration, one member observed that the oncologists on the committee appeared to favor a more liberal return of results compared with individuals with different training:
“…I'm actually glad that (name) is here so that we get two points of view because sometimes I think reflexively it's easy for a medical oncologist to just say ‘Oh, just give me everything’.” – (M4)
Thirdly, features of the group process also shaped decisions. For example, one reviewer commented that she was aware that what she emphasized in her presentation to the committee could influence the choices of her fellow committee members. She suggested that this effect could be greater for committee members who were less experienced and less able to critically evaluate her analysis:
“…depending on how you phrase everything, you can kind of color the discussion pretty significantly in favor of returning it or against returning it…I felt like I had to choose my words kind of carefully as far as how I interpreted it for the group.” – (I7)
A related observation was about how “a vocal few can really influence the group as a whole” – (I7). Another committee member during his interview noted that some discussions were limited to a small subset of the full committee and those individuals who voiced their opinions most often had the greatest influence on decisions:
“I'm a little perplexed by sometimes how this roomful comes down to a handful of people, and it's like a pretty small number that's making some of these decisions. I mean everybody's there and everybody has the opportunity to make the decision.” – (I3)
Fourthly, the committee was aware of the unique nature of the CPM-TB because they were a translational bridge between the laboratory and the clinical setting. Their role, unlike the typical tumor board, required that they determine which results would be moved out of the research context and into the clinical context. The uncertain nature of the information they were asked to evaluate and potentially pass on made this concern even more acute:
“…that goes back to the philosophical divide and I think it's interrelated with the question about trusting the oncologist that the information is going back to … I think the people who are more inclined to return things that are a little more uncertain have a higher level of trust of the providers.” – (I3)
The work of CPM-TB straddled the laboratory, preclinical and clinical domains. As such, the committee functioned as a gatekeeper, determining which information moved from the realm of science into that of clinical care. By showing how this multidisciplinary group of biomedical scientists and clinicians worked through the rules of evidence, this observational study offers a view into a ‘black box’ within which scientific data are translated into clinical evidence .
Despite the aim to arrive at clear and specific clinical end points, CPM-TB members confronted features of their undertaking that made their deliberations about the meaning and utility of the sequencing results challenging. This finding is consistent with the results of Klitzman et al.'s  study of scientists working in genomic research. He reported that the researchers in his study, like members of the CPM-TB, confronted uncertainties and found themselves making ‘judgment calls’. CPM-TB members were acutely aware of the dangers associated with withholding information that could be useful or releasing information that could be harmful.
Our findings are consistent with other studies that have shown how closely interpretation of genomic findings is intertwined with the social context of the biomedical enterprise . Subjectivities and values came into play when CPM-TB scientists and clinicians were faced with the need to simultaneously evaluate the technical and scientific features of the evidence with the clinical, social and ethical implications for a patient and his or her family . The influence of these subjectivities may have been accentuated in the case of this expert committee given the ‘fuzziness’ of the evidence they had to evaluate. Our observations revealed several instances of these subjectivities. The apparent preferences arising from professional training and personal experiences were concordant with Pagliari and Grimshaw's  observations about how professional role and status influenced the discussions and decisions of a multidisciplinary group charged with crafting clinical practice guidelines. Similarly, in examining how members of institutional review boards justified their decisions, Stark ) found that personal and professional experiences often were more influential than matters of fact.
Our study illustrates the challenges that scientists and clinicians face when a new innovation or technology is introduced into medical care. Uncertainty about data and outcomes can be greatest when an innovation is first introduced. The uncertainties associated with the current state of precision medicine can be uncomfortable for all involved. It may be particularly vexing for oncologists or patients who may lack experience with genomic sequencing results or knowledge to fully understand their implications [24,32].
Particularly in the early stages of integrating genomic sequencing into clinical practice, expert committees such as the CPM-TB might be one approach to aiding decisions about return of results . However, as this study and others show, review by a multidisciplinary committee does not always make evidence easier to interpret or limit subjective interpretation . In addition, precision medicine tumor boards such as the CPM-TB may be difficult to replicate in smaller centers where there may be fewer individuals with the requisite expertise (e.g., bioinformatics, genome science) or in centers where experts cannot devote considerable time to interpretation.
Working at the cutting edge of precision medicine in oncology, CPM-TB had to find a way to process massive volumes of complex data in a timely and rigorous manner. It sought to craft procedures and define rules for interpreting the analytic validity and clinical utility of what were often ambiguous results. The committee felt urgency to create heuristics to make the work more efficient and provide guidance to help it work through uncertainties. This imperative might be especially forceful in genomics, where volumes of data are large, interpretation is labor intensive and the associated uncertainties are substantial.
We should note the limitations of this study that are inherent to our methodological approach. For this study, we carried out in-depth observations of one precision tumor board and as such our findings cannot be generalized to all tumor boards. CPM-TB meetings were the primary focus of this analysis; we interviewed only a subset of the committee membership. Interviewees were selected to reflect the range of professional disciplines among committee members but it is possible that the members we did not interview might not share the views of those who were invited to participate. Finally, reporting observations to the CPM-TB at the study mid-point may have influenced the work of committee members; the degree to which the observations substantially altered their approach and the challenges they encountered is difficult to assess. However, we believe that the benefit of reflection and the opportunity for process improvement was essential given the clinical context in which the study occurred.
Given the challenges inherent in making decisions about the validity and utility of evidence, especially at the early stages of innovation, it is important to understand what values are applied as an evidence base is built and practice guidelines are established. While the creation of an evidence base may appear to be a technical process, it is clear from our observations that values become embedded throughout the process.
Researchers and clinicians need to be careful to not create rigid heuristics too quickly, because as they do so they may erroneously emphasize or de-emphasize evidence that would not likely be revisited and corrected. Clearly articulating principles early in the process can help to ensure that choices are consistent with desired values. Judgments applied early in the process may be less obvious to observers at a later time and therefore more difficult to extricate. As heuristics are later created and the science is settled , precedent may shape guidelines for return of results that may be erroneously restrictive or liberal.
As the knowledge of genomics evolves, precision medicine in cancer care will become more evidence-based. Guidelines for the interpretation and application of genomic information will evolve, potentially diminishing the need for subjectivities and for group deliberation. However, careful validation and expansion of published medical evidence are needed before quality interpretation of genomic results can be rendered by highly automated rule-based systems.
In summary, while weighing evidence is central to the practice of science and medicine , CPM-TB's efforts illustrated a paradox in the term ‘precision medicine’. The term suggests surgical exactitude but the interpretive procedures inherent in the practice remain ‘fuzzy’. Considerable uncertainty is to be expected early in the course of any scientific endeavor, before the research methods and interpretation become rule-based, evidence accumulates and the resulting knowledge becomes codified through replication and consensus. But even as the field matures, the move toward making inferences about individuals based on their unique genomic data suggests that uncertainty will always be part of the science and practice of precision medicine.
Co-authors S Joffe, SW Gray, J Garber, N Lindeman, LM Sholl, N Wagle, EM Van Allen, PA Jänne and LA Garraway served as members of CPM-TB and of the Executive Committee of the CanSeq project. The authors thank M Banerjee for her work coding the study transcripts.
Financial & competing interests disclosure
The research reported in this publication was supported by National Human Genome Research Institute U01HG006492 and U01 HG007303. SW Gray was also supported by the American Cancer Society (120529-MRSG-11-006-01-CPPB). SA McGraw was supported through a subcontract with The Dana-Farber Cancer Institute under NHGRI grant numbers U01HG006492 and U01 HG007303. J Garber has consulting or advisory roles for Helix and Biogen and receives research funding from Myriad Genetics and Ambry Genetics. A family member has a consulting role for Novartis Oncology, SV Life Sciences and Pfizer and receives research support from Novartis Oncology. PA Jänne receives post-marketing royalties from DFCI-owned intellectual property on EGFR mutation testing licensed to Lab Corp. EM Van Allen is a stockholder in Syapse, a consultant/advisor for Roche, Third Rock Ventures, and Takeda, and receives sponsored research support from Bristol-Myers Squibb. N Wagle is a stockholder in Foundation Medicine, a consultant/advisor for Novartis, and receives sponsored research support from Novartis and Pfizer. LA Garraway is a consultant for Foundation Medicine, Novartis, Boehringer Ingelheim, Third Rock and Eli Lilly, an equity holder in Foundation Medicine, and a member of the Scientific Advisory Board at Warp Drive. LA Garraway receives sponsored research support from Novartis, Astellas, BMS and Merck. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.
No writing assistance was utilized in the production of this manuscript.
Ethical conduct of research
The Dana-Farber Cancer Institute institutional review board approved the study and all CPM-TB members provided written informed consent for this ethnographic sub-study. The authors state that they have obtained appropriate institutional review board approval or have followed the principles outlined in the Declaration of Helsinki for all human or animal experimental investigations. In addition, for investigations involving human subjects, informed consent has been obtained from the participants involved.
This work is licensed under the Attribution-NonCommercial-NoDerivatives 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/4.0/
Papers of special note have been highlighted as: • of interest; •• of considerable interest