Outcomes of Sipuleucel-T Therapy

Appreciate the opportunity to review this report "outcomes of Sipuleucel T Therapy."
The technology report is timely, comprehensive and well written.
The 3 questions addressed are very relevant and impact the practioner's use of the product.

Table 6 was kept in.
Table 7 kept in as is.
Mentioned protocol on late complications.

I would add to the discussion of the immune monitoring data that the study design (in addition to its other flaws) prevents assessing the impact of the PA2024 on the results. The authors conclude that three injections of antigen-presenting cells pulsed with a tumor antigen improves survival; however, no role for the tumor antigen has been demonstrated. A better design would have been to administer autologous antigen-presenting cells stimulated in vitro with GM-CSF alone. Such treatment could well have resulted in an increase in CD54 expression both in vitro and in vivo and accounted for the observed effects. A difference between adoptive transfer of cells treated with GM-CSF alone and cells treated with GM-CSF plus PA2024 would have tested the question of whether the antigen plays any role at all in the survival increase.

A final point I would like to raise is the unexplainable nature of the survival effect. Many cancer treatments have been shown to have effects on the tumor, including partial responses or delay in tumor progression, but have NOT made an impact on overall survival. In general, affecting overall survival has always been the more difficult endpoint to influence with a new intervention. It is extremely unexpected for an intervention to make an impact on overall survival without any discernible effect on the tumor itself. Imagine the skepticism that would accompany a claim that a new antibiotic improved survival in tuberculosis without affecting the organism in any measureable way.

Further sentences regarding alternative analyses were added.

It is not clear from protocol documents regarding the reporting of adverse events proximate to the time of frozen salvage product. "Late" events are only reportable if "attributable" to sipuleucel-T.

In the report, under key question 3, the following is stated: "Three randomized clinical trials of sipuleucel-T are consistent with longer overall survival in patients meeting the FDA-labeled indication. This conclusion is tempered by consideration of a trial design with inherent potential for confounding due to frozen salvage product and post-progression variation in treatment. The quantity of benefit of sipuleucel-T is less certain because of these issues."

I would like to specifically address two points with this. The first has to do with the salvage product and the second has to do with post-progression variation in treatment.

The main difference in the salvage product (used in the control arm after cross-over) compared with the experimental arm product is that the salvage product was frozen. While this may have some impact on the efficacy of the salvage product (a currently untested hypothesis), that is not relevant to the point in issue. Generally a cross over design causes an under-estimation of the true therapeutic effect on overall survival. One would have to argue that the salvage product actually harmed patients to make the above case outlined in the Technology Assessment. This is not only very unlikely, but there is no biologic rationale to support this. First, the salvage product met the same product release criteria as the product used in the experimental arm. Second, the median predicted survival of the control arm based on a validated nomogram was the same (21 months) as the actual overall survival. If there had been substantial harm caused by the salvage product this should have negatively impacted the median overall survival for the entire control arm, something not seen. Third, as reported by Dan George in the MEDCAC meeting, there is data suggesting improved outcomes for the subset of patients in the control arm treated with salvage product compared with other patients even after accounting for clinically relevant variables (to be presented at the ASCO sponsored Genitourinary Cancers Symposium in 2011). While these types of subgroup analyses are subject to bias, all the available data suggest that there is no basis for the argument that the salvage product causes harm. Furthermore, as mentioned above there is no biologic rationale for harm caused by the salvage product, indeed available data and biologic rationale suggest a possible underestimation of the true efficacy impact on overall survival seen in this trial.
The argument that post-progression treatment led to the improvements seen in overall survival is also without merit. The data available in this study is as balanced in post-progression treatment as is likely to be seen in a trial in patients with advanced cancer. It would not be feasible, and indeed may be unethical, to prospectively determine post-study cancer therapies for cancer patients. There are many changes over the course of the disease course in cancer that may dictate what the appropriate next therapy should be. In addition, patients may decline or discontinue early some therapies due to side effects. About half of advanced prostate cancer patients never get chemotherapy. Fortunately for the interpretation of the results, there was only one agent shown to impact overall survival in patients with metastatic castration resistant prostate cancer that was in use during the time period when this study was conducted. This agent is docetaxel which demonstrated an improvement in overall survival of 2.4 months (Hazard Ratio 0.76). The documented post-progression receipt of docetaxel was roughly the same in the two arms (50 vs. 57%), certainly not enough of a difference to cause the observed treatment effect (improvement in median overall survival of 4.1 months, Hazard Ratio 0.775). There was also minimal imbalance in the time to receipt of docetaxel, moreover there is no data to suggest that the timing (early vs. late) of docetaxel results in improved survival. The extensive analysis done for docetaxel use post-treatment (prior to FDA approval, some of which were published in the New England Journal of Medicine) have consistently demonstrated that the only reasonable explanation for the results is due to the efficacy of sipuleucel-T. Finally, there are multiple new drugs that are emerging following docetaxel that may impact survival. Cabazitaxel was recently approved (June 2010) for use in men with prostate cancer following docetaxel based on an improvement in overall survival, and abiraterone was also recently shown to improve overall survival in the same patient population and is widely expected to be approved soon by the FDA. Both of these drugs will be used in the post-docetaxel setting (for abiraterone at least initially this will be the case) and thus after the likely sipuleucel-T use. Another very promising drug, MDV-3100, is being evaluated in a phase III study in the post-docetaxel setting. Any overall survival trials done in the future in the pre-docetaxel setting will thus have many more confounding variables affecting the primary endpoint. Therefore not only is the available data from the IMPACT study on post-trial standard therapies relatively well-balanced (with any slight imbalance not able to explain the improvement in overall survival), but any trial done in the pre-docetaxel setting in the future would likely have a greater degree of uncertainty than the IMPACT study due to multiple (rather than 1) potentially confounding variables.

Dr. Gulley addresses 2 issues that numerous commentors also commented on. I will title the responses to these concern #1) salvage therapy response, and #2) bias of subsequent treatments and adequacy of analysis. In subsequent comments I will simply refer back to #1) salvage therapy response and/or #2) bias of subsequent treatments.

#1) Salvage therapy response:

Analyses of salvage benefit are either unevaluable (unpublished) or do not take into account selection and survival biases. Hazard ratios cited of 0.52-0.58 in some comments for salvage therapy is greater than the effect for sipuleucel-T.
An unmeasured benefit of salvage therapy might be assumed if salvage therapy was identical to sipuleucel-T. In addition to being made from frozen cells, salvage product is produced from sipuleucel-T naïve cells. In standard treatment, the 2nd and 3rd treatment products come from sipuleucel-T exposed cells.

The TA does not hypothesize that frozen product is directly harmful to patients.

In the setting of these research studies, it is likely to have caused the observed delay in subsequent chemotherapy treatment and may have caused the lower proportion of patient receiving chemotherapy in IMPACT. The interposition of this treatment after progression in the control group may be responsible for the complex issues in analyzing this study. In other studies of cancer treatments where there is overt evidence of treatment response, salvage therapies will then be administered more frequently to the patients in the group receiving the less effective primary therapy. Because of the delay in initiating known effective therapy induced by treatment with salvage product, analysis of the sipuleucel-T trials was more problematic than other clinical trials of cancer therapy.

#2) Analysis of subsequent treatments:

TA reports these analyses, points out limitations, suggests alternative statistical techniques. Subsequent treatments are problematic to analysis of clinical trials when there is a systematic difference in application, as occurred here due to frozen salvage product. Opinions vary as to whether these methods successfully account for potential confounding.

The technology assessment raises the issue of salvage therapy on outcome in the sipuleucel-T clinical trials. At the time of progression, patients were unblinded and control patients were offered therapy with an autologous cellular immunotherapy, APC8015F, prepared from cells cryopreserved at the time the placebo was manufactured. APC8015F was otherwise manufactured like sipuleucel-T, and was required to meet the same release specifications. To explore the potential effect of this treatment on patient outcomes, we examined the survival of placebo patients from the time of disease progression in three randomized controlled trials.

Of 249 control subjects, 165 (66.3%) received APC8015F; the median time from randomization to first infusion was 5.2 months (range 1.8 to 33.1), median time from objective disease progression to first APC8015F infusion was 2.2 months (range 0.5 to 14.6), and 145 subjects (87.9%) received all 3 infusions. APC8015F-treated subjects (n=155) had improved post-progression survival relative to untreated controls (n=61) (HR=0.52 [95%CI 0.37, 0.73] p=0.0001, log rank test, unadjusted Cox regression). The beneficial effect of APC8015F was maintained in additional analyses which adjusted for baseline prognostic features and for post-randomization docetaxel use.

We recognize that measured and unmeasured factors may confound outcomes in patients who received APC8015F; nonetheless, these analyses suggest that post-progression treatment with APC8015F may have extended the survival of subjects in the control arms of these studies. There is no clinical evidence, nor biologic rationale, to suggest that APC8015F may have worsened outcomes in patients. Therefore, the inclusion of treatment with APC8015F following progression in the sipuleucel-T trials would be expected to lead to an underestimation of the true overall survival benefit seen. My co-authors and I have submitted these findings to the 2011 Genitourinary Cancers Symposium.1

¹ George D, Nabhan C, Gomella L, Whitmore JB, Frohlich MW. Subsequent Treatment with APC8015F May Have Prolonged Survival of the Control Arm in Phase 3 Sipuleucel-T Studies. Submitted to 2011 Genitourinary Cancers Symposium. Orlando, FL Feb 17-19, 2011

I have been in the field of prostate cancer for 24 years and have been involved in or led many studies relating to the major therapeutic advances in this disease area, and was the lead investigator on the IMPACT trial, published in the NEJM on July 29, 2010. I consider the clinical development of sipuleucel-T to be a very exciting advance in the treatment of patients with prostate cancer.

IMPACT was a double blind, randomized, multicenter, placebo-controlled trial in 512 patients with metastatic castration resistant prostate cancer conducted under a Special Protocol Assessment with the FDA. Placebo was used as a control as opposed to chemotherapy with the intent of creating a clinical niche for the development of treatments which prolonged survival while causing few treatment-related side effects. Designed to be identical in appearance to sipuleucel-T, the placebo also maintained the study blind.

Following disease progression, patients were treated at the physician's discretion, and patients on the placebo arm had the option of crossing over to treatment. This aspect of the study would be expected to decrease the observed difference in overall survival; the true benefit would likely have been greater in the absence of crossover.

The use of docetaxel following study treatment was comparable to what would be anticipated in clinical practice. Several analyses to investigate the potential role of docetaxel provided no evidence for an alternative explanation for the overall survival benefit. Although these analyses have limitations, the comparable use and time to initiation of docetaxel between the treatment arms, coupled with the large difference in overall survival observed make it implausible that differences in subsequent docetaxel use between the arms could account for the study findings. The Technology Assessment questions the confounding due to crossover and subsequent interventions. Given that the true benefit may be greater than that observed in the clinical trials, the strength of the evidence would if anything be considered to be greater than that observed.

In conclusion, the IMPACT trial confirms the overall survival findings of prior randomized trials. I view this trial as definitive proof that sipuleucel-T provides clinically important benefit to patients. The requirement for additional trials for the current indication does not seem wise or ethical. The treatment represents the largest median survival increment of any therapeutic in the treatment of CRPC patients to date, delivered with modest side effects and a short duration of therapy. Sipuleucel-T represents a needed advance for patients with lethal prostate cancer. Trials in the future should be designed to build upon the success of this treatment.

Comments on AHRQ Technology Assessment Draft Date of Draft 11/2/2010

1) The comments regarding the statistical issues inherent in a 2:1 randomized trial with a potential crossover are valid. An identical trial design was employed in IMPACT, D9902A and D9901. These statistical considerations must, however, be tempered by consideration of patient acceptability and fairness. In my clinical experience, many patients with metastatic, castrate-resistant cancer are unwilling to enroll in a 1:1 randomized, placebo controlled trial. Given the relatively short survival (14 months without treatment) of such patients, this attitude is understandable. My assumption is that the trial design was conceived with a fair degree of input from patient advocates.

2) The discussion of the salvage treatment option for the placebo group is handled somewhat unfairly in the assessment draft. It the agent (and salvage treatment) are active, then the allowed crossover would be expected lead to a relative UNDERestimation of treatment benefit. The fact that a consistent survival benefit was noted in spite of this confounding influence strongly supports the notion that the agent is active, and further suggests that the treatment benefit could potentially be greater than that measured.

3) The assessment makes no mention of the rationale underlying the change in primary endpoint for IMPACT from PFS to overall survival. It is important to note that this change was based on data gathered as 9901 and 9902A progressed. Generally, survival is considered to be a more robust endpoint than PFS, and the trial was thus modified toward a MORE rigorous endpoint. This should be properly appreciated.

4) The section implying that treatment with Sipuleucel T is associated with infection is not valid. To make that inference, the authors would need some data on baseline "infection" rates in this population, as they correctly emphasize that both the control and placebo groups received I.V. infusions of processed and shipped cells. Further, the use of the term "infection" is nebulous in this discussion.

Does table 14 include routine upper respiratory infections, with an obvious baseline prevalence? In reality, the only infections that are relevant in such a discussion are those that are clearly product-related ? those data are not shown. Further, the number of infections that are catheter-related is shown to be in the 3% range, which is more than would be hoped for, but does not seem to be out of line with general clinical experience.

5) The conclusion that the survival benefit of Sipuleucel-T is observed "only in the context of a substantial amount of eventual chemotherapeutic treatment" is not supported by the data or analysis presented. Standard clinical practice in oncology is to offer additional treatment options to patients who progress on either standard or experimental therapy. To withhold such treatment is unethical. So, the trial that the reviewers suggest, assessing the survival benefit of Sipuleucel-T in the absence of chemotherapy is simply not feasible or desirable.

One could consider a trial design that makes a greater effort to control the timing and dosage of additional treatment, but it should be appreciated that such a sequential trial design would NOT assess the agent in question, but rather would assess a combination treatment approach. It is not clear to me that such studies are the responsibility of a drug manufacturer ? instead those kinds of questions are typically posed in the setting of a cooperative group trial.

The reviewers should also be cognizant of the fact that the landscape of treatment options available to medical oncologists is in continual flux. In prostate cancer, for example, a second-line chemotherapy (cabazitaxel, Sanofi Aventis), was recently approved, and a new hormonal agent (abiraterone, J&J) will likely be approved in the next several months. It would seem a nearly impossible task for a drug manufacturer to control for all such eventualities in a clinical trial design. Instead, the standard approach has been the randomized controlled trial, which makes the implicit assumption that post-treatment interventions will be relatively balanced among the treatment arms. This is the standard that other cancer treatments have been held to, and it seems unfair to impose additional constraints on one particular manufacturer or approach.

6) Finally, it is noted that the research upon which the assessment was based was conducted by the "Blue Cross and Blue Shield Association Technology Evaluation Center" under contract to the AHRQ. As a large health insurance provider, Blue Cross and Blue Shield could be assumed to have a stake in coverage recommendations reached by the AHRQ, and it would be helpful to know what steps were taken to minimize or eliminate potential conflict and/or bias.

See #1) Salvage therapy response. 

Change in endpoint—survival as a robust end point is recognized in the TA. 

Infection—"product-related" infections as such are not known, it would be premature to presume to that certain infections are not product-related. 

Benefit of sipuleucel-T in "context" of chemotherapy—because sipuleucel-T does not produce measurable anti-tumor effects, the evidence of its effectiveness exists in the context of the trials at a time point beyond secondary treatments.  Clinical trials should avoid an explicit bias in use of subsequent treatments.  In the clinical trials salvage therapy induced a delay in initiation of effective therapies.

I would like to respond to several specific issues raised by the Technology Assessment. First, the authors raise a question about the level of evidence provided by the three published Phase III trials of sipuleucel-T, describing them as "small" and suggesting that they should therefore carry less weight than a "large" trial.

As the primary author of the SWOG 9916 trial, one of two trials that supported the approval of the first agent to show survival benefit in this patient population, I would like to point out that the number of patients in the arm of the TAX 327 trial randomized to receive docetaxel at its approved dosing schedule (75mg/Kg) every three weeks, was 335. In the IMPACT trial of sipuleucel-T, 341 patients were randomized to the active treatment arm. There was little question as to our certainty of benefit from docetaxel just as there is little question of our certainty of benefit from sipuleucel-T.

The Technology Assessment also expresses concern that it is unknown whether docetaxel use subsequent to treatment with sipuleucel-T could partially explain the survival benefit in these trials. The issue of confounding of a treatment effect by subsequent therapies is one that is common to all oncology trials. To explore the potential effects of post-progression treatment, several sensitivity analyses have been completed. None of these analyses suggest that earlier and/or more frequent use of docetaxel in the sipuleucel-T arm can explain the study result, nor do they suggest that sipuleucel-T was only effective in those patients who ultimately received docetaxel.

In three randomized phase III trial of sipuleucel-T, the overall treatment effect remained robust when adjusting for docetaxel use, and a sipuleucel-T treatment effect was observed both in patients who did and did not receive subsequent docetaxel. Taken together, these analyses provide no evidence to suggest that subsequent docetaxel use explains the observed sipuleucel-T effect. (Petrylak D., ASCO 2010).

Further support for the efficacy findings in the IMPACT study is shown in the consistency of the survival benefit in, in the sipuleucel-T arms of the three phase III trials, Study D9901, Study D9902A and the IMPACT trial.

In summary the data demonstrating an overall survival benefit of sipuleucel-T for men with asymptomatic or minimally symptomatic castrate resistant prostate cancer is based on a robustly sized dataset, cannot be explained by subsequent use of docetaxel, and is consistent across three Phase 3 studies. The level of evidence should be characterized as strong rather than moderate

Sample size- robustness of clinical trial results depends on the size of the smaller group, in this case the control group.  In the GRADE evaluation, the studies are no longer characterized as "small."

Dr. Mark and colleagues have performed a thorough review of sipuleucel-T in their Technology Assessment Report entitled "Outcomes of Sipuleucel-T Therapy." There are several areas that we are concerned do not accurately reflect the evidence generated by the clinical trials, and provide below comments and suggestions for changes to the report. The two areas of particular concern are the grading of the evidence as "moderate" rather than "strong," and the suggestion that additional clinical trials in the current label indication may be warranted.

------Executive Summary------

Grading strength of the evidence (ES page 2)

The Technology Assessment (TA) inappropriately grades the strength of the body of evidence for improved outcomes of sipuleucel-T therapy as "moderate." In reviewing the Agency for Healthcare Research and Quality (AHRQ) standards for grading the strength of a body of evidence (Owens 2009), we conclude that "strong" would be the appropriate grading for the sipuleucel-T data.

According to the AHRQ standards, evidence is evaluated in four domains: risk of bias, consistency, directness, and precision. Additional domains that are relevant to sipuleucel-T include: dose-response association, plausible confounding that would decrease the observed effect, and strength of association (magnitude of effect). Each of these categories and the available evidence is reviewed below.

Risk of bias

Per the AHRQ methods, risk of bias is assessed through two main elements: study design and aggregate quality of the studies. Randomized study designs carry the lowest risk of bias. The TA states (Table A4) that "there are unknown potential confounding effects of frozen salvage product and post-progression treatments, despite the use of statistical adjustment approaches. "

In labeling potential effects "unknown, " the TA fails to characterize the level of uncertainty or the probable directionality, and fails to provide a score for the "bias " domain. Per the AHRQ methods the domain of "bias, " scores should be denoted high, medium, or low. High risk of bias lowers the strength-of-evidence grade; low risk of bias raises it.

Lacking any score, the TA does not provide insight to the reviewers‘ thinking regarding these potential sources of bias. Other aspects of the TA suggest that the reviewers concluded that these risks were not only theoretical, but were also unlikely to reduce the strength of the evidence. To clarify, the potential bias introduced by salvage product and variations in post-progression treatment could have only acted in one of 3 ways " the two extremes of which are outlined on pages 12 and 13 of the TA.

Namely, the bias could have reduced the observed benefit relative to the "true" benefit (i.e., conservative bias), had no impact, or could have created an artificially large observed benefit relative to the "true" benefit (i.e., optimistic bias). Nowhere in the TA is there any suggestion that the latter of the 3 effects was at play in these studies, and the concern is considered mainly theoretical. In fact, there is no mention of any evidence within the TA suggesting that either the frozen salvage product or variations in post-progression treatment caused the observed effect to be larger than the "true" effect.

Such a phenomenon is the only type of bias that should concern reviewers that the evidence of benefit is weaker than it appears. Yet, the multiple sub-group analyses conducted by the FDA as well as data presented at the MEDCAC meeting provide no evidence of a bias in the direction that would weaken the strength of evidence.

Several prostate cancer experts at the MedCAC meeting noted that the crossover design of the study was likely to lead to a more conservative estimate of the overall survival benefit, and that post-progression chemotherapy could not explain the observed survival benefit. The TA itself notes on page 23 that whether such a bias is present in these studies is "difficult to determine."

Moreover, the TA notes that all of the analyses that adjusted (using multiple methods) for post-progression treatments failed to show any evidence of a bias that favored sipleucel-T. Potential confounding by the salvage product and post-progression variation in treatment are discussed individually below.

Disagree that potential bias of post-treatment chemotherapy is "theoretical," as IMPACT study showed earlier and more frequent use of known effective treatment.  This would cause an over-estimation of treatment effect.  See also #1) Salvage therapy response.   Given the differences between salvage therapy and sipuleucel-T, we do not assume an unmeasurable benefit of cross-over therapy.  GRADE table has been revised to be more clear regarding the presence of bias.

Salvage product.

The TA suggests the possibility that the salvage product could have had a negative impact on the survival of control-group patients. The TA should acknowledge that while this is a theoretical possibility, the available facts make it unlikely.

First, there is no biologic rationale to suggest this. The salvage product, APC8015F, is prepared from cells cryopreserved at the time of placebo generation. Once the cells are thawed, washed and placed into culture, the manufacturing process is exactly the same as for sipuleucel-T and the final salvage product must meet the same release specifications as sipuleucel-T (Kantoff 2010). There is a long history of the safe use of cryopreserved blood products, including red cells, platelets, plasma and stem cells (Sputtek 2007, Watt 2007).

Second, the overall outcome of the placebo arm is very favorable compared to the control arms of contemporaneous trials of men with asymptomatic or minimally symptomatic metastatic castrate resistant prostate cancer (Berthold 2008, Higano 2009, James 2009, Kantoff 2009), suggesting that it is unlikely that the observed survival benefit in the sipuleucel-T group is due to poor outcome of the placebo group.

Finally, survival of men on the placebo arm who specifically received APC8015F is very favorable and comparable to men who received sipuleucel-T. In the IMPACT trial, median survival for sipuleucel-T patients was 25.8 months vs. 23.8 months for those receiving placebo and APC8015F and 11.6 months for those receiving placebo only (Kantoff 2010). Interpretation of these outcomes is confounded by potential selection bias, but a significant effect of APC8015F on overall survival in placebo subjects persists after adjustment for potential confounding differences in baseline characteristics (HR=0.576; 95% CI: 0.380, 0.872).

Additional exploratory analyses have examined overall survival from the time of disease progression for placebo patients who received APC8015F compared to those who did not. A positive treatment effect was observed for placebo patients who received APC8015F, relative to those who did not. This result held both in an unadjusted analysis, and analyses which adjusted for baseline prognostic factors and subsequent docetaxel use (George 2011).

In summary, multiple analyses provide no indication that salvage treatment was harmful to patients, and in fact suggest if anything, the contrary. The observed overall survival benefit observed in the randomized trials of sipuleucel-T is therefore likely to be an underestimate of the true treatment effect.

 See #1) Salvage therapy response.   The studies of frozen salvage product-associated survival are unpublished and unevaluable.  If the analyses do not take into account  "immortal time bias," the fact that patients who receive frozen salvage product have 100% survival up to the time of receiving it, then the analyses are not valid.

Subsequent therapies.

Potential confounding from subsequent therapies is a feature which is common to all survival trials in oncology (Center for Drug Evaluation and Research, Food and Drug Administration. Guidance for Industry: Clinical Trial Endpoints for the Approval of Cancer Drugs and Biologics. May 2007).  Sensitivity analyses to explore the potential influence of docetaxel have included adjustment for docetaxel as a time dependent covariate in a Cox model, and an analysis censoring subjects at the time of docetaxel initiation.

These, and additional analyses performed by FDA and reviewed by external consultants (FDA Center for Biologics Evaluation and Research, statistical review of sipuleucel-T) failed to provide evidence that subsequent chemotherapy could explain the survival benefit.  Furthermore, future survival trials of agents like sipuleucel-T that are used in the pre-chemotherapy space in metastatic castrate resistant prostate cancer will involve a much greater degree of confounding from subsequent therapies, given that in addition to docetaxel, two new agents, cabazitaxel and abiraterone, have already been demonstrated to prolong overall survival.

The final TA should grade the strength of evidence regarding potential bias from salvage and subsequent therapy on outcomes observed with sipleucel-T.  Attempts should be made to quantify the directionality of hypothetical concerns, taking into consideration the biology, as well as the available clinical data and sensitivity analyses. If after reviewing these data the TA grades this bias as "high risk," it should provide what observations in the empiric data, and/or what biologic data, supports the contention that either cross-over or post-progression treatment variation biased findings towards an over-estimation of the survival benefit.

If the revised TA grades the risk of bias as "ow", as would be in keeping with the available data, this would lead to the overall strength of evidence being graded higher as specified in the AHRQ manual.

Consistency

As noted in the TA, "The survival findings of the studies are consistent in direction and magnitude.  Disease progression outcomes showed no difference."

Directness

As noted in the TA, "The outcome of overall survival is the most direct and least subject to bias."

Precision

The TA concludes that "the result is not precise due to the small overall sample size and unknown direction and magnitude of potential confounding variables."
In reaching this conclusion, the TA did not take the steps strongly encouraged in the AHRQ methods to generate an accurate estimate of precision.  Moreover, the TA confounds issues of "bias" with issues of "precision," and it also employs a descriptor for the sample size (i.e., "small") which has neither statistical nor clinical meaning.  It also fails to provide a grade for precision.  Specifically:

The AHRQ methods suggest that meta-analytic techniques should be used when "appropriate," which should be when multiple potentially combinable studies are available (page 3), and proposes that precision be assessed in "pooled analyses" (page 5).  The standards for "precision" involve whether the estimate and the boundaries of confidence around it are relevant to clinical decision makers.  The manual requires that: "EPCs [Evidence-based Practice Centers] should assess the boundaries of the pooled confidence interval for that effect estimate in relation to a threshold that would allow CER [Comparative Effectiveness Reviews] users to make judgments about the treatments being compared."

In the case of sipleucel-T, integration of the results from the three studies in metastatic castrate resistant prostate cancer is justified by the very similar patient populations and trial designs. This integrated analysis, based on 737 patients, demonstrated a HR of 0.734 (95% CI, 0.612, 0.881; P=0.0009) (Finn, FDA Summary Basis for Regulatory Action, 2010).  The upper bound estimate of 0.881 is not close to 1.0, and even the 12% reduction in the risk of death is clinically meaningful to patients when the outcome is overall survival. 

The AHRQ methods also provide some insight into what should be classified as "imprecise," giving as an example, "A truly imprecise estimate is one with a confidence interval so wide that it does not rule out the superiority or inferiority of either treatment being compared"that is, an estimate whose confidence interval includes two incompatible possibilities: one treatment is clinically significantly better than the other, and the difference is in the opposite direction? (page 5).  There are no data in the TA that even approach being internally contradictory. 

The TA mistakenly incorporates issue of "bias" into its consideration of "precision," essentially double-counting the concerns reviewers had about bias (even though those are purely speculative).  Per the AHRQ methods definition,  "Precision is the degree of certainty surrounding an estimate of effect with respect to a specific outcome (page 5)," and the AHRQ methods specifications explains that the dimensions of effect estimates have to do with specific statistical issues of confidence boundaries and how those boundaries interact with clinically important endpoints.  There is no suggestion in the AHRQ methods that issues of "bias" are to be included in the assessment of "precision" " a logical division given that the domain of "bias" is separately considered. 

The TA invokes the term "small" to describe the available sample size.  There are three problems with this term.  First is its imprecision.  The TA does not define what sample size would be considered "adequate" or "sufficient," so the subjective term "small" cannot be assessed for its validity or credibility.  Next, in characterizing the studied sample size as "small," the TA seems to be suggesting that the sample size is small with respect to some other study type or design.  If so, then the size of other clinical studies in similar populations should be contemplated for comparison. 

Under such an approach, the reviewers would have found that the size of the sample is consistent with that of other studies that led to treatment paradigm shifts in prostate cancer (details of such comparisons are provided below).  Last is the problem that in labeling the studies as small, the reviewers ignore the ethical requirement of clinical research to not over-enroll studies that could potentially place patients at risk when adequate answers can be gleaned from studies involving fewer patients.  It is axiomatic in clinical research that studies be appropriately sized, but not overpowered, particularly when the treatments being assessed could be toxic and the subjects being studied have life threatening conditions.  

For instance, the "Common Rule" includes, under ?46.111 Criteria for IRB approval of research:  (a) In order to approve research covered by this policy the IRB shall determine that all of the following requirements are satisfied: (1) Risks to subjects are minimized: (i) By using procedures which are consistent with sound research design and which do not unnecessarily expose subjects to risk.  Calling the studies "small" implies that the reviewers consider ethical requirements including the minimization of risk for volunteer subjects unimportant.

The TA fails to provide a score for the level of precision.  The AHRQ methods require a dichotomous choice between "precise" and "imprecise", explaining that a "precise estimate is an estimate that would allow a clinically useful conclusion. An imprecise estimate is one for which the confidence interval is wide enough to include clinically distinct conclusions. For example, results may be statistically compatible with both clinically important superiority and inferiority (i.e., the direction of effect is unknown), a circumstance that will preclude a valid conclusion.? 

It is inconceivable that the highly consistent data on sipleucel-T could be judged to be ?imprecise," because the confidence intervals are simply not wide enough to accommodate clinically distinct conclusions.  Moreover, it seems clear that the upper bound of the pooled confidence interval of 0.88 is sufficiently below 1.0 to conclude that the benefit is clinically meaningful.

The TA should be revised so that the level of precision of the overall survival benefit is evaluated in an integrated analysis of the three randomized studies in metastatic castrate resistant prostate cancer and that the standards for what constitutes a clinically important upper bound of the confidence limit be specified so it can be evaluated in a transparent manner.  Moreover, the theoretical issues of "bias" should be removed from the assessment of "precision" as they are already incorporated into the evaluation of "bias".  This will avoid the current double counting of these theoretical concerns. 

Lastly, the TA should include specific statements about sample size that clarify how the reviewers arrived at the judgment that the sample size was "small", and the reviewers should offer sample sizes that they would have deemed "adequate" and "excessive" (or whatever terms are appropriate).  They should also explain the rationale for considering the sample size small given the importance of protecting human subjects and the reality that the study’s power was adequate and appropriate in the eyes of the IRB’s that reviewed it and the FDA at the time of approval.   A score for the precision as either "precise" or "imprecise" should be included.

In a narrow interpretation of precision, which relates to the confidence interval of the estimate of treatment benefit, we now state that the point estimate of benefit across all 3 trials is precise.

Additional domains

Three of these AHRQ methods defined domains are relevant to the sipuleucel-T data.

Dose-response association

Per the AHRQ methods, "this association, either across or within studies, refers to a pattern of a larger effect with greater exposure (dose, duration, adherence)." Associations between overall survival and the total nucleated cell count, the absolute number of CD54 positive large cells, and the CD54 upregulation ratio have been observed in the sipuleucel-T trials. In an integrated analysis of the three randomized metastatic castrate resistant trials, there was a significant correlation between overall survival and each of these 3 product parameters, in analyses both unadjusted and adjusted for baseline parameters (Stewart 2010).

Per the TA, "The results between the two studies show a consistent positive direction of associations, but apparently the strength of the association of a particular parameter varies between studies." Based on these facts, the TA should score this domain as "Present".

Plausible confounding that would decrease the observed effect

Per the AHRQ methods, "Occasionally, in an observational study, plausible confounding factors would work in the direction opposite that of the observed effect. Had these confounders not been present, the observed effect would have been even larger than the one observed. In such a case, an EPC may wish to upgrade the level of evidence."

This is a case which is particularly relevant to the sipuleucel-T data, with the trial designs including cross-over to salvage (see above). The appropriate grading for this domain would therefore be "present," i.e., "confounding factors that would decrease the observed effect may be present."

Strength of the association (magnitude of the effect)

Per the AHRQ methods, "Strength of association refers to the likelihood that the observed effect is large enough that it cannot have occurred solely as a result of bias from potential confounding factors." The median survival benefit observed in the sipuleucel-T trials (4.5 and 4.1 months) is larger than any other FDA approved agent for metastatic castrate resistant prostate cancer.

In conclusion, the TA should score each of the four domains and the additional domains relevant to sipuleucel-T. The TA should also provide an explanation of how these scores are integrated to provide a grading of the level of evidence, given the AHRQ methods statement that "in arriving at an overall strength-of-evidence grade, the crucial requirement is transparency. " The level of evidence should be graded as strong based on the results being consistent, direct and precise, and there being a low risk of bias.

The level of evidence warrants upgrading based on the presence of a dose-response association, plausible confounding that would decrease observed effect, and a strong magnitude of the effect. Failing a change in the evidence designation to strong, the TA should clarify for readers that it appears impossible for oncologic survival trials to reach a rating of "strong," and therefore, sipuleucel-T has reached the highest level of evidence that the grading system allows for oncology.

Dose-response:  the analyses of product characteristics and outcomes does not constitute evidence of dose-reponse.

Plausible confounding:  the systematic bias of post-treatment chemotherapy caused by delay due to administration of  frozen salvage product in IMPACT is a plausible source of confounding.

Strength of association:  The direction of plausible confounding we believe is in the direction of the studies over-estimating the effectiveness of sipuleucel-T.  This leads to moderating the calculated magnitude of effect.

Risk of infection (ES page 3)

The TA currently states: "Sipuleucel-T also is associated with infections." This implies that sipuleucel-T treatment increases the general risk of infection, which is not the case. It should be clarified that there is a risk of infection associated with central lines which may be required for the leukapheresis procedure. (As the TA does correctly state, infections that appeared to be due to catheter related infections occurred in approximately 3 percent of subjects.) The results section of the TA (page 31), states, "However, 15.3 percent and 14.5 percent of subjects in each group developed infections within one week of their final infusion.

These infections are more likely related to leukapheresis and/or infusion."  Most of these infections are in fact not related to leukapheresis or infusion. First, it is important to understand that the period of data collection was from registration until 1 week after the last leukapheresis or infusion, whichever came later. Since treatment typically occurs over a 4-6 week period, the duration of adverse event collection would be approximately 5-7 weeks for the typical patient. 

A review of the adverse events in this period reveals events which might be anticipated in an elderly population followed for a 1-2 month period: pharyngitis, bronchitis, sinusitis, urinary tract infection, etc. As a cross-reference, the FDA review of an endothelin-A receptor antagonist, atrasentan, documents the frequency of adverse events related to infection in the advanced prostate cancer population (FDA clinical review, accessed Nov 24, 2010, http://www.fda.gov/ohrms/dockets/ac/05/briefing/2005-4174B1_02_01-FDA-Clinical-Xinlay.pdf). 

The duration of adverse event collection was longer than for the above sipuleucel-T analysis and likely roughly the duration of treatment, which was 16.5 weeks in the placebo arm. A comparison of specific adverse event terms from the FDA review is provided in the table below.  The frequency of these events is higher in the atrasentan trials consistent with the longer duration of follow-up for that analysis, but serves to substantiate the fact that these infectious adverse events are not uncommon in this patient population.

Considerations for future trials (ES page 3)

The TA states that "Consideration should be given for conducting unblinded trials without placebo infusions or salvage product, and for more thorough treatment protocols out to the survival end point." The TA should make statements that are consistent with the standards in oncologic research and the practices of FDA, study groups and IRB boards reviewing oncology clinical trials. We therefore have several concerns with this statement and suggest revising it to: "For future trials in other indications, consideration should be given to designs without cross-over, and with overall survival as the primary endpoint." The rationale for these changes is below:

The current sentence in the TA suggests that an unblinded, non-placebo controlled trial would lead to higher confidence of the trial outcome. This is not the case. Blinded placebo controlled trials may not be necessary in future trials for other indications where overall survival is again the primary endpoint, but the blinded, placebo controlled design of the studies under review considerably strengthens rather than diminishes the robustness of their findings.

Blinded, placebo controlled trials remain the gold standard design in oncology trials, given they are the least susceptible to bias. There is no biologic rationale that the placebo product used here could be harmful to patients; the product consists simply of the patients own peripheral blood mononuclear cells held at 2-8?C without addition of antigen. While there are adverse events associated with the leukapheresis and infusion procedures for both the sipuleucel-T and control groups, these risks can be readily quantified by the temporal association of these adverse events with leukapheresis and infusion (see "Risk of infection" section above).

The presence of the observed adverse events associated with the control arm is not sufficient grounds for the TA to claim that a non-placebo controlled trial design would actually be superior to a placebo-controlled trial.

The current sentence in the TA suggests that additional clinical trials in the current label indication would be important. As noted in the FDA review of sipuleucel-T, "Study D9902B is an adequate and well-controlled investigation in which sipuleucel-T demonstrates a clinically meaningful effect on survival in patients with metastatic prostate cancer. Because of the effect on survival, a second trial would not be ethical or feasible" (Bryan 2010).

When Dr. Mark presented the TA at the Nov. 17th Medicare Evidence Development and Coverage Advisory Committee (MedCAC) meeting, he did clarify that this statement regarding additional trials referred to additional indications, and not the current label indication. This should be clarified in the TA.

The current TA statement suggests that more thorough treatment protocols out to the survival endpoint need to be included in future clinical trials. While this would address the TA’s concern regarding variability of post-progression cancer care, it is neither feasible nor ethical. The standard design for oncology clinical trials is to specify treatment up until the time of disease progression. Mandating that all patients be required to receive, for example, docetaxel chemotherapy following disease progression would not be ethical or permissible by institutional review boards (IRBs).

The decision if and when to institute chemotherapy or other care must be individualized to each patient coping with metastatic cancer, and will depend on a variety of factors, including their degree of symptomatology, the rate of disease progression, and patient preferences at the time of treatment. Decisions about the long-term future medical therapy of ill patients cannot be dictated in a clinical protocol at the time of randomization.

TA was adjusted to emphasize that trials should avoid potential for systematic biases of application of subsequent treatments and equal quality of care across treatment arms.

---Introduction/Background---

Description of sipuleucel-T (page 3)

While sipuleucel-T contains dendritic cells, and early publications described the biological as a dendritic cell product, a more appropriate description of the product would be (additions in capital letters):

" The collected cells, which are mixture of PERIPHERAL BLOOD MONONUCLEAR CELLS (remove 'dendritic cells'), INCLUDING ANTIGEN PRESENTING CELLS, T-cells, (remove 'monocytes'), and B-cells, are then cultured with a protein called PA2024. PA2024 is a recombinant protein consisting of human prostatic acid phosphatase (PAP) fused with granulocyte-macrophage colony stimulating factor (GM-CSF). The interaction of the ANTIGEN PRESENTING CELLS (remove 'dendritic cells') with PA2024 is considered the essential process that stimulates the immune system. ANTIGEN PRESENTING CELLS(remove 'Dendritic cells') take in antigens and then "present" them to T-cells throughout the body, which should then react to cells with PAP such as prostate cancer cells as foreign substances."

The following paragraph on page 4 should also be modified as follows:

During the manufacture of sipuleucel-T, the number of CD54 molecules expressed on the(remove'dendritic') antigen PRESENTING CELLS increases. The amount of this increase can be quantified using specific assays. The quantity of this increase varies between patients, and varies depending on prior exposure to sipuleucel-T. Greater CD54 upregulation is observed after the first dose of treatment, indicating that (remove 'dendritic') antigen PRESENTING CELLS respond differently to culture in PA2024 after an initial systemic exposure to sipuleucel-T.

-----------Methods-----------

No comments.

-----------Results-----------

Gleason score (page 13)

"IMPACT enrolled a higher proportion of subjects with Gleason scores of 7 or less" should be a lower proportion, not a higher proportion, as demonstrated by the previous sentence: "The entry criteria Gleason score for IMPACT was changed during the trial from 7 or less to any Gleason score."

Progression endpoint (page 13)

The TA currently states: "However, studies D9901 and D9902A were designed for an end point of progression-free survival, and the primary end point of IMPACT was changed from progression-free survival to overall survival." Both occurrences of "progression-free survival" in this sentence should be changed to "time to disease progression," since "time to disease progression" was the pre-specified endpoint in these trials, not "progression-free survival." These endpoints are similar, but have distinct definitions.

Wording changes adopted.

Reviewer erred regarding Gleason score, or misunderstood.  IMPACT differed from the other 2 smaller trials in the proportion of patients with Gleason score 7 or less.

Time to development of cancer related pain (page 16)

The TA reports the lack of difference in the median time to development of cancer related pain in Studies D9901 and D9902A. While not statistically significant, Studies D9901 and IMPACT both showed a trend towards a delay in the time to development of cancer related pain (Small 2010, Petrylak 2010). In each of these studies there was a marked delayed separation of the Kaplan-Meier curves, accounting for the similar median time to development cancer related pain between the treatment arms in these studies. In the IMPACT trial, for example, the estimated percentage of patients free of cancer-related pain at 12 months was 32.7% in the sipuleucel-T arm compared to 14.5% in the control arm.

Time to docetaxel initiation (page 16)

The TA currently states, "In IMPACT, a greater proportion of sipuleucel-T-treated patients received docetaxel chemotherapy (57.2 percent versus 50.3 percent), and they also received it earlier (median 7.2 months versus 9.6 months)." The stated time to receipt of docetaxel is based on the median of those subjects who actually received docetaxel. While these numbers provide the time to docetaxel in those who actually received it, they ignore the large number of patients who did not receive docetaxel at all.

The Kaplan Meier method is the standard method for time to event analyses for endpoints such as time to disease progression or overall survival, and was therefore the method used to describe the median time to docetaxel initiation in the published manuscript in the New England Journal of Medicine: "The Kaplan?Meier estimate of the median time to docetaxel use was 12.3 months in the sipuleucel-T group and 13.9 months in the placebo group" (Kantoff 2010). It would therefore be more appropriate to cite these estimates in the TA, given the statement in the TA, "We used the peer-reviewed publication value whenever there was a discordance."

The longer time to initiation of docetaxel in the control arm, while small, depended on an event that would occur only in the study and not in the real-world setting, namely the interposition of salvage treatment with APC8015F for many patients on the control arm. A sensitivity analysis was performed using Kaplan-Meier estimates of time to initiation of docetaxel or salvage treatment with APC8015F, whichever came first. In the IMPACT trial, the median time to intervention was 12.3 months for sipuleucel-T compared with 6.5 months for control and for the integrated analysis of the 3 randomized trials in metastatic castrate resistant prostate cancer, 16.8 months for sipuleucel-T as compared to 6.3 months for control (Dendreon, data on file).

It is also noteworthy that the exploratory analyses undertaken to examine the potential effect of subsequent docetaxel use also adjusted for any potential differences in the timing of docetaxel initiation. These analyses include the adjustment for docetaxel use as a time dependent covariate in a Cox regression model, and the analysis censoring patients at the time of docetaxel initiation.

Sample size (page 19)

The TA states: "Although the findings of the studies are mostly consistent in showing a similar magnitude hazard ratio, estimates of the effectiveness of sipuleucel-T are imprecise due to the relatively small total sample size of the clinical trials"?

The overall sample size for the randomized studies of sipuleucel-T in castrate resistant prostate cancer was not small. The overall survival result for the IMPACT trial was based on 512 patients, and revealed a HR of 0.775 (95% CI, 0.614, 0.979; P=0.032). An integrated analysis of the 3 randomized trials in metastatic castrate resistant prostate cancer based on 737 patients demonstrated a HR of 0.734 (95% CI,0.612, 0.881; P=0.0009) (Finn, FDA Summary Basis for Regulatory Action, 2010). The registration trial for docetaxel, was based on 672 patients between the docetaxel every 3 week arm and the control arm (Tannock 2004).

The trial demonstrated a HR of 0.76 (95% CI, 0.62 to 0.94; P=0.009). Based on the 95% CI of the HRs, the estimate of the sipuleucel-T treatment effect based on the IMPACT trial was therefore comparably precise to that in the Tax327 trial, and more precise than the Tax327 trial based on the integrated analysis of the 3 randomized trials of sipuleucel-T in metastatic castrate resistant prostate cancer.

The pain data cited is unpublished.

Time to chemotherapy is not an outcome, therefore not critical to use Kaplan Meier.  Kaplan Meier estimate does not take into account competing risk of mortality, simply censors patients at death.

Reviewer cites unavailable data.

Biologic mechanism of action (page 23)

It is important to clarify that while the biologic mechanism of action of sipuleucel-T is not fully understood, as is the case for most agents in oncology, the available data support the proposed mechanism of action. Sipuleucel-T is designed to generate an immune response against the tumor antigen, prostatic acid phosphatase (PAP). Studies labeling the recombinant PAP-GMCSF fusion protein have demonstrated that the antigen is taken up into antigen presenting cells, defined as large CD54 expressing cells.

These cells have been demonstrated to present PAP peptides as assessed by the proliferation of T cell hybridomas recognizing PAP peptides (Sheikh 2008). We have documented evidence of both cellular and humoral immune response to PAP-GMCSF and to PAP (Frohlich 2010, Kantoff 2010). Furthermore, the demonstrated correlations between these measures and overall survival, as well as between measures of product potency and overall survival (Higano 2009, Kantoff 2010, Frohlich 2010), support the biologic importance of these biomarkers.

Nonfatal Serious Adverse Events (page 28)

This section and Table 12 on page 29 should be titled, simply, "Serious Adverse Events," as serious adverse events included fatal events.

Risk of cerebrovascular events (page 30)

Table 13 lists the incidence of cerebrovascular events at 4.0% and 2.9% in the sipuleucel-T and control groups, respectively. It should be clarified that these figures include transient ischemic attacks (TIAs). The figures of 3.5% and 2.6% in the sipuleucel-T and control groups, respectively, as stated in the FDA label exclude TIAs, since these events do not have the same clinical consequence for patients as the other cerebrovascular events, e,g., ischemic or hemorrhagic stroke.

Risk of infection (page 31)

See "Executive Summary" section.

Infusion Reaction Adverse Events (page 32)

The incidence of any infusion reaction adverse event in the sipuleucel-T group should be 71.2 percent rather than 71.4 percent, which is a typographical error within the CBER Clinical Review, Table 38. Please see the Package Insert/Prescribing Information, Section 5.1.

FDA document is in fact unclear, section heading of text differs from table title.