|Home | About | Journals | Submit | Contact Us | Français|
This article examines the effects of changing parameters in the test which was proposed by the FDA at the October 2005 Advisory Committee meeting for confirming delivered dose uniformity in orally inhaled and nasal drug products. This article is an extension of the characterization study presented in an accompanying article (Part 1). The goal of this study is to understand how parameters of the test affect the test performance. The effects of changing test parameters such as target interval, maximum allowable proportion in the tail area, and sample size are examined. The results show that changing the maximum allowable tail area and/or the target interval have the largest impact on the test outcomes, i.e., probability of acceptance for a given batch mean and standard deviation. The presented information may provide potential users of the test with a set of tools for optimizing the test characteristics for a particular product.
This paper is the second in a series examining the two one-sided tests (TOST) based on the parametric tolerance interval (PTI) approach as proposed by FDA at the October 2005 meeting of the Advisory Committee for Pharmaceutical Science (1). The first paper of this three-part series focused on the details and performance characteristics of the “default” test described by FDA as “95% confidence level, 87.5% coverage, target interval=80–120% of the label claim (LC), sample size=20+40 DDU observations in the 1st+2nd tier, each inhaler being tested in the beginning and end of container life.” However, the FDA presentation included the possibility that other sampling plans and test protocols (e.g., other sample sizes, number of tiers, size ratios between tiers 1 and 2, etc.) can be considered. In order to facilitate assessment of various test options and to raise awareness of the consequences of changing test parameters, the present paper studies the consequences of various options that might be chosen for the PTI-TOST in the course of developing a product and meaningful acceptance criteria for DDU of orally inhaled and nasal drug products (OINDPs). As an aid to the reader, a list of Notations and Abbreviations is included at the end of this article.
The PTI-TOST proposed by the FDA was described in detail in the first paper of this series and is briefly summarized below. For the operating characteristic (OC) curves, statistical simulations were used based on the Monte-Carlo technique by sampling with replacement from a normal distribution and following the specified sampling scheme. Other assumptions and notations are summarized in Part 1. The test parameters that were varied are: the maximum allowed tail area (PmaxTA), total sample size (N), the ratio of sample sizes in the first and second tier (N1/N2) and the target interval [L, U]. The overall significance level (α) and the structure of the test are not changed.
The “default” PTI-TOST was described in detail in the first paper of this series and is summarized here for convenience, as discussed below.
Tier 1: Collect 20 doses from 10 multi-dose OINDP units (a beginning-of-unit (BOU) and an end-of-unit (EOU) determination from each unit). The 20 observations must pass the following criteria:
If the sample fails any of the criteria (1a–1d), the test proceeds to the second tier. In Tier 2, collect an additional 40 observations and repeat the steps above with N=60 and K2 substituted for K1. In Table I, K1 and K2 are given for several sample size and PmaxTA options when α1=0.0226 and α2=0.0340, which were recommended by FDA based on the Pocock method (see the Appendix to Part 1 of this series) and match the K values of the FDA proposal (2). The value of (100-2*PmaxTA)% is given in Table I for ease of comparison with the coefficients presented by FDA, where this value is called “coverage.”
The data used in the section “Considerations for a Specific Product” were simulated based on a real product but adjusted to fit the assumptions of the PTI-TOST (i.e., normal distribution, BOU, and EOU means are the same, BOU and EOU variances are the same, and there is no correlation between the two). For this example, the grand mean was set at 98% LC for illustrative purposes. In summary, the data were generated so that, within a batch, BOU and EOU have equal means and variances and
For each batch, a single value was generated for (batch-to-batch variability) as normally distributed random variable with mean=0 and standard deviation=4.
Within a batch, 10 values were generated for BOU and 10 values for EOU for (within-batch variability) as a normally distributed random variable with mean=0 and standard deviation=6.
A total of 25 batches were created.
The following figures illustrate the effects of changing various test parameters on the acceptance probability of the PTI-TOST. In each plot, there are certain regions within the plot where acceptance probability is either 100% or 0%, the OC curves overlap, and changing of the test parameters does not matter. The description of results will focus on the intermediate region where acceptance probability is more than 0% but less than 100%.
Figure 1 (for the constant mean) shows that as the total sample size increases, the shape (steepness and position) of the OC curves approaches the “ideal” shape for an OC curve, which is a step function (a true step-function OC curve will rarely occur in practice, since the entire population is rarely tested, especially with destructive tests). As the total sample size increases, the OC curves become steeper and shift to the right so that in the intermediate region, the probability of acceptance increases for a given standard deviation. For example, in the 1st tier, for an on-target batch with sigma=7, probability of acceptance is ~0.3 when N1=10 but rises to ~0.95 when N1=30. Conversely, in tier 1, for acceptance probability=0.8, sample size N1=10 allows a sigma of ~5 but N2=30 allows a sigma of ~7.5.
Similarly, Fig. 2 (for the constant sigma) shows that in the intermediate region, as the total sample size increases, the probability of acceptance increases for a given batch mean. For example, in the 1st tier, for a batch mean of ~95% LC, probability of acceptance is ~0.5 when N1=10 but rises to ~1 when N1=30. Conversely, for a given probability of acceptance, an increased sample size allows a larger deviation of the mean from the target. For example, in tier 1, for acceptance probability=0.8, sample size N1=10 allows an off-target deviation of ~2% but N1=30 allows an off-target deviation of ~7%.
Table II illustrates the effect of changing sample size on the batch coverage required to pass with two specific acceptance probability values (99% and 5%). Both Figs. 1 and and22 and Table II reflect a basic property of PTI tests, namely that an increased sample size allows a larger standard deviation and/or mean deviation from the target.
Figure 3 (for the constant mean) shows that in the intermediate region, as the maximum allowable tail area increases, the probability of acceptance increases for a given batch standard deviation. For example, in tier 2, for sigma=10, PmaxTA=2.5, probability of acceptance is ~0.01 while for PmaxTA=10%, probability of acceptance rises to 0.94. Conversely, for a given probability of acceptance, increased PmaxTA allows a larger sigma. For example, in the right panel for the overall test, for acceptance probability=0.8, PmaxTA=2.5 % allows a sigma of ~7.3, while PmaxTA=10.0% allows a sigma of ~10.5.
Figure 4 (for the constant standard deviation) shows that in the intermediate region, as the maximum allowable tail area increases, the probability of acceptance increases for a given batch mean. For example, in tier 2, for a mean of ~90% LC, for PmaxTA=2.5, probability of acceptance is ~0.05, while for PmaxTA=10%, probability of acceptance rises to 0.95. Conversely, for a given probability of acceptance, increased PmaxTA allows a larger deviation of the mean from 100% LC. For example, in tier 2, for acceptance probability=0.8, PmaxTA=2.5 % allows a mean of ~93% LC, while PmaxTA=10.0% allows a batch mean of ~89% LC.
Figure 5 (for the constant mean) shows that in the intermediate region, as the target interval increases, the probability of acceptance increases for a given batch standard deviation. For example, in tier 2, a batch with sigma=10 and a target interval of 100±15% LC has a probability of acceptance ~0 while for the target interval 100±30% LC the probability of acceptance rises to ~1. Conversely, for a given probability of acceptance, an increased target interval allows a larger sigma. For example, in tier 2, for acceptance probability=0.8, the target interval of 100±15% LC allows a sigma of ~7, while the target interval of 100±30% LC allows a sigma of ~13.5.
Figure 6 (for the constant standard deviation) shows that in the intermediate region, as the target interval increases, the probability of acceptance increases for a given batch mean. For example, in tier 2, for a target interval 100±15% LC and a batch mean of 90% LC, probability of acceptance is ~0, while with the target interval of 100±30% LC, the probability of acceptance rises to ~1. Conversely, for a given probability of acceptance, increased target interval allows a larger deviation of the mean from 100% LC. For example, in tier 2, for acceptance probability=0.8, the target interval of 100±15% LC allows a mean of ~95% LC, while the target interval of 100±30% LC allows a batch mean of ~86% LC.
The previous section discussed the impact of changing total sample size, maximum allowable tail area, and target interval from a theoretical perspective. The OC curves illustrated the probability of accepting a single lot with certain characteristics. Given a product of certain quality, a sponsor may be interested in estimating the long-term conformance rate, tier 2 testing frequency, and expected average sample size for a number of sampling scenarios. This information can be used to choose an appropriate sampling scheme for the product that would allow the most efficient use of resources without compromising the test′s ability to manage product quality.
For this case study, simulated data (similar to that available from a typical product development program) were generated with an overall mean of 98.0, a batch-to-batch standard deviation of 4.0 (which causes batch means to deviate from 98.0), a within-batch standard deviation of 6.0, and an assumption that no difference in BOU and EOU means exists. A sampling of data from 25 batches is shown in Fig. 7. If a large number of batches are generated (e.g., 100,000), long-term conformance (pass/fail) rates, tier 2 frequency, and expected average sample size can be estimated for various sampling schemes, maximum allowable tail area, and target interval combinations. The overall significance level is assumed to be α=0.05 and the distribution of α in the tiers was the same as in the first article of this series. Four sampling plans, three maximum allowable tail areas, and three target intervals were considered and the simulation results are given in Table III. The plans are sorted by nonconformance rate for each tail area or target interval combination.
Figures 8 and and99 illustrate the scenarios for a maximum allowable tail area of 6.25%. Nonconformance (fail) rates will fall with larger overall sample sizes (Fig. 8). The expected nonconformance rates when the target limit is ±20% are likely higher than most sponsors would consider acceptable (Fig. 8) for any sampling scheme. The 20/40 plan had slightly lower nonconformance rates (Fig. 8) and lower expected average sample sizes (Fig. 9) than the 30/30 plan.
Simply plotting the batch data against the maximum allowable standard deviation for likely sample means will draw similar conclusions about the sampling plans. Figures 10 and and1111 illustrate the 25 batches against the passing criteria for 100±20% limits and maximum allowable tail area of 6.25%. Data points above the criteria would fail the PTI tier requirements. A single batch outside the tier 2 criteria represents an estimated 4% nonconformance rate (1/25). Sponsors would not have to resort to sophisticated simulations to get a rough evaluation of the sampling options.
Simulations may provide information that would support more detailed resource decisions. The sample plan decision is dependent upon which limits are ultimately assigned to this product. For instance, if the target interval is set at ±20%, the larger sampling 30/60 plan would pass approximately two lots out of 100 more than the 20/40 plan (5.06 vs. 6.72) at added cost of approximately eight samples per batch (40.4 vs. 31.7) but half the tier 2 testing (17.27 vs. 29.17). If the target interval was expanded to ±25%, the sponsor would be faced with different resource considerations, passing two more lots per 1,000 (0.18 vs. 0.36) at an added cost of nine samples per batch (31.0 vs. 21.8), but approximately one third of the second-tier testing (1.67 vs. 4.51). The added cost may not justify the expected gain.
This simulation was repeated with a grand mean at 100% (not included here) and the same general relationships hold, although the specific numerical results are different.
As described in the “Introduction” and in the first article of this series, FDA has set its regulatory expectations for assessing dose uniformity in OINDP using a parametric tolerance interval approach. In the course of developing a product, the sponsor may discover that after the best possible development of the product, manufacturing process, and analytical methods and after the demonstration of the product′s safety and effectiveness in the clinic, the “default” test parameters may lead to failing batches of acceptable quality. In this situation, the sponsor may consider changing sample size, target interval, and/or other parameters of PTI-TOST in order to consistently accept product that has been shown to be safe and efficacious. This approach will assure that when the quality of future batches is consistent with the quality of batches used to establish safety and efficacy, such batches will be accepted.
For a PTI test, there are a number of PTI-TOST parameters that can be varied to optimize the test for a given product. The effects of changing various parameters were presented in the “Results” section. Figures 12 and and1313 are composites of those results for ease of overall comparisons. Figure 12 presents a combined view of the plots describing the effects of changing parameters for a given batch mean (100% LC) as a function of the batch standard deviation. Figure 13 presents a combined view of the plots describing the effects of changing parameters for a given batch standard deviation of 5 as a function of the batch mean.
Both Figs. 12 and and1313 make it easier to observe trends and their relative impact on the test outcomes. For example, increasing sample size, maximum allowed tail area, or the target interval increases the probability of acceptance in the intermediate region. The results also show that changing the maximum allowable tail area and/or the target interval have the largest impact on the test outcomes (i.e., probability of acceptance for a given batch mean and standard deviation).
This study did not consider changing alpha levels because the value of 0.05 is fairly standard in numerous regulatory applications.
The effect of changing the number of tiers has not been addressed in detail either in an effort to keep this publication to a reasonable length. If the number of tiers is changed, however, the applicant would have to implement measures to maintain the overall alpha level at a preset value, consistent with common standard practice. As one example, in the case of a single-tier test, the overall alpha for the first (and only) tier would be 0.05.
The impact of changes in the PTI-TOST parameters presented here may help guide the design of an appropriate DDU test for a particular product if the “default” test parameters lead to inappropriate failures. The results illustrate that in the intermediate region (where acceptance probability is more than 0% but less than 100%), increasing sample size, target interval, or maximum allowable tail area increases the probability of acceptance of a batch with a given mean and standard deviation. Changing the maximum allowable tail area and/or the target interval has the largest impact on the test outcomes. The appropriateness of any particular set of parameters should be judged in the context of the particular product′s therapeutic goals and characteristics and should be agreed by individual sponsors in collaboration with a regulatory agency.
The authors thank the IPAC-RS Board and the IPAC-RS DDU working group for the consistent interest in this work and helpful feedback during manuscript preparation. The authors are also grateful to FDA for the opportunity to participate in the joint ACPS subgroup and interact to develop a PTI approach for control of dose uniformity in OINDP. Special thanks go to Bo Olsson and Dennis Sandell, whose work and vision inspired the IPAC-RS explorations of improved DDU tests for OINDP. We also acknowledge Walter Hauck for independently confirming certain calculations presented in this article. This article is part of a series which presents a factual description and analysis of the test proposed by the FDA and should not be construed as endorsement or advocacy by the authors or organizations with which they are affiliated.