PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of aapspharmspringer.comThis journalToc AlertsSubmit OnlineOpen Choice
 
AAPS PharmSciTech. 2009 September; 10(3): 841.
Published online 2009 June 24. doi:  10.1208/s12249-009-9269-3
PMCID: PMC2802172

A Two One-Sided Parametric Tolerance Interval Test for Control of Delivered Dose Uniformity—Part 2—Effect of Changing Parameters

Abstract

This article examines the effects of changing parameters in the test which was proposed by the FDA at the October 2005 Advisory Committee meeting for confirming delivered dose uniformity in orally inhaled and nasal drug products. This article is an extension of the characterization study presented in an accompanying article (Part 1). The goal of this study is to understand how parameters of the test affect the test performance. The effects of changing test parameters such as target interval, maximum allowable proportion in the tail area, and sample size are examined. The results show that changing the maximum allowable tail area and/or the target interval have the largest impact on the test outcomes, i.e., probability of acceptance for a given batch mean and standard deviation. The presented information may provide potential users of the test with a set of tools for optimizing the test characteristics for a particular product.

Key words: interval, parametric, tolerance, TOST, uniformity

INTRODUCTION

This paper is the second in a series examining the two one-sided tests (TOST) based on the parametric tolerance interval (PTI) approach as proposed by FDA at the October 2005 meeting of the Advisory Committee for Pharmaceutical Science (1). The first paper of this three-part series focused on the details and performance characteristics of the “default” test described by FDA as “95% confidence level, 87.5% coverage, target interval=80–120% of the label claim (LC), sample size=20 + 40 DDU observations in the 1st + 2nd tier, each inhaler being tested in the beginning and end of container life.” However, the FDA presentation included the possibility that other sampling plans and test protocols (e.g., other sample sizes, number of tiers, size ratios between tiers 1 and 2, etc.) can be considered. In order to facilitate assessment of various test options and to raise awareness of the consequences of changing test parameters, the present paper studies the consequences of various options that might be chosen for the PTI-TOST in the course of developing a product and meaningful acceptance criteria for DDU of orally inhaled and nasal drug products (OINDPs). As an aid to the reader, a list of Notations and Abbreviations is included at the end of this article.

MATERIALS AND METHODS

The PTI-TOST proposed by the FDA was described in detail in the first paper of this series and is briefly summarized below. For the operating characteristic (OC) curves, statistical simulations were used based on the Monte-Carlo technique by sampling with replacement from a normal distribution and following the specified sampling scheme. Other assumptions and notations are summarized in Part 1. The test parameters that were varied are: the maximum allowed tail area (PmaxTA), total sample size (N), the ratio of sample sizes in the first and second tier (N1/N2) and the target interval [L, U]. The overall significance level (α) and the structure of the test are not changed.

The “default” PTI-TOST was described in detail in the first paper of this series and is summarized here for convenience, as discussed below.

Tier 1: Collect 20 doses from 10 multi-dose OINDP units (a beginning-of-unit (BOU) and an end-of-unit (EOU) determination from each unit). The 20 observations must pass the following criteria:

  1. TL1 =equation M1 with PmaxTA = 6.25% and α1 = 0.0226 where K1 is listed in Table I.
    Table I
    K Coefficients for the PTI-TOST for Several PmaxTA and Sample Sizes, with α1 = 0.0226 and α2 = 0.0340
  2. TU1 =equation M2 with PmaxTA = 6.25% and α1 = 0.0226 where K1 is listed in Table I.
  3. 85  equation M3  115
  4. 85  equation M4  115

If the sample fails any of the criteria (1a–1d), the test proceeds to the second tier. In Tier 2, collect an additional 40 observations and repeat the steps above with N = 60 and K2 substituted for K1. In Table I, K1 and K2 are given for several sample size and PmaxTA options when α1 = 0.0226 and α2 = 0.0340, which were recommended by FDA based on the Pocock method (see the Appendix to Part 1 of this series) and match the K values of the FDA proposal (2). The value of (100-2*PmaxTA)% is given in Table I for ease of comparison with the coefficients presented by FDA, where this value is called “coverage.”

The data used in the section “Considerations for a Specific Product” were simulated based on a real product but adjusted to fit the assumptions of the PTI-TOST (i.e., normal distribution, BOU, and EOU means are the same, BOU and EOU variances are the same, and there is no correlation between the two). For this example, the grand mean was set at 98% LC for illustrative purposes. In summary, the data were generated so that, within a batch, BOU and EOU have equal means and variances and

BOU
= (batch-to-batch variability) + 98 + (within-batch variability)
EOU
= (batch-to-batch variability) + 98 + (within-batch variability)

For each batch, a single value was generated for (batch-to-batch variability) as normally distributed random variable with mean = 0 and standard deviation = 4.

Within a batch, 10 values were generated for BOU and 10 values for EOU for (within-batch variability) as a normally distributed random variable with mean = 0 and standard deviation = 6.

A total of 25 batches were created.

RESULTS

Changing Total Sample Size, Maximum Allowable Tail Area, and Target Interval

The following figures illustrate the effects of changing various test parameters on the acceptance probability of the PTI-TOST. In each plot, there are certain regions within the plot where acceptance probability is either 100% or 0%, the OC curves overlap, and changing of the test parameters does not matter. The description of results will focus on the intermediate region where acceptance probability is more than 0% but less than 100%.

Figure 1 (for the constant mean) shows that as the total sample size increases, the shape (steepness and position) of the OC curves approaches the “ideal” shape for an OC curve, which is a step function (a true step-function OC curve will rarely occur in practice, since the entire population is rarely tested, especially with destructive tests). As the total sample size increases, the OC curves become steeper and shift to the right so that in the intermediate region, the probability of acceptance increases for a given standard deviation. For example, in the 1st tier, for an on-target batch with sigma = 7, probability of acceptance is ~0.3 when N1 = 10 but rises to ~0.95 when N1 = 30. Conversely, in tier 1, for acceptance probability = 0.8, sample size N1 = 10 allows a sigma of ~5 but N2 = 30 allows a sigma of ~7.5.

Fig. 1
OC curves illustrating the effect of changing the total sample size (N) keeping the ratio N 1/N 2 constant, as a function of the batch sigma, for a batch with the mean at 100% LC. The left panel is for tier 1 and the right panel is for the overall test ...

Similarly, Fig. 2 (for the constant sigma) shows that in the intermediate region, as the total sample size increases, the probability of acceptance increases for a given batch mean. For example, in the 1st tier, for a batch mean of ~95% LC, probability of acceptance is ~0.5 when N1 = 10 but rises to ~1 when N1 = 30. Conversely, for a given probability of acceptance, an increased sample size allows a larger deviation of the mean from the target. For example, in tier 1, for acceptance probability = 0.8, sample size N1 = 10 allows an off-target deviation of ~2% but N1 = 30 allows an off-target deviation of ~7%.

Fig. 2
The effect of changing the total sample size (N) keeping the ratio N 1/N 2 constant, as a function of the batch mean, for a batch standard deviation of 5. The left panel is for tier 1 and the right panel is for the overall test (i.e., tier 1 and tier ...

Table II illustrates the effect of changing sample size on the batch coverage required to pass with two specific acceptance probability values (99% and 5%). Both Figs. 1 and and22 and Table II reflect a basic property of PTI tests, namely that an increased sample size allows a larger standard deviation and/or mean deviation from the target.

Table II
Coverage Requirements for 99% and 5% Acceptance Probability for PTI-TOST with Different Sample Sizes

Figure 3 (for the constant mean) shows that in the intermediate region, as the maximum allowable tail area increases, the probability of acceptance increases for a given batch standard deviation. For example, in tier 2, for sigma = 10, PmaxTA = 2.5, probability of acceptance is ~0.01 while for PmaxTA = 10%, probability of acceptance rises to 0.94. Conversely, for a given probability of acceptance, increased PmaxTA allows a larger sigma. For example, in the right panel for the overall test, for acceptance probability = 0.8, PmaxTA = 2.5 % allows a sigma of ~7.3, while PmaxTA = 10.0% allows a sigma of ~10.5.

Fig. 3
The effect of changing the maximum allowable tail area for a batch with the mean at 100% LC, as a function of the batch standard deviation. The left panel is for tier 1 and the right panel is for the overall test (i.e., tier 1 and tier 2 combined)

Figure 4 (for the constant standard deviation) shows that in the intermediate region, as the maximum allowable tail area increases, the probability of acceptance increases for a given batch mean. For example, in tier 2, for a mean of ~90% LC, for PmaxTA = 2.5, probability of acceptance is ~0.05, while for PmaxTA = 10%, probability of acceptance rises to 0.95. Conversely, for a given probability of acceptance, increased PmaxTA allows a larger deviation of the mean from 100% LC. For example, in tier 2, for acceptance probability = 0.8, PmaxTA = 2.5 % allows a mean of ~93% LC, while PmaxTA = 10.0% allows a batch mean of ~89% LC.

Fig. 4
The effect of changing the maximum allowable tail area for a batch with a sigma of 5, as a function of the batch mean. The left panel is for tier 1 and the right panel is for the overall test (i.e., tier 1 and tier 2 combined)

Figure 5 (for the constant mean) shows that in the intermediate region, as the target interval increases, the probability of acceptance increases for a given batch standard deviation. For example, in tier 2, a batch with sigma = 10 and a target interval of 100 ± 15% LC has a probability of acceptance ~0 while for the target interval 100 ± 30% LC the probability of acceptance rises to ~1. Conversely, for a given probability of acceptance, an increased target interval allows a larger sigma. For example, in tier 2, for acceptance probability = 0.8, the target interval of 100 ± 15% LC allows a sigma of ~7, while the target interval of 100 ± 30% LC allows a sigma of ~13.5.

Fig. 5
The effect of changing the target interval for a batch with the mean at 100% LC, as a function of the batch standard deviation. The left panel is for tier 1 and the right panel is for the overall test (i.e., tier 1 and tier 2 combined)

Figure 6 (for the constant standard deviation) shows that in the intermediate region, as the target interval increases, the probability of acceptance increases for a given batch mean. For example, in tier 2, for a target interval 100 ± 15% LC and a batch mean of 90% LC, probability of acceptance is ~0, while with the target interval of 100 ± 30% LC, the probability of acceptance rises to ~1. Conversely, for a given probability of acceptance, increased target interval allows a larger deviation of the mean from 100% LC. For example, in tier 2, for acceptance probability = 0.8, the target interval of 100 ± 15% LC allows a mean of ~95% LC, while the target interval of 100 ± 30% LC allows a batch mean of ~86% LC.

Fig. 6
The effect of changing the target interval for a batch with a sigma of 5, as a function of the batch mean. The left panel is for tier 1 and the right panel is for the overall test (i.e., tier 1 and tier 2 combined)

Considerations for a Specific Product

The previous section discussed the impact of changing total sample size, maximum allowable tail area, and target interval from a theoretical perspective. The OC curves illustrated the probability of accepting a single lot with certain characteristics. Given a product of certain quality, a sponsor may be interested in estimating the long-term conformance rate, tier 2 testing frequency, and expected average sample size for a number of sampling scenarios. This information can be used to choose an appropriate sampling scheme for the product that would allow the most efficient use of resources without compromising the test′s ability to manage product quality.

For this case study, simulated data (similar to that available from a typical product development program) were generated with an overall mean of 98.0, a batch-to-batch standard deviation of 4.0 (which causes batch means to deviate from 98.0), a within-batch standard deviation of 6.0, and an assumption that no difference in BOU and EOU means exists. A sampling of data from 25 batches is shown in Fig. 7. If a large number of batches are generated (e.g., 100,000), long-term conformance (pass/fail) rates, tier 2 frequency, and expected average sample size can be estimated for various sampling schemes, maximum allowable tail area, and target interval combinations. The overall significance level is assumed to be α = 0.05 and the distribution of α in the tiers was the same as in the first article of this series. Four sampling plans, three maximum allowable tail areas, and three target intervals were considered and the simulation results are given in Table III. The plans are sorted by nonconformance rate for each tail area or target interval combination.

Fig. 7
Simulated DDU observations. Different symbols show the beginning (crosses) and end (empty circles) life stages. The dashed line connects the batch means. Batches are sorted by batch means to better illustrate the range and distribution over that range ...
Table III
Parametric Tolerance Interval Simulation Results

Figures 8 and and99 illustrate the scenarios for a maximum allowable tail area of 6.25%. Nonconformance (fail) rates will fall with larger overall sample sizes (Fig. 8). The expected nonconformance rates when the target limit is ±20% are likely higher than most sponsors would consider acceptable (Fig. 8) for any sampling scheme. The 20/40 plan had slightly lower nonconformance rates (Fig. 8) and lower expected average sample sizes (Fig. 9) than the 30/30 plan.

Fig. 8
Nonconformance (fail) rates as a function of the width of the target interval. The lines connecting data points are only serving as a visual aid and should not be used for interpolating results for intermediate target intervals
Fig. 9
Expected N (long-term average sample size) as a function of the width of the target interval. The lines connecting data points are only serving as a visual aid and should not be used for interpolating results for intermediate target intervals

Simply plotting the batch data against the maximum allowable standard deviation for likely sample means will draw similar conclusions about the sampling plans. Figures 10 and and1111 illustrate the 25 batches against the passing criteria for 100 ± 20% limits and maximum allowable tail area of 6.25%. Data points above the criteria would fail the PTI tier requirements. A single batch outside the tier 2 criteria represents an estimated 4% nonconformance rate (1/25). Sponsors would not have to resort to sophisticated simulations to get a rough evaluation of the sampling options.

Fig. 10
Simulated batch characteristics versus maximum standard deviation criteria to pass first-tier testing
Fig. 11
Simulated batch characteristics versus maximum standard deviation criteria to pass second-tier testing

Simulations may provide information that would support more detailed resource decisions. The sample plan decision is dependent upon which limits are ultimately assigned to this product. For instance, if the target interval is set at ±20%, the larger sampling 30/60 plan would pass approximately two lots out of 100 more than the 20/40 plan (5.06 vs. 6.72) at added cost of approximately eight samples per batch (40.4 vs. 31.7) but half the tier 2 testing (17.27 vs. 29.17). If the target interval was expanded to ±25%, the sponsor would be faced with different resource considerations, passing two more lots per 1,000 (0.18 vs. 0.36) at an added cost of nine samples per batch (31.0 vs. 21.8), but approximately one third of the second-tier testing (1.67 vs. 4.51). The added cost may not justify the expected gain.

This simulation was repeated with a grand mean at 100% (not included here) and the same general relationships hold, although the specific numerical results are different.

DISCUSSION

As described in the “Introduction” and in the first article of this series, FDA has set its regulatory expectations for assessing dose uniformity in OINDP using a parametric tolerance interval approach. In the course of developing a product, the sponsor may discover that after the best possible development of the product, manufacturing process, and analytical methods and after the demonstration of the product′s safety and effectiveness in the clinic, the “default” test parameters may lead to failing batches of acceptable quality. In this situation, the sponsor may consider changing sample size, target interval, and/or other parameters of PTI-TOST in order to consistently accept product that has been shown to be safe and efficacious. This approach will assure that when the quality of future batches is consistent with the quality of batches used to establish safety and efficacy, such batches will be accepted.

For a PTI test, there are a number of PTI-TOST parameters that can be varied to optimize the test for a given product. The effects of changing various parameters were presented in the “Results” section. Figures 12 and and1313 are composites of those results for ease of overall comparisons. Figure 12 presents a combined view of the plots describing the effects of changing parameters for a given batch mean (100% LC) as a function of the batch standard deviation. Figure 13 presents a combined view of the plots describing the effects of changing parameters for a given batch standard deviation of 5 as a function of the batch mean.

Fig. 12
A summary of changing parameters on the outcome of the FDA DDU test, for a batch on target, as a function of batch standard deviation. The left panel represents tier 1 and the right panel represents the overall test (i.e., tiers 1 and 2 combined)
Fig. 13
A summary of effects of changing parameters on the outcome of the FDA DDU test, for a batch with sigma 5, as a function of the batch mean. The left panel represents tier 1 and the right panel represents the overall test (i.e., tiers 1 and 2 combined)

Both Figs. 12 and and1313 make it easier to observe trends and their relative impact on the test outcomes. For example, increasing sample size, maximum allowed tail area, or the target interval increases the probability of acceptance in the intermediate region. The results also show that changing the maximum allowable tail area and/or the target interval have the largest impact on the test outcomes (i.e., probability of acceptance for a given batch mean and standard deviation).

This study did not consider changing alpha levels because the value of 0.05 is fairly standard in numerous regulatory applications.

The effect of changing the number of tiers has not been addressed in detail either in an effort to keep this publication to a reasonable length. If the number of tiers is changed, however, the applicant would have to implement measures to maintain the overall alpha level at a preset value, consistent with common standard practice. As one example, in the case of a single-tier test, the overall alpha for the first (and only) tier would be 0.05.

CONCLUSIONS

The impact of changes in the PTI-TOST parameters presented here may help guide the design of an appropriate DDU test for a particular product if the “default” test parameters lead to inappropriate failures. The results illustrate that in the intermediate region (where acceptance probability is more than 0% but less than 100%), increasing sample size, target interval, or maximum allowable tail area increases the probability of acceptance of a batch with a given mean and standard deviation. Changing the maximum allowable tail area and/or the target interval has the largest impact on the test outcomes. The appropriateness of any particular set of parameters should be judged in the context of the particular product′s therapeutic goals and characteristics and should be agreed by individual sponsors in collaboration with a regulatory agency.

Acknowledgements

The authors thank the IPAC-RS Board and the IPAC-RS DDU working group for the consistent interest in this work and helpful feedback during manuscript preparation. The authors are also grateful to FDA for the opportunity to participate in the joint ACPS subgroup and interact to develop a PTI approach for control of dose uniformity in OINDP. Special thanks go to Bo Olsson and Dennis Sandell, whose work and vision inspired the IPAC-RS explorations of improved DDU tests for OINDP. We also acknowledge Walter Hauck for independently confirming certain calculations presented in this article. This article is part of a series which presents a factual description and analysis of the test proposed by the FDA and should not be construed as endorsement or advocacy by the authors or organizations with which they are affiliated.

Abbreviations:

ACPS
Advisory Committee for Pharmaceutical Science
BOU
beginning of unit
CMC
chemistry, manufacturing, and controls
DDU
delivered dose uniformity
DPI
dry powder inhaler
EOU
end of unit
IPAC-RS
International Pharmaceutical Aerosol Consortium on Regulation and Science
Intermediate
region of the OC curve where acceptance probability is more than 0% but less than 100%
LC
label claim
MDI
metered dose inhaler
OC
operating characteristic
OINDP
orally inhaled and nasal drug products
PmaxTA
maximum allowable proportion of doses in a tail area (left or right) outside the target interval
PTI
parametric tolerance interval
TOST
two one-sided tests
μ
population (batch) mean
N
total sample size
N1
sample size in the first tier
N2
additional units tested in the second tier
s
sample standard deviation
σ
population (batch) standard deviation
equation M5
sample mean

References

1. Nasr MM. Parametric Tolerance Interval Test for Delivered Dose Uniformity (DDU). Presentation to Advisory Committee for Pharmaceutical Science on 25 October 2005. http://www.fda.gov./ohrms/dockets/ac/05/slides/2005-4187S1_13_Nasr.ppt. Accessed May 26, 2009.
2. Golden M. Parametric Tolerance Interval (PTI) Test for Delivered Dose Uniformity (DDU) for Orally Inhaled and Nasal Drug Products (OINDP). Presentation to Advisory Committee for Pharmaceutical Science on 26 October 2005. http://www.fda.gov/ohrms/dockets/ac/05/slides/2005-4187S2_10_Golden.ppt Accessed May 26, 2009.

Articles from AAPS PharmSciTech are provided here courtesy of American Association of Pharmaceutical Scientists