Introduction
This package assumes that a hierarchical testing procedure for the three-arm gold-standard non-inferiority design is applied. The first test aims to establish assay sensitivity of the trial. It is a test of superiority of the experimental treatment (T) against the placebo treatment (P). If assay sensitivity is successfully established, the treatment is tested for non-inferiority against the control treatment (C). Individual observations are assumed to be normally distributed, where higher values correspond to better treatment effects. Testing is assumed to be done via Z test statistics.
We highly recommend reading our open-access article (Meis et al., 2022) where the theoretical background of this package is explained.
Some examples from the paper
To showcase the capabilities of this package, we will reproduce some results from the paper in the following.
It should be noted that the results will not completely agree with the results from the paper, as the calculations in the paper used much lower error tolerances and more function evaluations.
To achieve results closer to the results from the paper, you can supply the following options, though this will significantly increase computation times:
mvnorm_algorithm = mvtnorm::Miwa(
# steps = 128,
steps = 4097,
checkCorr = FALSE,
maxval = 1000),
nloptr_opts = list(algorithm = "NLOPT_LN_SBPLX",
# xtol_abs = 1e-3,
# xtol_rel = 1e-2,
# maxeval = 2000,
xtol_abs = 1e-10,
xtol_rel = 1e-9,
maxeval = 2000,
print_level = 0)You may also want to put
print_progress = TRUEwhen running code interactively to see the progress of the optimization.
Design from Table 2
The designs from in Table 2 from the paper are optimized to minimize the expected sample size under the alternative hypothesis.
Design 1, \(\beta = 0.2\)
This is (approximately) the first line in Table 2 from the paper:
tab1_D1 <- optimize_design_onestage(
alpha = .025,
beta = .2,
alternative_TP = .4,
alternative_TC = 0,
Delta = .2,
print_progress = FALSE
)
tab1_D1
#> Sample sizes (stage 1): T: 413, P: 125, C: 404
#> Efficacy boundaries (stage 1): Z_TP_e: 1.95996, Z_TC_e: 1.95996
#> Maximum overall sample size: 942
#> Placebo penalty at optimum (kappa * nP): 0.0
#> Objective function value: 942.0
#> Type I error for TP testing: 2.5%
#> Type I error for TC testing: 2.5%
#> Power: 80.2%Design 2, \(\beta = 0.2\)
This is (approximately) the second line in Table 2 from the paper:
optimize_design_twostage(
cP1 = tab1_D1$stagec[[1]]$P, # The allocation ratios are enforced to be
cC1 = tab1_D1$stagec[[1]]$C, # the same as in the optimal single-stage design.
cT2 = 1,
cP2 = tab1_D1$stagec[[1]]$P,
cC2 = tab1_D1$stagec[[1]]$C,
bTP1f = -Inf, # These two boundary conditions enforce no futility stops.
bTC1f = -Inf,
beta = 0.2,
alternative_TP = 0.4,
alternative_TC = 0,
Delta = 0.2,
print_progress = FALSE
)
#> Sample sizes (stage 1): T: 224, P: 68, C: 219
#> Sample sizes (stage 2): T: 224, P: 68, C: 219
#> Efficacy boundaries (stage 1): Z_TP_e: 2.10510, Z_TC_e: 2.27093
#> Futility boundaries (stage 1): Z_TP_f: -Inf, Z_TC_f: -Inf
#> Efficacy boundaries (stage 2): Z_TP_e: 2.27188, Z_TC_e: 2.10568
#> Inverse normal combination test weights (TP): w1: 0.70711, w2: 0.70711
#> Inverse normal combination test weights (TC): w1: 0.70711, w2: 0.70711
#> Maximum overall sample size: 1022
#> Expected sample size (H1): 801.2
#> Expected sample size (H0): 1020.3
#> Expected placebo group sample size (H1): 82.8
#> Expected placebo group sample size (H0): 134.8
#> Objective function value: 801.2
#> (local) type I error for TP testing: 2.50%
#> (local) type I error for TC testing: 2.50%
#> Probability of futility stop (H1): 0.00%
#> Probability of futility stop (H0): 0.00%
#> Minimum conditional power: 0.00%
#> Power: 80.20%
#> Futility boundaries: nonbinding
#> Futility testing method: always both futility testsDesign 3, \(\beta = 0.2\)
This is (approximately) the third line in Table 2 from the paper:
optimize_design_twostage(
bTP1f = -Inf, # These two boundary conditions enforce no futility stops.
bTC1f = -Inf,
beta = 0.2,
alternative_TP = 0.4,
alternative_TC = 0,
Delta = 0.2,
print_progress = FALSE
)
#> Sample sizes (stage 1): T: 229, P: 83, C: 233
#> Sample sizes (stage 2): T: 201, P: 114, C: 182
#> Efficacy boundaries (stage 1): Z_TP_e: 2.04885, Z_TC_e: 2.29026
#> Futility boundaries (stage 1): Z_TP_f: -Inf, Z_TC_f: -Inf
#> Efficacy boundaries (stage 2): Z_TP_e: 2.41637, Z_TC_e: 2.08294
#> Inverse normal combination test weights (TP): w1: 0.67093, w2: 0.74152
#> Inverse normal combination test weights (TC): w1: 0.73968, w2: 0.67295
#> Maximum overall sample size: 1042
#> Expected sample size (H1): 786.2
#> Expected sample size (H0): 1039.2
#> Expected placebo group sample size (H1): 99.2
#> Expected placebo group sample size (H0): 194.7
#> Objective function value: 786.2
#> (local) type I error for TP testing: 2.50%
#> (local) type I error for TC testing: 2.50%
#> Probability of futility stop (H1): 0.00%
#> Probability of futility stop (H0): 0.00%
#> Minimum conditional power: 0.00%
#> Power: 80.17%
#> Futility boundaries: nonbinding
#> Futility testing method: always both futility testsDesign 4, \(\beta = 0.2\)
This is (approximately) the fourth line in Table 2 from the paper:
optimize_design_twostage(
beta = 0.2,
alternative_TP = 0.4,
alternative_TC = 0,
Delta = 0.2,
print_progress = FALSE,
binding_futility = FALSE
)
#> Sample sizes (stage 1): T: 229, P: 90, C: 234
#> Sample sizes (stage 2): T: 205, P: 108, C: 198
#> Efficacy boundaries (stage 1): Z_TP_e: 2.04187, Z_TC_e: 2.27093
#> Futility boundaries (stage 1): Z_TP_f: -0.13512, Z_TC_f: 0.45510
#> Efficacy boundaries (stage 2): Z_TP_e: 2.42507, Z_TC_e: 2.09677
#> Inverse normal combination test weights (TP): w1: 0.68947, w2: 0.72432
#> Inverse normal combination test weights (TC): w1: 0.73143, w2: 0.68192
#> Maximum overall sample size: 1064
#> Expected sample size (H1): 777.6
#> Expected sample size (H0): 670.2
#> Expected placebo group sample size (H1): 101.5
#> Expected placebo group sample size (H0): 113.7
#> Objective function value: 777.6
#> (local) type I error for TP testing: 2.50%
#> (local) type I error for TC testing: 2.45%
#> Probability of futility stop (H1): 4.53%
#> Probability of futility stop (H0): 76.64%
#> Minimum conditional power: 18.04%
#> Power: 80.00%
#> Futility boundaries: nonbinding
#> Note: Results are presented as if futility boundaries were strictly obeyed.
#> Futility testing method: always both futility testsDesign 5, \(\beta = 0.2\)
This is (approximately) the fourth line in Table 2 from the paper:
optimize_design_twostage(
beta = 0.2,
alternative_TP = 0.4,
alternative_TC = 0,
Delta = 0.2,
print_progress = FALSE,
binding_futility = TRUE
)
#> Sample sizes (stage 1): T: 239, P: 84, C: 242
#> Sample sizes (stage 2): T: 205, P: 132, C: 193
#> Efficacy boundaries (stage 1): Z_TP_e: 2.02371, Z_TC_e: 2.25877
#> Futility boundaries (stage 1): Z_TP_f: -0.18467, Z_TC_f: 0.84015
#> Efficacy boundaries (stage 2): Z_TP_e: 2.51188, Z_TC_e: 2.05525
#> Inverse normal combination test weights (TP): w1: 0.65467, w2: 0.75592
#> Inverse normal combination test weights (TC): w1: 0.73976, w2: 0.67287
#> Maximum overall sample size: 1095
#> Expected sample size (H1): 765.2
#> Expected sample size (H0): 644.4
#> Expected placebo group sample size (H1): 97.4
#> Expected placebo group sample size (H0): 102.8
#> Objective function value: 765.2
#> (local) type I error for TP testing: 2.50%
#> (local) type I error for TC testing: 2.50%
#> Probability of futility stop (H1): 8.82%
#> Probability of futility stop (H0): 84.63%
#> Minimum conditional power: 31.20%
#> Power: 80.06%
#> Futility boundaries: binding
#> Futility testing method: always both futility testsDesign from Table 3
Next, we will optimize a design under a combination of null and alternative hypothesis.
Design 5, \(\beta = 0.2\), \(\lambda = 0.9\)
This is (approximately) the third line in Table 3 from the paper:
optimize_design_twostage(
beta = 0.2,
alternative_TP = 0.4,
alternative_TC = 0,
Delta = 0.2,
print_progress = FALSE,
binding_futility = TRUE,
lambda = 0.9
)
#> Sample sizes (stage 1): T: 225, P: 84, C: 225
#> Sample sizes (stage 2): T: 224, P: 100, C: 225
#> Efficacy boundaries (stage 1): Z_TP_e: 2.04545, Z_TC_e: 2.27674
#> Futility boundaries (stage 1): Z_TP_f: 0.82000, Z_TC_f: 0.71736
#> Efficacy boundaries (stage 2): Z_TP_e: 2.37035, Z_TC_e: 2.05812
#> Inverse normal combination test weights (TP): w1: 0.68406, w2: 0.72942
#> Inverse normal combination test weights (TC): w1: 0.70754, w2: 0.70667
#> Maximum overall sample size: 1083
#> Expected sample size (H1): 771.2
#> Expected sample size (H0): 578.9
#> Expected placebo group sample size (H1): 94.6
#> Expected placebo group sample size (H0): 91.3
#> Objective function value: 757.6
#> (local) type I error for TP testing: 2.50%
#> (local) type I error for TC testing: 2.50%
#> Probability of futility stop (H1): 8.73%
#> Probability of futility stop (H0): 91.50%
#> Minimum conditional power: 41.59%
#> Power: 80.12%
#> Futility boundaries: binding
#> Futility testing method: always both futility testsDesign from Table 4
Now we will optimize a design under the alternative while putting an extra penalty on placebo group sample size.
Design 5, \(\beta = 0.2\), \(\kappa = 0.5\)
This is (approximately) the fourth line in Table 2 from the paper:
optimize_design_twostage(
beta = 0.2,
alternative_TP = 0.4,
alternative_TC = 0,
Delta = 0.2,
print_progress = FALSE,
binding_futility = TRUE,
kappa = 0.5
)
#> Sample sizes (stage 1): T: 238, P: 74, C: 240
#> Sample sizes (stage 2): T: 214, P: 115, C: 194
#> Efficacy boundaries (stage 1): Z_TP_e: 2.03307, Z_TC_e: 2.27081
#> Futility boundaries (stage 1): Z_TP_f: -0.11340, Z_TC_f: 0.82841
#> Efficacy boundaries (stage 2): Z_TP_e: 2.47697, Z_TC_e: 2.04892
#> Inverse normal combination test weights (TP): w1: 0.65101, w2: 0.75907
#> Inverse normal combination test weights (TC): w1: 0.73478, w2: 0.67831
#> Maximum overall sample size: 1075
#> Expected sample size (H1): 767.2
#> Expected sample size (H0): 628.3
#> Expected placebo group sample size (H1): 89.4
#> Expected placebo group sample size (H0): 90.0
#> Objective function value: 811.8
#> (local) type I error for TP testing: 2.50%
#> (local) type I error for TC testing: 2.50%
#> Probability of futility stop (H1): 8.79%
#> Probability of futility stop (H0): 85.07%
#> Minimum conditional power: 31.27%
#> Power: 80.02%
#> Futility boundaries: binding
#> Futility testing method: always both futility testsDesign from Table 5
Next, we will optimize a design under a combination of null and alternative hypothesis while including a penalty on the placebo group sample size.
Design 5, \(\beta = 0.2\), \(\lambda = 0.9\), \(\kappa = 1\)
This is (approximately) the seventh line in Table 2 from the paper:
optimize_design_twostage(
beta = 0.2,
alternative_TP = 0.4,
alternative_TC = 0,
Delta = 0.2,
print_progress = FALSE,
binding_futility = TRUE,
lambda = .9,
kappa = 1
)
#> Sample sizes (stage 1): T: 242, P: 70, C: 233
#> Sample sizes (stage 2): T: 217, P: 97, C: 214
#> Efficacy boundaries (stage 1): Z_TP_e: 2.04651, Z_TC_e: 2.30167
#> Futility boundaries (stage 1): Z_TP_f: 0.73905, Z_TC_f: 0.81077
#> Efficacy boundaries (stage 2): Z_TP_e: 2.38259, Z_TC_e: 2.03241
#> Inverse normal combination test weights (TP): w1: 0.66460, w2: 0.74720
#> Inverse normal combination test weights (TC): w1: 0.72394, w2: 0.68986
#> Maximum overall sample size: 1073
#> Expected sample size (H1): 775.9
#> Expected sample size (H0): 585.7
#> Expected placebo group sample size (H1): 84.0
#> Expected placebo group sample size (H0): 76.8
#> Objective function value: 846.2
#> (local) type I error for TP testing: 2.50%
#> (local) type I error for TC testing: 2.50%
#> Probability of futility stop (H1): 9.58%
#> Probability of futility stop (H0): 92.01%
#> Minimum conditional power: 41.71%
#> Power: 80.06%
#> Futility boundaries: binding
#> Futility testing method: always both futility testsOther options
Penalizing the maximum sample size
optimize_design_twostage(
beta = 0.2,
alternative_TP = 0.4,
alternative_TC = 0,
Delta = 0.2,
print_progress = FALSE,
eta = 1
)
#> Sample sizes (stage 1): T: 227, P: 82, C: 256
#> Sample sizes (stage 2): T: 194, P: 56, C: 150
#> Efficacy boundaries (stage 1): Z_TP_e: 2.28123, Z_TC_e: 2.53039
#> Futility boundaries (stage 1): Z_TP_f: -0.23040, Z_TC_f: -0.04295
#> Efficacy boundaries (stage 2): Z_TP_e: 2.07884, Z_TC_e: 2.00081
#> Inverse normal combination test weights (TP): w1: 0.76268, w2: 0.64678
#> Inverse normal combination test weights (TC): w1: 0.76299, w2: 0.64641
#> Maximum overall sample size: 965
#> Expected sample size (H1): 801.9
#> Expected sample size (H0): 710.3
#> Expected placebo group sample size (H1): 93.1
#> Expected placebo group sample size (H0): 101.9
#> Objective function value: 1766.9
#> (local) type I error for TP testing: 2.50%
#> (local) type I error for TC testing: 2.49%
#> Probability of futility stop (H1): 1.30%
#> Probability of futility stop (H0): 63.48%
#> Minimum conditional power: 5.00%
#> Power: 80.11%
#> Futility boundaries: nonbinding
#> Note: Results are presented as if futility boundaries were strictly obeyed.
#> Futility testing method: always both futility testsOptimizing between group allocation with enforced between stage allocation at 1
optimize_design_twostage(
cT2 = 1, # These three boundary conditions enforce a
cP2 = quote(cP1), # between-stage allocation ratio of one.
cC2 = quote(cC1), # The quote() command is necessary for this to work.
bTP1f = -Inf, # These two boundary conditions enforce no futility stops.
bTC1f = -Inf,
beta = 0.2,
alternative_TP = 0.4,
alternative_TC = 0,
Delta = 0.2,
print_progress = FALSE
)
#> Sample sizes (stage 1): T: 217, P: 87, C: 212
#> Sample sizes (stage 2): T: 217, P: 87, C: 212
#> Efficacy boundaries (stage 1): Z_TP_e: 2.06549, Z_TC_e: 2.28000
#> Futility boundaries (stage 1): Z_TP_f: -Inf, Z_TC_f: -Inf
#> Efficacy boundaries (stage 2): Z_TP_e: 2.34934, Z_TC_e: 2.10025
#> Inverse normal combination test weights (TP): w1: 0.70711, w2: 0.70711
#> Inverse normal combination test weights (TC): w1: 0.70711, w2: 0.70711
#> Maximum overall sample size: 1032
#> Expected sample size (H1): 789.8
#> Expected sample size (H0): 1029.7
#> Expected placebo group sample size (H1): 99.0
#> Expected placebo group sample size (H0): 172.3
#> Objective function value: 789.8
#> (local) type I error for TP testing: 2.50%
#> (local) type I error for TC testing: 2.50%
#> Probability of futility stop (H1): 0.00%
#> Probability of futility stop (H0): 0.00%
#> Minimum conditional power: 0.00%
#> Power: 80.09%
#> Futility boundaries: nonbinding
#> Futility testing method: always both futility testsUsing a custom objective function
You can replace the default objective function by any quoted
expression. In the following example, we optimize the design parameters
to minimize the expected squared sample size under the alternative
hypothesis. These expressions can make use of internal objects created
in the objective evaluation methods, check out the source code of
optimize_design_twostage in the
optimization_methods.R file for more information.
ASN, ASNP, n and
final_state_probs could be useful object for crafting a
custom objective function.
optimize_design_twostage(
beta = 0.2,
alternative_TP = 0.4,
alternative_TC = 0,
Delta = 0.2,
print_progress = FALSE,
objective = quote((final_state_probs[["H1"]][["TP1E_TC1E"]] + final_state_probs[["H1"]][["TP1F_TC1F"]]) *
(n[[1]][["T"]] + n[[1]][["P"]] + n[[1]][["C"]])^2 +
(final_state_probs[["H1"]][["TP1E_TC12E"]] + final_state_probs[["H1"]][["TP1E_TC12F"]]) *
(n[[1]][["T"]] + n[[1]][["P"]] + n[[1]][["C"]] + n[[2]][["T"]] + n[[2]][["C"]])^2 +
(final_state_probs[["H1"]][["TP12F_TC1"]] + final_state_probs[["H1"]][["TP12E_TC12E"]] +
final_state_probs[["H1"]][["TP12E_TC12F"]]) *
(n[[1]][["T"]] + n[[1]][["P"]] + n[[1]][["C"]] + n[[2]][["T"]] + n[[2]][["P"]] + n[[2]][["C"]])^2)
)
#> Sample sizes (stage 1): T: 251, P: 86, C: 272
#> Sample sizes (stage 2): T: 171, P: 115, C: 148
#> Efficacy boundaries (stage 1): Z_TP_e: 2.04280, Z_TC_e: 2.30831
#> Futility boundaries (stage 1): Z_TP_f: -0.36770, Z_TC_f: 0.63122
#> Efficacy boundaries (stage 2): Z_TP_e: 2.42546, Z_TC_e: 2.05589
#> Inverse normal combination test weights (TP): w1: 0.68470, w2: 0.72882
#> Inverse normal combination test weights (TC): w1: 0.78770, w2: 0.61607
#> Maximum overall sample size: 1043
#> Expected sample size (H1): 778.4
#> Expected sample size (H0): 699.4
#> Expected placebo group sample size (H1): 98.5
#> Expected placebo group sample size (H0): 109.0
#> Objective function value: 636683.9
#> (local) type I error for TP testing: 2.50%
#> (local) type I error for TC testing: 2.45%
#> Probability of futility stop (H1): 4.91%
#> Probability of futility stop (H0): 78.74%
#> Minimum conditional power: 13.92%
#> Power: 80.08%
#> Futility boundaries: nonbinding
#> Note: Results are presented as if futility boundaries were strictly obeyed.
#> Futility testing method: always both futility testsReferences
Meis, J, Pilz, M, Herrmann, C, Bokelmann, B, Rauch, G, Kieser, M. Optimization of the two-stage group sequential three-arm gold-standard design for non-inferiority trials. Statistics in Medicine. 2023; 42( 4): 536– 558. doi:10.1002/sim.9630.