Multi-arm survival designs

Daniel Sabanés Bové, Friedrich Pahlke

RPACT

June 9, 2026

Overview

• This example is adapted from the binary multi-arm example in the vignette written by Stephen Schueuerhuis, Gernot Wassmer and Friedrich Pahlke
• We look at implementing MAMS survival designs with futility bounds, sample size calculations and simulations
• We will try different treatment selection approaches

Design specification

• Multi-arm, multi-stage (MAMS) design:
  • 3 arms total: 2 active treatment arms + 1 control arm
  • 3 stages
• One-sided global \(\alpha = 0.025\).
• Pairwise power target: \(80\%\).
• O’Brien-Fleming type alpha-spending boundaries.
• Equal information rates for planning.

But first we will need to prepare the basics on the pairwise comparison level.

Pairwise design preparation

Futility strategy on treatment-effect scale

We target stage-wise futility in terms of hazard ratio (HR):

• Stage 1: stop an arm if observed HR is not better than 0.95.
• Stage 2: stop an arm if observed HR is not better than 0.90.

Because design input uses \(z\)-scale futility bounds, we calibrate \(z\) values to match these approximate HR thresholds.

Initialize design with candidate futility bounds

library(rpact)
futStage1 <- 0.15
futStage2 <- 0.41

designFut <- getDesignGroupSequential(
    kMax = 3,
    alpha = 0.025,
    beta = 0.2,
    sided = 1,
    typeOfDesign = "asOF",
    informationRates = c(1 / 3, 2 / 3, 1),
    futilityBounds = c(futStage1, futStage2)
)
designFut

Design parameters and output of group sequential design

User defined parameters

• Type of design: O’Brien & Fleming type alpha spending
• Futility bounds (non-binding): 0.150, 0.410

Derived from user defined parameters

• Maximum number of stages: 3

Default parameters

• Stages: 1, 2, 3
• Information rates: 0.333, 0.667, 1.000
• Significance level: 0.0250
• Type II error rate: 0.2000
• Binding futility: FALSE
• Test: one-sided
• Tolerance: 1e-08
• Type of beta spending: none

Output

• Cumulative alpha spending: 0.0001035, 0.0060484, 0.0250000
• Critical values: 3.710, 2.511, 1.993
• Stage levels (one-sided): 0.0001035, 0.0060122, 0.0231281

Translate candidate bounds to survival effect scale

For calibration we assume:

• Control median survival: 12 months.
• Target hazard ratio: 0.70.
• 5% dropout after 12 months.
• Uniform accrual over 18 months and 12 months follow-up.

designFut |> getSampleSizeSurvival(
    hazardRatio = 0.7,
    median2 = 12,
    dropoutRate1 = 0.05,
    dropoutRate2 = 0.05,
    dropoutTime = 12,
    accrualTime = 18,
    followUpTime = 12
) |> fetch(futilityBoundsEffectScale)

$futilityBoundsEffectScale
          [,1]
[1,] 0.9687222
[2,] 0.9404301

Tune futility bounds with one-dimensional search

We tune each stage approximately using stats::uniroot().

targetHr1 <- 0.95
targetHr2 <- 0.90

fStage1 <- uniroot(
    f = function(z) {
        ssTmp <- getDesignGroupSequential(
            kMax = 3,
            alpha = 0.025,
            beta = 0.2,
            sided = 1,
            typeOfDesign = "asOF",
            informationRates = c(1 / 3, 2 / 3, 1),
            futilityBounds = c(z, -6)
        ) |>
            getSampleSizeSurvival(
                hazardRatio = 0.7,
                median2 = 12,
                dropoutRate1 = 0.05,
                dropoutRate2 = 0.05,
                dropoutTime = 12,
                accrualTime = 18,
                followUpTime = 12
            )

        futVals <- as.numeric(ssTmp$futilityBoundsEffectScale)

        futVals[1] - targetHr1
    },
    interval = c(-1, 1)
)$root

fStage2 <- uniroot(
    f = function(z) {
        ssTmp <- getDesignGroupSequential(
            kMax = 3,
            alpha = 0.025,
            beta = 0.2,
            sided = 1,
            typeOfDesign = "asOF",
            informationRates = c(1 / 3, 2 / 3, 1),
            futilityBounds = c(-6, z)
        ) |>
            getSampleSizeSurvival(
                hazardRatio = 0.7,
                median2 = 12,
                dropoutRate1 = 0.05,
                dropoutRate2 = 0.05,
                dropoutTime = 12,
                accrualTime = 18,
                followUpTime = 12
            )

        futVals <- as.numeric(ssTmp$futilityBoundsEffectScale)

        futVals[2] - targetHr2
    },
    interval = c(-1, 1)
)$root

fStage1

[1] 0.2439991

fStage2

[1] 0.6843932

Final design implementation

designGS <- getDesignGroupSequential(
    kMax = 3,
    alpha = 0.025,
    beta = 0.2,
    sided = 1,
    typeOfDesign = "asOF",
    informationRates = c(1 / 3, 2 / 3, 1),
    futilityBounds = c(fStage1, fStage2)
)
designGS |> summary()

Sequential analysis with a maximum of 3 looks (group sequential design)

O’Brien & Fleming type alpha spending design, non-binding futility, one-sided overall significance level 2.5%, power 80%, undefined endpoint, inflation factor 1.1076, ASN H1 0.8697, ASN H01 0.8262, ASN H0 0.5888.

Stage	1	2	3
Planned information rate	33.3%	66.7%	100%
Cumulative alpha spent	0.0001	0.0060	0.0250
Stage levels (one-sided)	0.0001	0.0060	0.0231
Efficacy boundary (z-value scale)	3.710	2.511	1.993
Futility boundary (z-value scale)	0.244	0.684
Cumulative power	0.0223	0.4570	0.8000
Futility probabilities under H1	0.072	0.020

Sample size planning

Our survival endpoint assumptions:

• Control median survival: 12 months.
• Alternative hazard ratio: 0.70.
• Uniform accrual over 18 months + follow-up 12 months.
• Dropout 5% after 12 months.
• Note that here we can also confirm that the futility bounds correspond to the desired HR thresholds:

sampleSize <- designGS |> getSampleSizeSurvival(
    hazardRatio = 0.7,
    median2 = 12,
    dropoutRate1 = 0.05,
    dropoutRate2 = 0.05,
    dropoutTime = 12,
    accrualTime = 18,
    followUpTime = 12
)
sampleSize |> summary()

Sample size calculation for a survival endpoint

Sequential analysis with a maximum of 3 looks (group sequential design), one-sided overall significance level 2.5%, power 80%. The results were calculated for a two-sample logrank test, H0: hazard ratio = 1, H1: hazard ratio = 0.7, control median(2) = 12, accrual time = 18, accrual intensity = 25.2, follow-up time = 12, dropout rate(1) = 0.05, dropout rate(2) = 0.05, dropout time = 12.

Stage	1	2	3
Planned information rate	33.3%	66.7%	100%
Cumulative alpha spent	0.0001	0.0060	0.0250
Stage levels (one-sided)	0.0001	0.0060	0.0231
Efficacy boundary (z-value scale)	3.710	2.511	1.993
Futility boundary (z-value scale)	0.244	0.684
Efficacy boundary (t)	0.460	0.689	0.786
Futility boundary (t)	0.950	0.904
Cumulative power	0.0223	0.4570	0.8000
Number of subjects	343.7	453.4	453.4
Expected number of subjects under H1			443.0
Cumulative number of events	91.1	182.2	273.3
Expected number of events under H1			214.6
Analysis time	13.64	20.58	30.00
Expected study duration under H1			24.16
Overall exit probability (under H0)	0.5965	0.2123
Overall exit probability (under H1)	0.0947	0.4550
Exit probability for efficacy (under H0)	0.0001	0.0059
Exit probability for efficacy (under H1)	0.0223	0.4347
Exit probability for futility (under H0)	0.5964	0.2064
Exit probability for futility (under H1)	0.0724	0.0203

Legend:

• (t): treatment effect scale

Approximate MAMS sample size from pairwise planning

The above used getSampleSizeSurvival() gives pairwise planning output (one active vs control).

For a rough MAMS planning number with equal arm sizes:

nPerArmPerStage <- ceiling(sampleSize$numberOfSubjects[1, 1] / 2)
nApproxTotal <- 3 * 3 * nPerArmPerStage

nPerArmPerStage

[1] 172

nApproxTotal

[1] 1548

This approximation is refined by simulation under multi-arm assumptions in the next steps.

MAMS simulations

Inverse normal design for MAMS survival

We now proceed towards simulating the full survival MAMS design with the inverse normal combination test, in order to protect the type I error rate:

designIN <- getDesignInverseNormal(
    kMax = 3,
    alpha = 0.025,
    beta = 0.2,
    sided = 1,
    typeOfDesign = "asOF",
    informationRates = c(1 / 3, 2 / 3, 1),
    futilityBounds = c(fStage1, fStage2)
)
designIN

Design parameters and output of inverse normal combination test design

User defined parameters

• Type of design: O’Brien & Fleming type alpha spending
• Futility bounds (non-binding): 0.244, 0.684

Derived from user defined parameters

• Maximum number of stages: 3

Default parameters

• Stages: 1, 2, 3
• Information rates: 0.333, 0.667, 1.000
• Significance level: 0.0250
• Type II error rate: 0.2000
• Binding futility: FALSE
• Test: one-sided
• Tolerance: 1e-08
• Type of beta spending: none

Output

• Cumulative alpha spending: 0.0001035, 0.0060484, 0.0250000
• Critical values: 3.710, 2.511, 1.993
• Stage levels (one-sided): 0.0001035, 0.0060122, 0.0231281

Simulation setup: scenarios and planned events

We define three scenarios using hazard ratios of active arms vs control:

• Scenario 1: \((1.00, 1.00)\) under global null.
• Scenario 2: \((0.70, 0.70)\) both active arms are effective.
• Scenario 3: \((0.75, 0.65)\) active arms have mixed efficacy.

Below we can then obtain results directly for all three scenarios at once.

# Just to get quick results here we choose this so low!
maxNumberOfIterations <- 100

hazardRatioMatrix <- matrix(
    c(
        1.00, 1.00,
        0.70, 0.70,
        0.75, 0.65
    ),
    nrow = 3,
    byrow = TRUE
)
hazardRatioMatrix

     [,1] [,2]
[1,] 1.00 1.00
[2,] 0.70 0.70
[3,] 0.75 0.65

plannedEvents <- ceiling(c(sampleSize$cumulativeEventsPerStage[, 1]))
plannedEvents

[1]  92 183 274

Different intersection tests

We can choose different intersection tests for the multi-arm setting:

• “Dunnett”: uses the Dunnett distribution to account for correlation between overlapping arm comparisons.
• “Bonferroni”: applies a simple Bonferroni correction, treating comparisons as independent.
• “Sidak”: applies a Sidak correction, which is slightly less conservative than Bonferroni.
• “Simes”: uses the Simes procedure, which is more powerful than Bonferroni and Sidak
• “Hierarchical”: applies a sequential testing procedure where arms are tested in a pre-specified order, and testing stops once an arm fails to reject

Simulation 1: select the best arm at first interim

At the first interim we select the best active arm (in terms of hazard ratio estimate) and continue with that arm and control for the rest of the trial:

sim1 <- designIN |> getSimulationMultiArmSurvival(
    activeArms = 2,
    effectMatrix = hazardRatioMatrix,
    typeOfShape = "userDefined",
    plannedEvents = plannedEvents,
    maxNumberOfIterations = maxNumberOfIterations,
    directionUpper = FALSE,
    intersectionTest = "Simes",
    typeOfSelection = "best",
    effectMeasure = "effectEstimate",
    successCriterion = "all",
    seed = 12345
)

sim1 |> summary()

Simulation of a survival endpoint (multi-arm design)

Sequential analysis with a maximum of 3 looks (inverse normal combination test design), one-sided overall significance level 2.5%. The results were simulated for a multi-arm logrank test (2 treatments vs. control), H0: hazard ratio(i) = 1, power directed towards smaller values, H1: omega_max as specified, planned cumulative events = c(92, 183, 274), effect shape = user defined, intersection test = Simes, selection = best, effect measure based on effect estimate, success criterion: all, simulation runs = 100, seed = 12345.

Stage	1	2	3
Fixed weight	0.577	0.577	0.577
Cumulative alpha spent	0.0001	0.0060	0.0250
Stage levels (one-sided)	0.0001	0.0060	0.0231
Efficacy boundary (z-value scale)	3.710	2.511	1.993
Futility boundary (z-value scale)	0.244	0.684
Reject at least one [1]			0.0200
Reject at least one [2]			0.8200
Reject at least one [3]			0.7200
Rejected arms per stage [1]
Treatment arm 1 vs. control	0	0	0.0100
Treatment arm 2 vs. control	0	0.0100	0
Rejected arms per stage [2]
Treatment arm 1 vs. control	0.0300	0.2000	0.2400
Treatment arm 2 vs. control	0.0200	0.1800	0.1600
Rejected arms per stage [3]
Treatment arm 1 vs. control	0	0.0300	0.1700
Treatment arm 2 vs. control	0	0.3800	0.1400
Success per stage [1]	0	0.0100	0.0100
Success per stage [2]	0.0100	0.4100	0.4000
Success per stage [3]	0	0.4100	0.3100
Cumulative number of events [1]
Treatment arm 1 vs. control	61.3	127.9	198.7
Treatment arm 2 vs. control	61.3	131.2	197.0
Cumulative number of events [2]
Treatment arm 1 vs. control	65.2	140.6	218.0
Treatment arm 2 vs. control	65.2	134.3	201.4
Cumulative number of events [3]
Treatment arm 1 vs. control	67.1	134.3	211.0
Treatment arm 2 vs. control	63.3	141.1	208.7
Expected number of events under H1 [1]			145.7
Expected number of events under H1 [2]			215.8
Expected number of events under H1 [3]			213.9
Overall exit probability [1]	0.5900	0.2300
Overall exit probability [2]	0.1100	0.4200
Overall exit probability [3]	0.1100	0.4400
Selected arms [1]
Treatment arm 1 vs. control	1.0000	0.1900	0.1000
Treatment arm 2 vs. control	1.0000	0.2200	0.0800
Selected arms [2]
Treatment arm 1 vs. control	1.0000	0.5200	0.3000
Treatment arm 2 vs. control	1.0000	0.3700	0.1700
Selected arms [3]
Treatment arm 1 vs. control	1.0000	0.3000	0.2700
Treatment arm 2 vs. control	1.0000	0.5900	0.1800
Number of active arms [1]	2.000	1.000	1.000
Number of active arms [2]	2.000	1.000	1.000
Number of active arms [3]	2.000	1.000	1.000
Conditional power (achieved) [1]		0.2651	0.3384
Conditional power (achieved) [2]		0.5940	0.7344
Conditional power (achieved) [3]		0.5558	0.6877
Exit probability for efficacy	0.0223	0.4347
Exit probability for futility [1]	0.5900	0.2200
Exit probability for futility [2]	0.1000	0.0100
Exit probability for futility [3]	0.1100	0.0300

Legend:

• (i): results of treatment arm i vs. control arm
• [j]: effect matrix row j (situation to consider)

Simulation 2: select arms not much worse than best arm

Here the selection works with the difference of hazard ratios. (One could consider allowing also the ratio of hazard ratios in the future maybe…) For this we can use the “epsilon” selection type:

sim2 <- designIN |> getSimulationMultiArmSurvival(
    activeArms = 2,
    effectMatrix = hazardRatioMatrix,
    typeOfShape = "userDefined",
    plannedEvents = plannedEvents,
    maxNumberOfIterations = maxNumberOfIterations,
    directionUpper = FALSE,
    intersectionTest = "Simes",
    typeOfSelection = "epsilon",
    # Select arms with hazard ratio not more than 0.2 worse than the best arm:
    epsilonValue = 0.2,
    effectMeasure = "effectEstimate",
    successCriterion = "all",
    seed = 12345
)

sim2 |> summary()

Simulation of a survival endpoint (multi-arm design)

Sequential analysis with a maximum of 3 looks (inverse normal combination test design), one-sided overall significance level 2.5%. The results were simulated for a multi-arm logrank test (2 treatments vs. control), H0: hazard ratio(i) = 1, power directed towards smaller values, H1: omega_max as specified, planned cumulative events = c(92, 183, 274), effect shape = user defined, intersection test = Simes, selection = epsilon rule, eps = 0.2, effect measure based on effect estimate, success criterion: all, simulation runs = 100, seed = 12345.

Stage	1	2	3
Fixed weight	0.577	0.577	0.577
Cumulative alpha spent	0.0001	0.0060	0.0250
Stage levels (one-sided)	0.0001	0.0060	0.0231
Efficacy boundary (z-value scale)	3.710	2.511	1.993
Futility boundary (z-value scale)	0.244	0.684
Reject at least one [1]			0.0600
Reject at least one [2]			0.7700
Reject at least one [3]			0.7400
Rejected arms per stage [1]
Treatment arm 1 vs. control	0	0	0.0400
Treatment arm 2 vs. control	0	0	0.0200
Rejected arms per stage [2]
Treatment arm 1 vs. control	0	0.2200	0.2500
Treatment arm 2 vs. control	0	0.2500	0.2300
Rejected arms per stage [3]
Treatment arm 1 vs. control	0.0100	0.1200	0.1700
Treatment arm 2 vs. control	0.0100	0.3400	0.2500
Success per stage [1]	0	0	0.0500
Success per stage [2]	0	0.3400	0.4000
Success per stage [3]	0.0100	0.3700	0.3400
Cumulative number of events [1]
Treatment arm 1 vs. control	61.3	125.0	191.5
Treatment arm 2 vs. control	61.3	125.0	189.0
Cumulative number of events [2]
Treatment arm 1 vs. control	65.2	132.8	202.0
Treatment arm 2 vs. control	65.2	135.4	206.0
Cumulative number of events [3]
Treatment arm 1 vs. control	67.1	133.5	202.4
Treatment arm 2 vs. control	63.3	136.2	206.2
Expected number of events under H1 [1]			150.2
Expected number of events under H1 [2]			218.5
Expected number of events under H1 [3]			224.9
Overall exit probability [1]	0.5400	0.2800
Overall exit probability [2]	0.1300	0.3500
Overall exit probability [3]	0.0700	0.4000
Selected arms [1]
Treatment arm 1 vs. control	1.0000	0.3700	0.1300
Treatment arm 2 vs. control	1.0000	0.3700	0.1200
Selected arms [2]
Treatment arm 1 vs. control	1.0000	0.5900	0.3300
Treatment arm 2 vs. control	1.0000	0.6500	0.3500
Selected arms [3]
Treatment arm 1 vs. control	1.0000	0.5200	0.3400
Treatment arm 2 vs. control	1.0000	0.7400	0.3900
Number of active arms [1]	2.000	1.609	1.389
Number of active arms [2]	2.000	1.425	1.308
Number of active arms [3]	2.000	1.355	1.377
Conditional power (achieved) [1]		0.2061	0.4156
Conditional power (achieved) [2]		0.5733	0.6850
Conditional power (achieved) [3]		0.5983	0.6241
Exit probability for efficacy	0.0223	0.4347
Exit probability for futility [1]	0.5400	0.2800
Exit probability for futility [2]	0.1300	0.0100
Exit probability for futility [3]	0.0600	0.0300

Legend:

• (i): results of treatment arm i vs. control arm
• [j]: effect matrix row j (situation to consider)

Compare power results

Then we could compare e.g. the overall power (reject at least one active arm) across the two simulations and three scenarios:

simCompare <- data.frame(
    scenario = c("[1] null", "[2] both active", "[3] mixed"),
    rejectAtLeastOne_sim1 = sim1 |> fetch(rejectAtLeastOne),
    rejectAtLeastOne_sim2 = sim2 |> fetch(rejectAtLeastOne)
)
simCompare

         scenario rejectAtLeastOne rejectAtLeastOne.1
1        [1] null             0.02               0.06
2 [2] both active             0.82               0.77
3       [3] mixed             0.72               0.74

Patient-level simulations

Note that in the current rpact release we cannot yet specify:

• drop-out rates
• accrual times and intensities
• maximum number of subjects

This is because the simulations are still done on the level of test statistics, not patient-level data.

However, we have almost finished the implementation of patient-level simulations for MAMS survival designs, which will come in the next rpact release!

Planning conclusions

• MAMS survival planning starts with careful futility calibration on the treatment-effect scale.
• Pairwise sample size provides a baseline, while simulation quantifies true multi-arm operating characteristics.
• Arm-selection strategy has direct impact on expected sample size and power.
• Custom selection rules are also possible