---
title: "Multiple - methods for trials with multiple endpoints"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Multiple - methods for trials with multiple endpoints}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

Suppose we are planning a drug development program testing the superiority of an experimental treatment over a control treatment. Our drug development program consists of an exploratory phase II trial which is, in case of promising results, followed by a confirmatory phase III trial.

The drugdevelopR package enables us to optimally plan such programs using a utility-maximizing approach. To get a brief introduction, we presented a very basic example on how the package works in [Introduction to planning phase II and phase III trials with drugdevelopR](https://sterniii3.github.io/drugdevelopR/articles/Introduction-to-drugdevelopR.html). Contrary to the basic setting, where only one phase II and one phase III trial were conducted, in this scenario we investigate what happens when multiple endpoints are of interest. The functions currently implemented only allow for the conduction of trials with up to two different endpoints. The drugdevelopR package enables programs where only one endpoints has to show a significant positive treatment effect or all endpoints have to show a significant positive treatment effect.

After installing the package according to the [installation instructions](https://sterniii3.github.io/drugdevelopR/#Installation), we can load it using the following code, and start right into the two examples:
```{r}
library(drugdevelopR)
```

# Only one endpoint has to be significant

Suppose we are developing a new tumor treatment, *exper*.  The two endpoints we consider interesting are overall survival (OS) and progression-free survival (PFS). These are time-to-event outcome variables. Therefore, we will use the function `optimal_multiple_tte`, which calculates optimal sample sizes and threshold decisions values for time-to-event outcomes when multiple endpoints are investigated. 

Within our drug development program, we will compare our experimental treatment to the control treatment *contro*. The treatment effect measure of *exper* for each endpoint $i \in \{1,2\}$ is given by $\theta_i = −\log(HR_i)$, where the hazard ratio $HR_i$ is the ratio of the hazard rates between the experimental treatment and the control treatment, for each endpoint. For more information on the hazard ratio, see the vignette on [time-to-event endpoints](https://sterniii3.github.io/drugdevelopR/articles/Time-to-event_outcomes.html). 


## Defining all necessary parameters
The parameters in the setting with multiple endpoints differ slightly from the other cases ([bias adjustment](https://sterniii3.github.io/drugdevelopR/articles/Bias_adjustment.html) and [multitrial](https://sterniii3.github.io/drugdevelopR/articles/Multitrial.html)).
As in the basic setting, the treatment effect may be fixed (as in this example) or may follow a prior distribution (see [Fixed or Prior](https://sterniii3.github.io/drugdevelopR/articles/Fixed_and_prior_distributions.html)) Note that in this case, contrary to the other settings, the second treatment effect describes the effect of the second endpoint and is also relevant for `fixed = TRUE`. Furthermore, some options to adapt the program to your specific needs are also available in this setting (see [More parameters](https://sterniii3.github.io/drugdevelopR/articles/More_Parameters.html)), however skipping phase II and choosing different treatment effects in phase II and III are not possible.

* `hr1` is the assumed hazard ratio of the endpoint OS. `hr2` is the assumed hazard ratio of the endpoint PFS. We assume that our experimental treatment *exper* leads to a hazard reduction of 80\% compared to the control treatment *contro* for endpoint OS and a reduction of 75\% for PFS. Therefore, we set `hr1 = 0.8` and `hr2 = 0.75`.
* `n2min` and `n2max` specify the minimal and maximal number of participants (not events) for the phase II trial. The package will search for the optimal sample size within this region. For now, we want the program to search for the optimal sample size in the interval between 20 and 400 participants. In addition, we will tell the program to search this region in steps of ten participants at a time by setting `stepn2 = 10`.
* Differing from the basis setting, one has to provide two sets of benefits triples, one triple `b11`, `b21`, `b31` in the case endpoint OS is significant (independent of PFS beiing significant or not) and one triple `b12`, `b22`, `b32` in the case OS is not significant. The intuition is that overall survival is the "more relevant" endpoint, i.e. if this endpoints shows a significant result, we have higher gains than if only the "less important" endpoint PFS shows a significant result. We assign benefit triples of {1000, 2000, 3000} for the case OS is significant and {1000, 1500, 2000} if not (all in units of 10^5\$).
* Finally, the parameter `rho` determines the correlation between the treatment effects of the two endpoints. In our case, we set `rho = 0.6`, indicating the correlation between the treatment effects of OS and PFS is 0.6.

```{r,eval = FALSE}
set.seed(123)
res1 <- optimal_multiple_tte(hr1 = 0.8, hr2 = 0.75,                   # define assumed true HRs
                    id1 = NULL, id2 = NULL,
                    n2min = 20, n2max = 400, stepn2 = 10,             # define optimization set for n2
                    hrgomin = 0.7, hrgomax = 0.9, stephrgo = 0.05,    # define optimization set for HRgo
                    alpha = 0.025, beta = 0.1,                        # drug development planning parameters
                    c2 = 0.75, c3 = 1, c02 = 100, c03 = 150,          # define costs for phase II and III
                    K = Inf, N = Inf, S = -Inf,                       # set constraints
                    steps1 = 1,  stepm1 = 0.95,  stepl1 = 0.85,       # effect size categories
                    b11 = 1000, b21 = 2000, b31 = 3000,
                    b12 = 1000, b22 = 1500, b32 = 2000,               # define expected benefits (both categories)
                    rho = 0.6, fixed = TRUE,                          # correlation and treatment effect
                    num_cl = 2)
```

```{r, eval=TRUE, include=FALSE}
# Comment this chunk after running it once
# set.seed(123)
# res1 <- optimal_multiple_tte(hr1 = 0.8, hr2 = 0.75,                   # define assumed true HRs
#                    id1 = NULL, id2 = NULL,
#                    n2min = 20, n2max = 400, stepn2 = 10,             # define optimization set for n2
#                    hrgomin = 0.7, hrgomax = 0.9, stephrgo = 0.05,    # define optimization set for HRgo
#                    alpha = 0.025, beta = 0.1,                        # drug development planning parameters
#                    c2 = 0.75, c3 = 1, c02 = 100, c03 = 150,          # define costs for phase II and III
#                    K = Inf, N = Inf, S = -Inf,                       # set constraints
#                    steps1 = 1,  stepm1 = 0.95,  stepl1 = 0.85,       # effect size categories
#                    b11 = 1000, b21 = 2000, b31 = 3000,
#                    b12 = 1000, b22 = 1500, b32 = 2000,               # define expected benefits (both categories)
#                    rho = 0.6, fixed = TRUE,                          # correlation and treatment effect
#                    num_cl = 2)
# saveRDS(res1, file="optimal_multiple_tte.RDS")
```
```{r, eval=TRUE, include=FALSE}
res1 <- readRDS(file="optimal_multiple_tte.RDS")
```

## Interpreting the output

After setting all these input parameters and running the function, let's take a look at the output of the program.

```{r}
res1
```
The program returns a data frame, with the following results:

* `res1$n2` is the optimal number of participants for phase II and `res$n3` the resulting number of events for phase III. We see that the optimal scenario requires 90 participants in phase II and 301 participants in phase III.
* `res1$HRgo` is the optimal threshold value for the go/no-go decision rule. We see that we need a hazard ratio of less than 0.85 in phase II in order to proceed to phase III. 
* `res1$u` is the expected utility of the program for the optimal sample size and threshold value. In our case it amounts to 106.61, i.e. we have an expected utility of 10 661 000\$.
* `res1$OS` is the probability that endpoint OS is significant, given that the program is successful overall. The probability of a successful program is 0.4, thus, if the program is successful the probability that the endpoint OS is significant and we obtain higher benefits is 0.71.

# Both endpoints have to be significant

In this scenario, suppose we are developing a new Alzheimer medication, *exper*.  The two endpoints we consider interesting are improving cognition (cognitive endpoint) and improving activities of daily living (functional endpoint). We want both outcomes to be show statistically significant differences for the trial to be successful. These are normally distributed outcome variables. Therefore, we will use the function `optimal_multiple_normal`, which calculates optimal sample sizes and threshold decisions values for normally distributed outcomes when multiple endpoints are investigated.  

Within our drug development program, we will compare our experimental treatment to the control treatment *contro*. The treatment effect measure of *exper* for each endpoint $i \in \{1,2\}$ is given by $\Delta_i = \mu_{exper} - \mu_{contro}$, which is the difference in mean between the experimental treatment and the control treatment, for both endpoints. Note that this is not divided by the standard deviation, hence is it not standardized. For more information, see the vignette on [normally distributed outcomes](https://sterniii3.github.io/drugdevelopR/articles/Introduction-to-drugdevelopR.html).

## Defining all necessary parameters
Again, the parameters in the setting with multiple endpoints differ slightly from the other cases ([bias adjustment](https://sterniii3.github.io/drugdevelopR/articles/Bias_adjustment.html) and [multitrial](https://sterniii3.github.io/drugdevelopR/articles/Multitrial.html)).
As in the basic setting, the treatment effect may be fixed (as in this example) or may follow a prior distribution (see [Fixed or Prior](https://sterniii3.github.io/drugdevelopR/articles/Fixed_and_prior_distributions.html)) Note, that in this case, contrary to the other settings, the second treatment effect describes the effect of the second endpoint and is also relevant for `fixed = TRUE`. Furthermore, some options to adapt the program to your specific needs are also available in this setting (see [More parameters](https://sterniii3.github.io/drugdevelopR/articles/More_Parameters.html)), however skipping phase II and choosing different treatment effects in phase II and III are not possible.

* `Delta1` is our assumed true treatment effect given as difference in means between *exper* and *contro* for the cognitive endpoint and `Delta2` is our assumed true treatment effect given as difference in means between *exper* and *contro* for the functional endpoint. We assume true treatment effects to be 0.8 and 0.5 for each endpoint respectively, hence `Delta1 = 0.8` and `Delta2 = 0.5`.
* `n2min` and `n2max` specify the minimal and maximal number of participants for the phase II trial. The package will search for the optimal sample size within this region. For now, we want the program to search for the optimal sample size in the interval between 20 and 200 participants. In addition, we will tell the program to search this region in steps of ten participants at a time by setting `stepn2 = 10`.
* We have to include two parameters that amount for the standard deviations $\sigma_i$ for $i \in \{1,2\}$. We set `sigma1 = 2` and `sigma2 = 1` for our example. 
* The parameter `relaxed` decides how the effect sizes of the two endpoints are combined into one overall effect size (large, medium or small overall treatment effect). If `relaxed = FALSE` , we consider a strict rule assigning a large overall effect if both endpoints show an effect of large size, a small overall effect if at least one of the endpoints shows a small effect, and a medium overall effect otherwise. If `relaxed = TRUE`, we investigate the characteristics for a more relaxed rule assigning a large overall effect if at least one of the endpoints shows a large effect, a small effect if both endpoints show a small effect, and a medium overall effect otherwise. For our example, we set `relaxed = TRUE`
* Finally, the parameter `rho` determines the correlation between the treatment effects of the two endpoints. In our case, we set `rho = 0.5`, indicating the correlation between the treatment effects is 0.5.

```{r,eval = FALSE}
set.seed(123)
res2 <- optimal_multiple_normal(Delta1 = 0.8, Delta2 = 0.5,  # define assumed true treatment effects
   in1= NULL, in2= NULL, sigma1 = 2, sigma2= 1,              # standard deviations
   n2min = 20, n2max = 200, stepn2 = 10,                     # define optimization set for n2
   kappamin = 0.02, kappamax = 0.2, stepkappa = 0.02,        # define optimization set for HRgo
   alpha = 0.025, beta = 0.1,                                # drug development planning parameters
   c2 = 0.75, c3 = 1, c02 = 100, c03 = 150,                  # define fixed and variable costs for phase II and III
   K = Inf, N = Inf, S = -Inf,                               # set constraints
   steps1 = 0,  stepm1 = 0.5, stepl1 = 0.8,                  # benefit categories
   b1 = 1000, b2 = 2000, b3 = 3000,                          # define expected benefits 
   rho = 0.5, relaxed = TRUE,                                # relaxed "TRUE"
   fixed = TRUE,                                             # fixed treatment effect 
   num_cl = 2)
```

```{r, eval=TRUE, include=FALSE}
# Comment this chunk after running it once
# set.seed(123)
# res2 <- optimal_multiple_normal(Delta1 = 0.8, Delta2 = 0.5, # define assumed true treatment effects
#   in1= NULL, in2= NULL, sigma1 = 2, sigma2= 1,              # standard deviations
#   n2min = 20, n2max = 200, stepn2 = 10,                     # define optimization set for n2
#   kappamin = 0.02, kappamax = 0.2, stepkappa = 0.02,        # define optimization set for HRgo
#   alpha = 0.025, beta = 0.1,                                # drug development planning parameters
#   c2 = 0.75, c3 = 1, c02 = 100, c03 = 150,                  # define fixed and variable costs for phase II and III
#   K = Inf, N = Inf, S = -Inf,                               # set constraints
#   steps1 = 0,  stepm1 = 0.5, stepl1 = 0.8,                  # benefit categories
#   b1 = 1000, b2 = 2000, b3 = 3000,                          # define expected benefits
#   rho = 0.5, relaxed = TRUE,                                # relaxed "TRUE"
#   fixed = TRUE,                                             # fixed treatment effect
#   num_cl = 6)
# saveRDS(res2, file="optimal_multiple_normal.RDS")
```
```{r, eval=TRUE, include=FALSE}
res2 <- readRDS(file="optimal_multiple_normal.RDS")
```

## Interpreting the output

After setting all these input parameters and running the function, let's take a look at the output of the program.

```{r}
res2
```
The program returns a data frame, with the following results:

* `res$n2` is the optimal number of participants for phase II and `res$n3` the resulting number of events for phase III. We see that the optimal scenario requires 120 participants in phase II and 171 participants in phase III.
* `res$Kappa` is the optimal threshold value for the go/no-go decision rule. We see that we need a hazard ratio of less than 0.02 in phase II in order to proceed to phase III. 
* `res$u` is the expected utility of the program for the optimal sample size and threshold value. In our case it amounts to 342.07, i.e. we have an expected utility of 34 207 000\$.

# Where to go from here
In this article we presented examples when multiple endpoints are of interest.

For more information on how to use the package, see:

* [*Introduction to drugdevelopR:*](https://sterniii3.github.io/drugdevelopR/articles/Introduction-to-drugdevelopR.html) See how the package works in a basic example.
* *Different outcomes:* Apply it to [binary endpoints](https://sterniii3.github.io/drugdevelopR/articles/Binary_outcomes.html) and [time-to-event endpoints](https://sterniii3.github.io/drugdevelopR/articles/Time-to-event_outcomes.html).
* [*Interpreting the rest of the output:*](https://sterniii3.github.io/drugdevelopR/articles/Interpreting_Output.html) Obtain further details on your drug development program.
* [*Fixed or prior:*](https://sterniii3.github.io/drugdevelopR/articles/Fixed_and_prior_distributions.html) Model the assumed treatment effect on a prior distribution.
* [*More parameters:*](https://sterniii3.github.io/drugdevelopR/articles/More_Parameters.html) Define custom effect size categories. Put constraints on the optimization by defining maximum costs, the total expected sample size of the program or the minimum expected probability of a successful program. Define an expected difference in treatment effect between phase II and III. Skip phase II.
* *Complex drug development programs:* Adapt to situations with [biased effect estimators](https://sterniii3.github.io/drugdevelopR/articles/Bias_adjustment.html), [multiple phase III trials](https://sterniii3.github.io/drugdevelopR/articles/Multitrial.html) or [multi-arm trials](https://sterniii3.github.io/drugdevelopR/articles/Multiarm_Trials.html).
* [*Parallel computing:*](https://sterniii3.github.io/drugdevelopR/articles/More_Parameters.html#parallel-computing) Be faster at calculating the optimum by using parallel computing.
