---
title: "Eudract and CT.gov Safety XML"
author: "Simon Bond"
date: "`r Sys.Date()`"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Eudract and CT.gov Safety XML}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

# Introduction

The European Clinical Trials Database [Eudract](https://eudract.ema.europa.eu/) is run by the European Medicines Agency. All studies that are officially registered clinical trials have to enter the results of the final analysis into the website to be made public. 

An equivalent database hosted by the U.S. government is [CT.gov](https://clinicaltrials.gov/). 

There is a large amount of documentation online that will not be repeated here.

The detailed entering of safety information is an onerous task if it is to be done by hand. However we do have the facility to upload an XML file to automate this step. This R package seeks to enable the production of an XML file from a standard structure of Safety data that is recorded on a patient-level. 

The key functions are:
 
 * `safety_summary`
 * `simple_safety_xml`
 * `eudract_convert`

We provide a dummy safety data set that is one line per patient-event: `?safety`. The format of the data, specifically the variable names is described in the help file printed below. This needs to be turned into several frequency tables: one given information at a group level; one given at the event level, broken down into serious and non-serious events. The term "group" here corresponds to treatment arms in a randomised control trial. You need to define a group within EudraCT even if it is a one-armed study, in which case it can be a "dummy" label. 

Some information is defined in a hard-coded fashion below, but it is understood that this will be generated by code if applied in real life. Each entry in the vectors below correspond to counts in each of the two groups.

```{r meta}
subjectsExposed <- c("Control"=99,"Experimental"=101)
#count of deaths not in the Safety data. Could be c(0,0)
deathsExternal <- c("Control"=3,"Experimental"=5)
```

Coded adverse events are required to be helpful and avoid the task of reconciling minor spelling or text inconsistencies. This package and vignette assumes this is the case, and will not work in the absence of coding. We cannot provide the full MedDRA dictionary, due to copy right reasons. But normally this is available to sponsors. However, for upload into EudraCT, as a minimal requirement, only the System Organ Class (SOC) needs to be fully coded into the EudraCT internal version coding system. We have provided an internal data set, derived from the [eutct site](http://eutct.ema.europa.eu/eutct/viewListDisplay.do?listId=100000000006&firstTime=true&d-5037238-p=1&d-5037238-n=1&d-5037238-o=1&d-5037238-s=termName) in the package to use this; see `?soc_code`.

## Safety data set

There are two possible formats for inputting the safety data

* The package's original data format with an ad hoc choice of variable names
* Using [ADaM](https://www.cdisc.org/standards/foundational/adam) formats from the [CDISC](https://www.cdisc.org/) standards

### Original Format

A sample data set with the required fields is included.

```{r input, echo=FALSE, results="asis"}
library(eudract)
file <- utils:::.getHelpFile(help("safety"))
tmp <- tempfile()
tools:::Rd2HTML(file, out=tmp)
out <- readLines(tmp)
headfoot <- grep("body", out)
cat(out[(headfoot[1] + 1):(headfoot[2] - 1)], sep = "\n")
```

### ADaM format

A pair of example data sets are taken from the [pharmaverseadam](https://doi.org/10.32614/CRAN.package.pharmaverseadam) package. Specifically the individual  subject-level data `pharmaverseadam::adsl`  and adverse event data `pharmaverseadam::adae`. The adverse event data is readily convertible to the original format data, just with different variable names. The subject level data is used to internally calculate the numbers exposed, and excess deaths not included in the adverse event data. 

## Calculate Summary Statistics

We provide a function that derives the patient and event counts as required in a format internal to R.

### Original Data Format

```{r counting}
safety_statistics <- safety_summary(safety, 
                                    exposed=subjectsExposed, 
                                    excess_deaths = deathsExternal, 
                                    freq_threshold = 1
                                    )
safety_statistics
```

### ADaM data format

The same arguments for `freq_trheshold` and   `na.action` are used. The `excess_deaths` and `exposed` are internally calculated and so not needed. The ADaM data has a text field `AEREL` that is unconstrained, so we can provide an argument `related_terms` that gives the set of terms used to identify related events; the default is `c("POSSIBLE","PROBABLE","DEFINITELY"))` .

```{r adam_counting}
library(pharmaverseadam)
library(dplyr)
data("adsl")
data("adae")

safety_statistics_adam <- safety_summary_adam(
  adsl |> filter(ARM != "Screen Failure"),
  adae)
safety_statistics_adam 
```

## Convert to XML

If you have produced these statistics through separate coding, then you can use the `eudract:::create.summary_statistics()` function to put them into the correct internal format and start the conversion to XML directly.

First we export the `safety_statistics` to a XML document that is human readable "simple.xml". Then we convert to the EudraCT and CT.gov formats.

```{r xml}
simple <- tempfile(fileext = ".xml")
eudract_upload_file <- tempfile(fileext = ".xml")
ct_upload_file <- tempfile(fileext = ".xml")
simple_safety_xml(safety_statistics, simple)
eudract_convert(input=simple,
                output=eudract_upload_file)
clintrials_gov_convert(input=simple,
                       original=system.file("extdata", "1234.xml", package ="eudract"),
                output=ct_upload_file)


```

Note that for the ClinicalTrials.gov, there must **first** be a study set-up within website, and then a download of the XML taken. This is the `original` argument. Then the original file has the safety events data over-written, and can be manually uploaded back into ClinicalTrials.gov

Alternatively, if you have a user account within CT.gov, then the initial study needs to be set up within there, but we can use the API to directly upload without needing to manually interact with the site.

```{r upload, eval=FALSE}
# Not actually run. It needs real user account details: the ones below are fictitious.
clintrials_gov_upload(
    input=simple,
    orgname="CTU",
    username="Student",
    password="Guinness",
    studyid="1234"
    )

```




# Output


The key outputs are 

* [safety_upload.xml](../doc/safety_upload.xml), for eudraCT 
* [ct_safety_upload.xml](../doc/ct_safety_upload.xml) for CT.gov


We can validate the output against the XML schemas provided by EudraCT and CT.gov, although the calls to 
`eudract_convert()` and `clintrials_gov_convert()` also do this behind the scenes, returning the value `TRUE` if there are no errors against the schema validation. 

* [eudract schema](https://eudract.ema.europa.eu/docs/technical/schemas/clinicaltrial/results/adverseEvents.xsd), was downloaded to this package on 15NOV2016
* [CT.gov overall xml schema](https://cdn.clinicaltrials.gov/documents/xsd/prs/ProtocolRecordSchema.xsd), was downloaded to this package on 29JUL2021
* [CT.gov results xml schema](https://cdn.clinicaltrials.gov/documents/xsd/prs/RRSUploadSchema.xsd), was downloaded to this package on 31MAR2021


note these  are semi-readable files of code/data rather than a standard web page.


```{r check_schema}
myschema <- xml2::read_xml(system.file("extdata","adverseEvents.xsd", package="eudract"))
aes <- xml2::read_xml(eudract_upload_file)
check <- xml2::xml_validate(aes,myschema)
if(check){print("Validation against eudraCT schema has passed!")}


myschema <- xml2::read_xml(system.file("extdata","ProtocolRecordSchema.xsd", package="eudract"))
aes <- xml2::read_xml(ct_upload_file)
check <- xml2::xml_validate(aes,myschema)
if(check){print("Validation against CT.gov schema has passed!")}
```

# Manual Upload

To use the resulting eudraCT xml file navigate and log in online to the study specific area of the EudraCT site. On the top banner is a link "Upload XML" which you follow. Choose the option "Adverse Events" rather than "Full data set", and select the file xml you have produced. The resulting information can be viewed in the browser interactively or with a static [pdf file](../doc/ct_result_0004-000245-38.pdf) (note this is a fictitious study and fictitious data). This is not the only step in completing the EudraCT report, as the description of the study, baseline characteristics and efficacy analysis will all need to be added. That is not the remit of this package though.

For the ClinicalTrials.gov, once logged in, there is a button titled "Records", near the top. From there select "Upload Record (XML)". On the new page, use the "Choose File" button to select the newly created XML file, and click "Upload".

To extract the original study record for over-writing, you need to go into the specific study record from the home page. In there beneath the initial section titled "Record Status", there is a link "Download XML", which will enable you to save locally the required file. 
