---
title: 'Using Saved Models'
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Using Saved Models from R} 
  %\VignetteEngine{knitr::rmarkdown} 
  %\VignetteEncoding{UTF-8}
type: docs
repo: https://github.com/rstudio/tfdeploy
menu:
  main:
    name: "Using Saved Models"
    identifier: "tools-tfdeploy-using-saved-models"
    parent: "tfdeploy-top"
    weight: 20
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(eval = FALSE)
```

## Overview

The main goal of the tfdeploy package is to create models in R and then export, test, and deploy those models to environments without R. However, there may be cases when it makes sense to use a saved model directly from R:

- If another R user has saved and/or deployed a model that you would like to use for predictions from R.
- If you want to use a saved or deployed model in a Shiny application.
- If you want to compare predictions between a saved or deployed model and a new model that is under development.

One way to use a deployed model from R would be to execute HTTP requests using a package like `httr`. For non-deployed models, it is possible to use `serve_savedmodel()` - as we did for local testing - along with a tool like `httr`. However, there is an easier way to make predictions from a saved model using the `predict_savedmodel()` function.

## Example

Using the same MNIST model described previously, we can easily make predictions for new pre-processed images. For example, we can load the MNIST test data set and create predictions for the first 10 images:

```{r}
library(keras)
library(tfdeploy)

test_images <- dataset_mnist()$test$x 
test_images <- array_reshape(test_images, dim = c(nrow(test_images), 784)) / 255
test_images <- lapply(1:10, function(i) {test_images[i,]})

predict_savedmodel(processed_test_list, 'savedmodel')
```
```
Prediction 1:
$prediction
 [1] 3.002971e-37 8.401216e-29 2.932129e-24 4.048731e-22 0.000000e+00 9.172148e-37
 [7] 0.000000e+00 1.000000e+00 4.337524e-31 1.772979e-17

Prediction 2:
$prediction
 [1] 0.000000e+00 4.548326e-22 1.000000e+00 2.261879e-31 0.000000e+00 0.000000e+00
 [7] 0.000000e+00 0.000000e+00 2.390626e-38 0.000000e+00
 
 ...
```

A few things to keep in mind:

1. Just like the HTTP POST requests, `predict_savedmodel()` expects the new instance data to be pre-processed.

2. `predict_savedmodel()` requires the new data to be in a list, and it always returns a list. This requirement faciliates models with more complex inputs or ouputs.

In the previous example we used `predict_savedmodel()` with the directory, 'savedmodel', which was created with the `export_savedmodel()` function In addition to providing a path to a saved model directory, `predict_savedmodel()` can also be used with a deployed model by supplying a REST URL, a CloudML model by supplying a CloudML name and version, or by supplying a graph object loaded with `load_savedmodel()`.

The last option above references the `load_savedmodel()` function. `load_savedmodel()` should be used alongside of `predict_savedmodel()` if you'll be calling the prediction function multiple times. `load_savedmodel()` effectively caches the model graph in memory and can speed up repeated calls to `predict_savedmodel()`. This caching is useful, for example, in a Shiny application where user input would drive calls to `predict_savedmodel()`.


```{r}
# if there will only be one batch of predictions 
predict_savedmodel(instances, 'savedmodel')

# if there will be multiple batches of predictions
sess <- tensorflow::tf$Session()
graph <- load_savedmodel(sess, 'savedmodel')
predict_savedmodel(instances, graph)
# ... more work ... 
predict_savedmodel(instances, graph)
```



## Model Representations

There are a few distinct ways that a model can be represented in R. The most straightforward representation is the in-memory, R model object. This object is what is created and used while developing and training a model. 

A second representation is the on-disk saved model. This representation of the model can be used by the `*_savedmodel` functions. As a special case, `load_savedmodel()` creates a new R object pointing to the model graph. It is important to keep in mind that these saved models are not the full R model object. For example, you can not update or re-train a graph from a saved model.

Finally, for Keras models there are 2 other representations: HDF5 files and serialized R objects. Each of these represenations captures the entire in-memory R object. For example, using `save_model_hdf5()` and then `load_model_hdf5()` will result in a model that can be updated or retrained. Use the `serialize_model()` and `unserialized_model()` to save models as R objects.

### What represenation should I use? 

If you are developing a model and have access to the in-memory R model object, you should use the model object for predictions using R's `predict` function.

If you are developing a Keras model and would like to save the model for use in a different session, you should use the HDF5 file or serialize the model and then save it to an R data format like RDS.

If you are going to deploy a model and want to test it's HTTP interface, you should export the model using `export_savedmodel()` and then test with either `serve_savedmodel()` and your HTTP client or `predict_savedmodel()`.

If you are using R and want to create predictions from a deployed or saved model, and you don't have access to the in-memory R model object, you should use `predict_savedmode()l`.


