`vetiver_deploy_sagemaker()` fails the example
Closed this issue · 19 comments
Hi Julia,
Thanks for the great new features.
I am trying to run the veitver_deploy_sagemaker() example, which is featured on Posit blog here but I am getting an error.
I've tried both the v0.2.1 and v0.2.1.9000.
Here's the reprex:
library(vetiver)
cars_lm <- lm(mpg ~ ., data=mtcars)
v <- vetiver_model(cars_lm, "cars-linear-test")
library(pins)
identifier <- "sagemaker-vetiver"
board <- board_s3(bucket = identifier, region = "ca-central-1")
vetiver_pin_write(board, v)
new_endpoint <-
vetiver_deploy_sagemaker(
board = board,
name = "cars-linear-test",
instance_type = "ml.t2.medium"
)
#> Error in vetiver_deploy_sagemaker(board = board, name = "cars-linear-test", : could not find function "vetiver_deploy_sagemaker"Created on 2023-06-05 with reprex v2.0.2
Session info
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#> setting value
#> version R version 4.3.0 (2023-04-21 ucrt)
#> os Windows 11 x64 (build 22621)
#> system x86_64, mingw32
#> ui RTerm
#> language (EN)
#> collate English_Canada.utf8
#> ctype English_Canada.utf8
#> tz America/Vancouver
#> date 2023-06-05
#> pandoc 2.19.2 @ C:/Program Files/RStudio/resources/app/bin/quarto/bin/tools/ (via rmarkdown)
#>
#> ─ Packages ───────────────────────────────────────────────────────────────────
#> package * version date (UTC) lib source
#> cli 3.6.1 2023-03-23 [1] CRAN (R 4.3.0)
#> digest 0.6.31 2022-12-11 [1] CRAN (R 4.3.0)
#> evaluate 0.21 2023-05-05 [1] CRAN (R 4.3.0)
#> fastmap 1.1.1 2023-02-24 [1] CRAN (R 4.3.0)
#> fs 1.6.2 2023-04-25 [1] CRAN (R 4.3.0)
#> glue 1.6.2 2022-02-24 [1] CRAN (R 4.3.0)
#> htmltools 0.5.5 2023-03-23 [1] CRAN (R 4.3.0)
#> knitr 1.43 2023-05-25 [1] CRAN (R 4.3.0)
#> lifecycle 1.0.3 2022-10-07 [1] CRAN (R 4.3.0)
#> reprex 2.0.2 2022-08-17 [1] CRAN (R 4.3.0)
#> rlang 1.1.1 2023-04-28 [1] CRAN (R 4.3.0)
#> rmarkdown 2.22 2023-06-01 [1] CRAN (R 4.3.0)
#> rstudioapi 0.14 2022-08-22 [1] CRAN (R 4.3.0)
#> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.3.0)
#> withr 2.5.0 2022-03-03 [1] CRAN (R 4.3.0)
#> xfun 0.39 2023-04-20 [1] CRAN (R 4.3.0)
#> yaml 2.3.7 2023-01-23 [1] CRAN (R 4.3.0)
#>
#> [1] C:/Users/maadi/AppData/Local/R/win-library/4.3
#> [2] C:/Program Files/R/R-4.3.0/library
#>
#> ──────────────────────────────────────────────────────────────────────────────My AWS credentials with full access to S3 and SageMaker are stored in the AWS shared credential file, ~/.aws/credentials, and I have no problem pushing to S3.
So sorry you are having troubles @aminadibi! The error message you are reporting sounds like vetiver isn't really loaded, maybe:
could not find function "vetiver_deploy_sagemaker"
Is that the error message you see when you are running this function on SageMaker?
My AWS credentials with full access to S3 and SageMaker are stored in the AWS shared credential file, ~/.aws/credentials
I don't believe this is necessary when you are working on SageMaker (although there are differences in how your instance can be set up). When you are in the RStudio IDE on SageMaker, the credentials for, say, pushing to ECS or creating a model endpoint are typically automatically available in the SageMaker environment. Are you working within the SageMaker environment?
Thanks @juliasilge. Sorry, the actual error message that I get is Error in enc2utf8(data): argument is not a character vector (I messed up the original reprex above🤦)
Here's the correct reprex:
library(vetiver)
cars_lm <- lm(mpg ~ ., data=mtcars)
v <- vetiver_model(cars_lm, "cars-linear-test")
library(pins)
identifier <- "sagemaker-vetiver"
board <- board_s3(bucket = identifier, region = "ca-central-1")
vetiver_pin_write(board, v)
new_endpoint <-
vetiver_deploy_sagemaker(
board = board,
name = "cars-linear-test",
instance_type = "ml.t2.medium"
)
#> The version of R recorded in the lockfile will be updated:
#> - R [* -> 4.3.0]
#>
#> * Lockfile written to 'vetiver_renv.lock'.
#> Error in enc2utf8(data): argument is not a character vectorCreated on 2023-06-06 with reprex v2.0.2
Session info
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#> setting value
#> version R version 4.3.0 (2023-04-21 ucrt)
#> os Windows 11 x64 (build 22621)
#> system x86_64, mingw32
#> ui RTerm
#> language (EN)
#> collate English_Canada.utf8
#> ctype English_Canada.utf8
#> tz America/Vancouver
#> date 2023-06-06
#> pandoc 2.19.2 @ C:/Program Files/RStudio/resources/app/bin/quarto/bin/tools/ (via rmarkdown)
#>
#> ─ Packages ───────────────────────────────────────────────────────────────────
#> package * version date (UTC) lib source
#> base64enc 0.1-3 2015-07-28 [1] CRAN (R 4.3.0)
#> bit 4.0.5 2022-11-15 [1] CRAN (R 4.3.0)
#> bit64 4.0.5 2020-08-30 [1] CRAN (R 4.3.0)
#> bundle 0.1.0 2022-09-15 [1] CRAN (R 4.3.0)
#> butcher 0.3.2 2023-03-08 [1] CRAN (R 4.3.0)
#> cli 3.6.1 2023-03-23 [1] CRAN (R 4.3.0)
#> crayon 1.5.2 2022-09-29 [1] CRAN (R 4.3.0)
#> curl 5.0.0 2023-01-12 [1] CRAN (R 4.3.0)
#> digest 0.6.31 2022-12-11 [1] CRAN (R 4.3.0)
#> ellipsis 0.3.2 2021-04-29 [1] CRAN (R 4.3.0)
#> evaluate 0.21 2023-05-05 [1] CRAN (R 4.3.0)
#> fansi 1.0.4 2023-01-22 [1] CRAN (R 4.3.0)
#> fastmap 1.1.1 2023-02-24 [1] CRAN (R 4.3.0)
#> fs 1.6.2 2023-04-25 [1] CRAN (R 4.3.0)
#> generics 0.1.3 2022-07-05 [1] CRAN (R 4.3.0)
#> glue 1.6.2 2022-02-24 [1] CRAN (R 4.3.0)
#> hms 1.1.3 2023-03-21 [1] CRAN (R 4.3.0)
#> htmltools 0.5.5 2023-03-23 [1] CRAN (R 4.3.0)
#> httr 1.4.6 2023-05-08 [1] CRAN (R 4.3.0)
#> jsonlite 1.8.4 2022-12-06 [1] CRAN (R 4.3.0)
#> knitr 1.43 2023-05-25 [1] CRAN (R 4.3.0)
#> later 1.3.1 2023-05-02 [1] CRAN (R 4.3.0)
#> lifecycle 1.0.3 2022-10-07 [1] CRAN (R 4.3.0)
#> magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.3.0)
#> paws.common 0.5.6 2023-03-06 [1] CRAN (R 4.3.0)
#> paws.compute 0.2.0 2023-02-08 [1] CRAN (R 4.3.0)
#> paws.developer.tools 0.2.0 2023-02-08 [1] CRAN (R 4.3.0)
#> paws.machine.learning 0.2.0 2023-02-08 [1] CRAN (R 4.3.0)
#> paws.management 0.2.0 2023-02-08 [1] CRAN (R 4.3.0)
#> paws.security.identity 0.2.0 2023-02-08 [1] CRAN (R 4.3.0)
#> paws.storage 0.2.0 2023-02-08 [1] CRAN (R 4.3.0)
#> pillar 1.9.0 2023-03-22 [1] CRAN (R 4.3.0)
#> pins * 1.2.0 2023-05-18 [1] CRAN (R 4.3.0)
#> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.3.0)
#> plumber 1.2.1 2022-09-06 [1] CRAN (R 4.3.0)
#> promises 1.2.0.1 2021-02-11 [1] CRAN (R 4.3.0)
#> purrr 1.0.1 2023-01-10 [1] CRAN (R 4.3.0)
#> R.cache 0.16.0 2022-07-21 [1] CRAN (R 4.3.0)
#> R.methodsS3 1.8.2 2022-06-13 [1] CRAN (R 4.3.0)
#> R.oo 1.25.0 2022-06-12 [1] CRAN (R 4.3.0)
#> R.utils 2.12.2 2022-11-11 [1] CRAN (R 4.3.0)
#> R6 2.5.1 2021-08-19 [1] CRAN (R 4.3.0)
#> rappdirs 0.3.3 2021-01-31 [1] CRAN (R 4.3.0)
#> Rcpp 1.0.10 2023-01-22 [1] CRAN (R 4.3.0)
#> readr 2.1.4 2023-02-10 [1] CRAN (R 4.3.0)
#> reprex 2.0.2 2022-08-17 [1] CRAN (R 4.3.0)
#> rlang 1.1.1 2023-04-28 [1] CRAN (R 4.3.0)
#> rmarkdown 2.22 2023-06-01 [1] CRAN (R 4.3.0)
#> rstudioapi 0.14 2022-08-22 [1] CRAN (R 4.3.0)
#> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.3.0)
#> smdocker 0.1.3 2023-05-15 [1] CRAN (R 4.3.0)
#> stringi 1.7.12 2023-01-11 [1] CRAN (R 4.3.0)
#> styler 1.10.1 2023-06-05 [1] CRAN (R 4.3.0)
#> swagger 3.33.1 2020-10-02 [1] CRAN (R 4.3.0)
#> tibble 3.2.1 2023-03-20 [1] CRAN (R 4.3.0)
#> tidyselect 1.2.0 2022-10-10 [1] CRAN (R 4.3.0)
#> tzdb 0.4.0 2023-05-12 [1] CRAN (R 4.3.0)
#> utf8 1.2.3 2023-01-31 [1] CRAN (R 4.3.0)
#> vctrs 0.6.2 2023-04-19 [1] CRAN (R 4.3.0)
#> vetiver * 0.2.1.9000 2023-06-06 [1] Github (rstudio/vetiver-r@defb903)
#> vroom 1.6.3 2023-04-28 [1] CRAN (R 4.3.0)
#> webutils 1.1 2020-04-28 [1] CRAN (R 4.3.0)
#> withr 2.5.0 2022-03-03 [1] CRAN (R 4.3.0)
#> xfun 0.39 2023-04-20 [1] CRAN (R 4.3.0)
#> xml2 1.3.4 2023-04-27 [1] CRAN (R 4.3.0)
#> yaml 2.3.7 2023-01-23 [1] CRAN (R 4.3.0)
#> zip 2.3.0 2023-04-17 [1] CRAN (R 4.3.0)
#>
#> [1] C:/Users/maadi/AppData/Local/R/win-library/4.3
#> [2] C:/Program Files/R/R-4.3.0/library
#>
#> ──────────────────────────────────────────────────────────────────────────────I am working off the free version of RStudio Desktop on my Mac, not within RStudio IDE on SageMaker. Is vetiver_deploy_sagemaker()only supposed to work within the paid RStudio SageMaker IDE?
You can do all this part locally in a straightforward way (as long as the bucket is one that you have access to from SageMaker):
library(vetiver)
cars_lm <- lm(mpg ~ ., data=mtcars)
v <- vetiver_model(cars_lm, "cars-linear-test")
library(pins)
identifier <- "sagemaker-vetiver"
board <- board_s3(bucket = identifier, region = "ca-central-1")
vetiver_pin_write(board, v)When it comes to vetiver_deploy_sagemaker(), that does make some assumptions about where you are executing, specifically that you have the SageMaker Studio execution role. If you want to deploy to SageMaker from your local machine, you'll need to use the three more modular functions. Especially look at the role arg and the related options there.
These functions build on top of smdocker by @DyfanJones so it may be helpful to look at that as well.
As @juliasilge has mentioned you will need to have an execution role so that smdocker can build your docker image
on AWS CodeBuild and for AWS Sagemaker to host your model on a rest api. Finally you will need to make sure your own role has the correct AWS Sagemaker permissions, so that you can create an AWS SageMaker Model
library(vetiver)
cars_lm <- lm(mpg ~ ., data=mtcars)
v <- vetiver_model(cars_lm, "cars-linear-test")
library(pins)
identifier <- "sagemaker-vetiver"
board <- board_s3(bucket = identifier, region = "ca-central-1")
vetiver_pin_write(board, v)execution_role <- "arn:aws:iam::1234567890:role/sagemaker-role"
# Sends dockerfile to AWS Codebuild for docker build
image_uri <- vetiver_sm_build(
board = board,
name = name,
predict_args = predict_args,
docker_args = docker_args,
repository = glue::glue("vetiver-sagemaker-{name}:{strftime(Sys.time(), '%Y-%m-%d')}"),
bucket = board$bucket
role = execution_role
)# Creates AWS Sagemaker Model
model_name <- vetiver_sm_model(image_uri)
# Creates AWS Sagemaker Model Endpoint
endpoint <- vetiver_sm_endpoint(
model_name = model_name,
instance_type = "ml.t2.medium"
role = execution_role
)You can also run R on Sagemaker Studio as well, however you will have to provide a docker image:
https://aws.amazon.com/blogs/machine-learning/bringing-your-own-r-environment-to-amazon-sagemaker-studio/
I hope this helps 😄
Thanks so much @DyfanJones!
I just opened a PR to clarify the documentation around those two roles (build vs. model); I think it was not clear enough before.
Thank you @juliasilge and @DyfanJones for clarifications. I am still having issues with vetiver_sm_build() though. The errors are not very informative Error in eval(parse(text = text, keep.source = FALSE), envir): object 'name' not found:
library(vetiver)
cars_lm <- lm(mpg ~ ., data=mtcars)
v <- vetiver_model(cars_lm, "cars-linear-test")
library(pins)
identifier <- "vetiver-sagemaker"
board <- board_s3(bucket = identifier, region = "us-east-1")
vetiver_pin_write(board, v)
#> Creating new version '20230606T210132Z-2195d'
#> Writing to pin 'cars-linear-test'
#>
#> Create a Model Card for your published model
#> • Model Cards provide a framework for transparent, responsible reporting
#> • Use the vetiver `.Rmd` template as a place to start
execution_role <- "arn:aws:iam::922336262886:role/vetiver-test"
# Sends dockerfile to AWS Codebuild for docker build
image_uri <- vetiver_sm_build(
board = board,
name = "cars-linear-test",
# predict_args = predict_args,
# docker_args = docker_args,
repository = glue::glue("vetiver-sagemaker-{name}:{strftime(Sys.time(), '%Y-%m-%d')}"),
bucket = board$bucket,
role = execution_role
)
#> The version of R recorded in the lockfile will be updated:
#> - R [* -> 4.3.0]
#>
#> * Lockfile written to 'vetiver_renv.lock'.
#> Error in eval(parse(text = text, keep.source = FALSE), envir): object 'name' not foundCreated on 2023-06-06 with reprex v2.0.2
Session info
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#> setting value
#> version R version 4.3.0 (2023-04-21 ucrt)
#> os Windows 11 x64 (build 22621)
#> system x86_64, mingw32
#> ui RTerm
#> language (EN)
#> collate English_Canada.utf8
#> ctype English_Canada.utf8
#> tz America/Vancouver
#> date 2023-06-06
#> pandoc 2.19.2 @ C:/Program Files/RStudio/resources/app/bin/quarto/bin/tools/ (via rmarkdown)
#>
#> ─ Packages ───────────────────────────────────────────────────────────────────
#> package * version date (UTC) lib source
#> base64enc 0.1-3 2015-07-28 [1] CRAN (R 4.3.0)
#> bit 4.0.5 2022-11-15 [1] CRAN (R 4.3.0)
#> bit64 4.0.5 2020-08-30 [1] CRAN (R 4.3.0)
#> bundle 0.1.0 2022-09-15 [1] CRAN (R 4.3.0)
#> butcher 0.3.2 2023-03-08 [1] CRAN (R 4.3.0)
#> cli 3.6.1 2023-03-23 [1] CRAN (R 4.3.0)
#> crayon 1.5.2 2022-09-29 [1] CRAN (R 4.3.0)
#> curl 5.0.0 2023-01-12 [1] CRAN (R 4.3.0)
#> digest 0.6.31 2022-12-11 [1] CRAN (R 4.3.0)
#> ellipsis 0.3.2 2021-04-29 [1] CRAN (R 4.3.0)
#> evaluate 0.21 2023-05-05 [1] CRAN (R 4.3.0)
#> fansi 1.0.4 2023-01-22 [1] CRAN (R 4.3.0)
#> fastmap 1.1.1 2023-02-24 [1] CRAN (R 4.3.0)
#> fs 1.6.2 2023-04-25 [1] CRAN (R 4.3.0)
#> generics 0.1.3 2022-07-05 [1] CRAN (R 4.3.0)
#> glue 1.6.2 2022-02-24 [1] CRAN (R 4.3.0)
#> hms 1.1.3 2023-03-21 [1] CRAN (R 4.3.0)
#> htmltools 0.5.5 2023-03-23 [1] CRAN (R 4.3.0)
#> httr 1.4.6 2023-05-08 [1] CRAN (R 4.3.0)
#> jsonlite 1.8.4 2022-12-06 [1] CRAN (R 4.3.0)
#> knitr 1.43 2023-05-25 [1] CRAN (R 4.3.0)
#> later 1.3.1 2023-05-02 [1] CRAN (R 4.3.0)
#> lifecycle 1.0.3 2022-10-07 [1] CRAN (R 4.3.0)
#> magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.3.0)
#> mime 0.12 2021-09-28 [1] CRAN (R 4.3.0)
#> paws.common 0.5.6 2023-03-06 [1] CRAN (R 4.3.0)
#> paws.compute 0.2.0 2023-02-08 [1] CRAN (R 4.3.0)
#> paws.developer.tools 0.2.0 2023-02-08 [1] CRAN (R 4.3.0)
#> paws.machine.learning 0.2.0 2023-02-08 [1] CRAN (R 4.3.0)
#> paws.management 0.2.0 2023-02-08 [1] CRAN (R 4.3.0)
#> paws.security.identity 0.2.0 2023-02-08 [1] CRAN (R 4.3.0)
#> paws.storage 0.2.0 2023-02-08 [1] CRAN (R 4.3.0)
#> pillar 1.9.0 2023-03-22 [1] CRAN (R 4.3.0)
#> pins * 1.2.0 2023-05-18 [1] CRAN (R 4.3.0)
#> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.3.0)
#> plumber 1.2.1 2022-09-06 [1] CRAN (R 4.3.0)
#> promises 1.2.0.1 2021-02-11 [1] CRAN (R 4.3.0)
#> purrr 1.0.1 2023-01-10 [1] CRAN (R 4.3.0)
#> R.cache 0.16.0 2022-07-21 [1] CRAN (R 4.3.0)
#> R.methodsS3 1.8.2 2022-06-13 [1] CRAN (R 4.3.0)
#> R.oo 1.25.0 2022-06-12 [1] CRAN (R 4.3.0)
#> R.utils 2.12.2 2022-11-11 [1] CRAN (R 4.3.0)
#> R6 2.5.1 2021-08-19 [1] CRAN (R 4.3.0)
#> rappdirs 0.3.3 2021-01-31 [1] CRAN (R 4.3.0)
#> Rcpp 1.0.10 2023-01-22 [1] CRAN (R 4.3.0)
#> readr 2.1.4 2023-02-10 [1] CRAN (R 4.3.0)
#> reprex 2.0.2 2022-08-17 [1] CRAN (R 4.3.0)
#> rlang 1.1.1 2023-04-28 [1] CRAN (R 4.3.0)
#> rmarkdown 2.22 2023-06-01 [1] CRAN (R 4.3.0)
#> rstudioapi 0.14 2022-08-22 [1] CRAN (R 4.3.0)
#> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.3.0)
#> smdocker 0.1.3 2023-05-15 [1] CRAN (R 4.3.0)
#> stringi 1.7.12 2023-01-11 [1] CRAN (R 4.3.0)
#> styler 1.10.1 2023-06-05 [1] CRAN (R 4.3.0)
#> swagger 3.33.1 2020-10-02 [1] CRAN (R 4.3.0)
#> tibble 3.2.1 2023-03-20 [1] CRAN (R 4.3.0)
#> tidyselect 1.2.0 2022-10-10 [1] CRAN (R 4.3.0)
#> tzdb 0.4.0 2023-05-12 [1] CRAN (R 4.3.0)
#> utf8 1.2.3 2023-01-31 [1] CRAN (R 4.3.0)
#> vctrs 0.6.2 2023-04-19 [1] CRAN (R 4.3.0)
#> vetiver * 0.2.1.9000 2023-06-06 [1] Github (rstudio/vetiver-r@defb903)
#> vroom 1.6.3 2023-04-28 [1] CRAN (R 4.3.0)
#> webutils 1.1 2020-04-28 [1] CRAN (R 4.3.0)
#> withr 2.5.0 2022-03-03 [1] CRAN (R 4.3.0)
#> xfun 0.39 2023-04-20 [1] CRAN (R 4.3.0)
#> xml2 1.3.4 2023-04-27 [1] CRAN (R 4.3.0)
#> yaml 2.3.7 2023-01-23 [1] CRAN (R 4.3.0)
#> zip 2.3.0 2023-04-17 [1] CRAN (R 4.3.0)
#>
#> [1] C:/Users/maadi/AppData/Local/R/win-library/4.3
#> [2] C:/Program Files/R/R-4.3.0/library
#>
#> ──────────────────────────────────────────────────────────────────────────────For the execution role, I have set the trust policy as
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"codebuild.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
}
and permissions as
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"codebuild:DeleteProject",
"codebuild:CreateProject",
"codebuild:BatchGetBuilds",
"codebuild:StartBuild"
],
"Resource": "arn:aws:codebuild:*:*:project/sagemaker-studio*"
},
{
"Effect": "Allow",
"Action": "logs:CreateLogStream",
"Resource": "arn:aws:logs:*:*:log-group:/aws/codebuild/sagemaker-studio*"
},
{
"Effect": "Allow",
"Action": [
"logs:GetLogEvents",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:*:*:log-group:/aws/codebuild/sagemaker-studio*:log-stream:*"
},
{
"Effect": "Allow",
"Action": "logs:CreateLogGroup",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"ecr:CreateRepository",
"ecr:BatchGetImage",
"ecr:CompleteLayerUpload",
"ecr:DescribeImages",
"ecr:DescribeRepositories",
"ecr:UploadLayerPart",
"ecr:ListImages",
"ecr:InitiateLayerUpload",
"ecr:BatchCheckLayerAvailability",
"ecr:PutImage"
],
"Resource": "arn:aws:ecr:*:*:repository/sagemaker-studio*"
},
{
"Sid": "ReadAccessToPrebuiltAwsImages",
"Effect": "Allow",
"Action": [
"ecr:BatchGetImage",
"ecr:GetDownloadUrlForLayer"
],
"Resource": [
"arn:aws:ecr:*:763104351884:repository/*",
"arn:aws:ecr:*:217643126080:repository/*",
"arn:aws:ecr:*:727897471807:repository/*",
"arn:aws:ecr:*:626614931356:repository/*",
"arn:aws:ecr:*:683313688378:repository/*",
"arn:aws:ecr:*:520713654638:repository/*",
"arn:aws:ecr:*:462105765813:repository/*"
]
},
{
"Sid": "EcrAuthorizationTokenRetrieval",
"Effect": "Allow",
"Action": [
"ecr:GetAuthorizationToken"
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:DeleteObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::sagemaker-*/*"
},
{
"Effect": "Allow",
"Action": [
"s3:CreateBucket"
],
"Resource": "arn:aws:s3:::sagemaker*"
},
{
"Effect": "Allow",
"Action": [
"iam:GetRole",
"iam:ListRoles"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "arn:aws:iam::*:role/*",
"Condition": {
"StringLikeIfExists": {
"iam:PassedToService": "codebuild.amazonaws.com"
}
}
}
]
}
Ah @aminadibi that is my fault, in the repository parameter I put
glue::glue("vetiver-sagemaker-{name}:{strftime(Sys.time(), '%Y-%m-%d')}")But I didn't define name, my bad.
name <- "cars-linear-test"
image_uri <- vetiver_sm_build(
board = board,
name = name,
# predict_args = predict_args,
# docker_args = docker_args,
repository = glue::glue("vetiver-sagemaker-{name}:{strftime(Sys.time(), '%Y-%m-%d')}"),
bucket = board$bucket,
role = execution_role
)I believe this should work
Oops! How did I miss that?
It works now from my local desktop RStudio. There is only one peculiarity though: it seems that vetiver_sm_build expects the role name alone, without the full ARN, while vetiver_sm_model requires the role to be passed as the full ARN. See below:
execution_role <- "arn:aws:iam::922336262886:role/vetiver-test-3"
name <- "cars-linear-test"
# Sends dockerfile to AWS Codebuild for docker build
image_uri <- vetiver_sm_build(
board = board,
name = name,
repository = glue::glue("vetiver-sagemaker-{name}:{strftime(Sys.time(), '%Y-%m-%d')}"),
bucket = board$bucket,
role = execution_role
)
#> The version of R recorded in the lockfile will be updated:
#> - R [* -> 4.3.0]
#>
#> * Lockfile written to 'vetiver_renv.lock'.
#> Error: InvalidInputException (HTTP 400). CodeBuild is not authorized to perform: sts:AssumeRole on arn:aws:iam::922336262886:role/arn:aws:iam::922336262886:role/vetiver-test-3Created on 2023-06-06 with reprex v2.0.2
Also found a typo in the documentation for the role arg. Should be "the ARN IAM role names", not "ARM IAM role names". Will send a pull request.
It looks like AWS CodeBuild can't assume execution role: arn:aws:iam::922336262886:role/vetiver-test-3 make sure you have the policy attached:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"codebuild.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
}
It looks like AWS CodeBuild can't assume execution role:
arn:aws:iam::922336262886:role/vetiver-test-3make sure you have the policy attached:
Oh I have the policy attached and have no problem when I pass the role arg as the
vetiver_sm_build() as execution_role <-"vetiver-test-3".
This works:
execution_role <- "vetiver-test-3"
name <- "cars-linear-test"
# Sends dockerfile to AWS Codebuild for docker build
image_uri <- vetiver_sm_build(
board = board,
name = name,
repository = glue::glue("vetiver-sagemaker-{name}:{strftime(Sys.time(), '%Y-%m-%d')}"),
bucket = board$bucket,
role = execution_role
But when I pass the role arg as the full ARN "arn:aws:iam::922336262886:role/vetiver-test-3", sm_build() duplicates the ARN string which results in a CodeBuild exceution role error, as per my reprex above:
CodeBuild is not authorized to perform: sts:AssumeRole on arn:aws:iam::922336262886:role/arn:aws:iam::922336262886:role/vetiver-test-3
It's fine, except that vetiver_sm_model() requires the full ARN. Would be nice to make the role arg consistent among the two functions.
Ah I see, I will raise this on smdocker so that it doesn't duplicate arn roles. Thanks for pointing this out :)
@aminadibi smdocker 0.1.4 is now on the cran with the fix to full arn. This should make the behaviour of using roles from vetiver consistent :)
Let us know if you have further questions @aminadibi!
@DyfanJones please forgive my ignorance, can you elaborate on what roles and policies are needed to use vetiver_sm_build from local R to push a model to sagemaker? Thanks
@jgarrigan Did you check out the documentation for smdocker (what vetiver_sm_build() uses)?
https://dyfanjones.r-universe.dev/smdocker
There are details there on role and policies.
hi all,
AWS has an IAM template role called: AmazonSageMakerFullAccess. This execution role should have all the permissions you need to work from local :) So when you create your role, if you select the template you should be ready to go (policy below).
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sagemaker:*",
"sagemaker-geospatial:*"
],
"NotResource": [
"arn:aws:sagemaker:*:*:domain/*",
"arn:aws:sagemaker:*:*:user-profile/*",
"arn:aws:sagemaker:*:*:app/*",
"arn:aws:sagemaker:*:*:space/*",
"arn:aws:sagemaker:*:*:flow-definition/*"
]
},
{
"Effect": "Allow",
"Action": [
"sagemaker:AddTags"
],
"Resource": [
"arn:aws:sagemaker:*:*:app/*"
]
},
{
"Effect": "Allow",
"Action": [
"sagemaker:CreatePresignedDomainUrl",
"sagemaker:DescribeDomain",
"sagemaker:ListDomains",
"sagemaker:DescribeUserProfile",
"sagemaker:ListUserProfiles",
"sagemaker:DescribeSpace",
"sagemaker:ListSpaces",
"sagemaker:*App",
"sagemaker:ListApps"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "sagemaker:*",
"Resource": [
"arn:aws:sagemaker:*:*:flow-definition/*"
],
"Condition": {
"StringEqualsIfExists": {
"sagemaker:WorkteamType": [
"private-crowd",
"vendor-crowd"
]
}
}
},
{
"Effect": "Allow",
"Action": [
"application-autoscaling:DeleteScalingPolicy",
"application-autoscaling:DeleteScheduledAction",
"application-autoscaling:DeregisterScalableTarget",
"application-autoscaling:DescribeScalableTargets",
"application-autoscaling:DescribeScalingActivities",
"application-autoscaling:DescribeScalingPolicies",
"application-autoscaling:DescribeScheduledActions",
"application-autoscaling:PutScalingPolicy",
"application-autoscaling:PutScheduledAction",
"application-autoscaling:RegisterScalableTarget",
"aws-marketplace:ViewSubscriptions",
"cloudformation:GetTemplateSummary",
"cloudwatch:DeleteAlarms",
"cloudwatch:DescribeAlarms",
"cloudwatch:GetMetricData",
"cloudwatch:GetMetricStatistics",
"cloudwatch:ListMetrics",
"cloudwatch:PutMetricAlarm",
"cloudwatch:PutMetricData",
"codecommit:BatchGetRepositories",
"codecommit:CreateRepository",
"codecommit:GetRepository",
"codecommit:List*",
"cognito-idp:AdminAddUserToGroup",
"cognito-idp:AdminCreateUser",
"cognito-idp:AdminDeleteUser",
"cognito-idp:AdminDisableUser",
"cognito-idp:AdminEnableUser",
"cognito-idp:AdminRemoveUserFromGroup",
"cognito-idp:CreateGroup",
"cognito-idp:CreateUserPool",
"cognito-idp:CreateUserPoolClient",
"cognito-idp:CreateUserPoolDomain",
"cognito-idp:DescribeUserPool",
"cognito-idp:DescribeUserPoolClient",
"cognito-idp:List*",
"cognito-idp:UpdateUserPool",
"cognito-idp:UpdateUserPoolClient",
"ec2:CreateNetworkInterface",
"ec2:CreateNetworkInterfacePermission",
"ec2:CreateVpcEndpoint",
"ec2:DeleteNetworkInterface",
"ec2:DeleteNetworkInterfacePermission",
"ec2:DescribeDhcpOptions",
"ec2:DescribeNetworkInterfaces",
"ec2:DescribeRouteTables",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeVpcEndpoints",
"ec2:DescribeVpcs",
"ecr:BatchCheckLayerAvailability",
"ecr:BatchGetImage",
"ecr:CreateRepository",
"ecr:Describe*",
"ecr:GetAuthorizationToken",
"ecr:GetDownloadUrlForLayer",
"ecr:StartImageScan",
"elastic-inference:Connect",
"elasticfilesystem:DescribeFileSystems",
"elasticfilesystem:DescribeMountTargets",
"fsx:DescribeFileSystems",
"glue:CreateJob",
"glue:DeleteJob",
"glue:GetJob*",
"glue:GetTable*",
"glue:GetWorkflowRun",
"glue:ResetJobBookmark",
"glue:StartJobRun",
"glue:StartWorkflowRun",
"glue:UpdateJob",
"groundtruthlabeling:*",
"iam:ListRoles",
"kms:DescribeKey",
"kms:ListAliases",
"lambda:ListFunctions",
"logs:CreateLogDelivery",
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:DeleteLogDelivery",
"logs:Describe*",
"logs:GetLogDelivery",
"logs:GetLogEvents",
"logs:ListLogDeliveries",
"logs:PutLogEvents",
"logs:PutResourcePolicy",
"logs:UpdateLogDelivery",
"robomaker:CreateSimulationApplication",
"robomaker:DescribeSimulationApplication",
"robomaker:DeleteSimulationApplication",
"robomaker:CreateSimulationJob",
"robomaker:DescribeSimulationJob",
"robomaker:CancelSimulationJob",
"secretsmanager:ListSecrets",
"servicecatalog:Describe*",
"servicecatalog:List*",
"servicecatalog:ScanProvisionedProducts",
"servicecatalog:SearchProducts",
"servicecatalog:SearchProvisionedProducts",
"sns:ListTopics",
"tag:GetResources"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"ecr:SetRepositoryPolicy",
"ecr:CompleteLayerUpload",
"ecr:BatchDeleteImage",
"ecr:UploadLayerPart",
"ecr:DeleteRepositoryPolicy",
"ecr:InitiateLayerUpload",
"ecr:DeleteRepository",
"ecr:PutImage"
],
"Resource": [
"arn:aws:ecr:*:*:repository/*sagemaker*"
]
},
{
"Effect": "Allow",
"Action": [
"codecommit:GitPull",
"codecommit:GitPush"
],
"Resource": [
"arn:aws:codecommit:*:*:*sagemaker*",
"arn:aws:codecommit:*:*:*SageMaker*",
"arn:aws:codecommit:*:*:*Sagemaker*"
]
},
{
"Action": [
"codebuild:BatchGetBuilds",
"codebuild:StartBuild"
],
"Resource": [
"arn:aws:codebuild:*:*:project/sagemaker*",
"arn:aws:codebuild:*:*:build/*"
],
"Effect": "Allow"
},
{
"Action": [
"states:DescribeExecution",
"states:GetExecutionHistory",
"states:StartExecution",
"states:StopExecution",
"states:UpdateStateMachine"
],
"Resource": [
"arn:aws:states:*:*:statemachine:*sagemaker*",
"arn:aws:states:*:*:execution:*sagemaker*:*"
],
"Effect": "Allow"
},
{
"Effect": "Allow",
"Action": [
"secretsmanager:DescribeSecret",
"secretsmanager:GetSecretValue",
"secretsmanager:CreateSecret"
],
"Resource": [
"arn:aws:secretsmanager:*:*:secret:AmazonSageMaker-*"
]
},
{
"Effect": "Allow",
"Action": [
"secretsmanager:DescribeSecret",
"secretsmanager:GetSecretValue"
],
"Resource": "*",
"Condition": {
"StringEquals": {
"secretsmanager:ResourceTag/SageMaker": "true"
}
}
},
{
"Effect": "Allow",
"Action": [
"servicecatalog:ProvisionProduct"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"servicecatalog:TerminateProvisionedProduct",
"servicecatalog:UpdateProvisionedProduct"
],
"Resource": "*",
"Condition": {
"StringEquals": {
"servicecatalog:userLevel": "self"
}
}
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject",
"s3:AbortMultipartUpload"
],
"Resource": [
"arn:aws:s3:::*SageMaker*",
"arn:aws:s3:::*Sagemaker*",
"arn:aws:s3:::*sagemaker*",
"arn:aws:s3:::*aws-glue*"
]
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::*"
],
"Condition": {
"StringEqualsIgnoreCase": {
"s3:ExistingObjectTag/SageMaker": "true"
}
}
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::*"
],
"Condition": {
"StringEquals": {
"s3:ExistingObjectTag/servicecatalog:provisioning": "true"
}
}
},
{
"Effect": "Allow",
"Action": [
"s3:CreateBucket",
"s3:GetBucketLocation",
"s3:ListBucket",
"s3:ListAllMyBuckets",
"s3:GetBucketCors",
"s3:PutBucketCors"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"s3:GetBucketAcl",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::*SageMaker*",
"arn:aws:s3:::*Sagemaker*",
"arn:aws:s3:::*sagemaker*"
]
},
{
"Effect": "Allow",
"Action": [
"lambda:InvokeFunction"
],
"Resource": [
"arn:aws:lambda:*:*:function:*SageMaker*",
"arn:aws:lambda:*:*:function:*sagemaker*",
"arn:aws:lambda:*:*:function:*Sagemaker*",
"arn:aws:lambda:*:*:function:*LabelingFunction*"
]
},
{
"Action": "iam:CreateServiceLinkedRole",
"Effect": "Allow",
"Resource": "arn:aws:iam::*:role/aws-service-role/sagemaker.application-autoscaling.amazonaws.com/AWSServiceRoleForApplicationAutoScaling_SageMakerEndpoint",
"Condition": {
"StringLike": {
"iam:AWSServiceName": "sagemaker.application-autoscaling.amazonaws.com"
}
}
},
{
"Effect": "Allow",
"Action": "iam:CreateServiceLinkedRole",
"Resource": "*",
"Condition": {
"StringEquals": {
"iam:AWSServiceName": "robomaker.amazonaws.com"
}
}
},
{
"Effect": "Allow",
"Action": [
"sns:Subscribe",
"sns:CreateTopic",
"sns:Publish"
],
"Resource": [
"arn:aws:sns:*:*:*SageMaker*",
"arn:aws:sns:*:*:*Sagemaker*",
"arn:aws:sns:*:*:*sagemaker*"
]
},
{
"Effect": "Allow",
"Action": [
"iam:PassRole"
],
"Resource": "arn:aws:iam::*:role/*AmazonSageMaker*",
"Condition": {
"StringEquals": {
"iam:PassedToService": [
"glue.amazonaws.com",
"robomaker.amazonaws.com",
"states.amazonaws.com"
]
}
}
},
{
"Effect": "Allow",
"Action": [
"iam:PassRole"
],
"Resource": "arn:aws:iam::*:role/*",
"Condition": {
"StringEquals": {
"iam:PassedToService": "sagemaker.amazonaws.com"
}
}
},
{
"Effect": "Allow",
"Action": [
"athena:ListDataCatalogs",
"athena:ListDatabases",
"athena:ListTableMetadata",
"athena:GetQueryExecution",
"athena:GetQueryResults",
"athena:StartQueryExecution",
"athena:StopQueryExecution"
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"glue:CreateTable"
],
"Resource": [
"arn:aws:glue:*:*:table/*/sagemaker_tmp_*",
"arn:aws:glue:*:*:table/sagemaker_featurestore/*",
"arn:aws:glue:*:*:catalog",
"arn:aws:glue:*:*:database/*"
]
},
{
"Effect": "Allow",
"Action": [
"glue:UpdateTable"
],
"Resource": [
"arn:aws:glue:*:*:table/sagemaker_featurestore/*",
"arn:aws:glue:*:*:catalog",
"arn:aws:glue:*:*:database/sagemaker_featurestore"
]
},
{
"Effect": "Allow",
"Action": [
"glue:DeleteTable"
],
"Resource": [
"arn:aws:glue:*:*:table/*/sagemaker_tmp_*",
"arn:aws:glue:*:*:catalog",
"arn:aws:glue:*:*:database/*"
]
},
{
"Effect": "Allow",
"Action": [
"glue:GetDatabases",
"glue:GetTable",
"glue:GetTables"
],
"Resource": [
"arn:aws:glue:*:*:table/*",
"arn:aws:glue:*:*:catalog",
"arn:aws:glue:*:*:database/*"
]
},
{
"Effect": "Allow",
"Action": [
"glue:CreateDatabase",
"glue:GetDatabase"
],
"Resource": [
"arn:aws:glue:*:*:catalog",
"arn:aws:glue:*:*:database/sagemaker_featurestore",
"arn:aws:glue:*:*:database/sagemaker_processing",
"arn:aws:glue:*:*:database/default",
"arn:aws:glue:*:*:database/sagemaker_data_wrangler"
]
},
{
"Effect": "Allow",
"Action": [
"redshift-data:ExecuteStatement",
"redshift-data:DescribeStatement",
"redshift-data:CancelStatement",
"redshift-data:GetStatementResult",
"redshift-data:ListSchemas",
"redshift-data:ListTables"
],
"Resource": [
"*"
]
},
{
"Effect": "Allow",
"Action": [
"redshift:GetClusterCredentials"
],
"Resource": [
"arn:aws:redshift:*:*:dbuser:*/sagemaker_access*",
"arn:aws:redshift:*:*:dbname:*"
]
},
{
"Effect": "Allow",
"Action": [
"sagemaker:ListTags"
],
"Resource": [
"arn:aws:sagemaker:*:*:user-profile/*"
]
},
{
"Effect": "Allow",
"Action": [
"cloudformation:ListStackResources"
],
"Resource": "arn:aws:cloudformation:*:*:stack/SC-*"
}
]
}
However, you might need to add your AWS S3 bucket ARN (amazon resource number) to it. The default role only allows s3 buckets that have "sagemaker" in the bucket title. So if you bucket is called "my-bucket" it will fail. You can add your bucket to the resources and it should work for you :)
{
... # rest of the IAM role policy
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject",
"s3:AbortMultipartUpload"
],
"Resource": [
"arn:aws:s3:::*SageMaker*",
"arn:aws:s3:::*Sagemaker*",
"arn:aws:s3:::*sagemaker*",
"arn:aws:s3:::*my-bucket*", # add custom s3 bucket
"arn:aws:s3:::*aws-glue*"
]
}
... # rest of the IAM role policy
}
I hope this helps :)
Hi everyone,
Thanks a lot for this discussion, it really helped me a lot!
However, I'm still struggling to deploy a Vetiver API in SageMaker from my local Rstudio (not within SageMaker). Following your input in this issue, I managed to set up my ARN IAM roles accordingly so that vetiver_sm_build() and vetiver_sm_model() run without any further problems. However, I am somehow failing to set up an API endpoint with vetiver_sm_endpoint().
I am running the following code where I am using an execution role setup according to your explanations above and also my aws_acces_key_id and aws_secret_access_key are provided in .Renviron.
library(vetiver)
library(pins)
name <- "cars-lm"
lm_model <- lm(mpg ~ ., data = mtcars)
v_model <-
vetiver_model(
lm_model,
name
)
model_board <-
board_s3(
bucket = "mlr4all-test-bucket",
region = "us-east-1"
)
model_board %>%
vetiver_pin_write(v_model)
execution_role <- "arn:aws:iam::069650456288:role/vetivertest"
new_image_uri <-
vetiver_sm_build(
board = model_board,
name = name,
repository = glue::glue("vetiver-sagemaker-{name}:{strftime(Sys.time(), '%Y-%m-%d')}"),
role = execution_role,
bucket = model_board$bucket
)
model_name <-
vetiver_sm_model(
image_uri = new_image_uri,
role = execution_role
)
new_endpoint <-
vetiver_sm_endpoint(
model_name = model_name,
instance_type = "ml.t2.medium"
)Everything works smoothly until vetiver_sm_endpoint(). There I get the following error message:
When I check CloudWatch for the respective endpoint I find the following:
I don't really understand that error message and I also don't know where this error could come from.
Any help would be really nice!
Thank you guys a lot for your work!
@stefanlinner Do you mind opening a new issue with this? Sounds like there's a slightly different auth problem.
Yes sure!