samhstn/website

Speed codebuild using cache

Closed this issue · 3 comments

samhstn commented

Our current codebuild takes around 214 seconds and has the following phase details:

SUBMITTED	<1 sec	
QUEUED	<1 sec	
PROVISIONING	60 secs	
DOWNLOAD_SOURCE	2 secs	
INSTALL	1 sec	
PRE_BUILD	39 secs	
BUILD	41 secs	
POST_BUILD	69 secs	
UPLOAD_ARTIFACTS	<1 sec	
FINALIZING	2 secs

It should be possible to speed this up by some form of caching of our _build, deps and priv directories.

  • Reduce subsequent PRE_BUILD and BUILD build phase times by adding a codebuild LOCAL_CACHE.
  • Configure our MasterBuildProject to use it's cache and upload it's cache to s3.
  • Attempt to download a master cache from s3 for an initial, new branch build.
  • Experiment with other caching options and see if it affects build time - possibly looking to reduce the PROVISIONING build time.
samhstn commented

Running in to the following build error after adding cache in buildspec.yml:

...

===> Compiling cowboy
--
66 | src/cowboy_clear.erl:16: Warning: behaviour ranch_protocol undefined
67 |  
68 | ===> Compiling src/cowboy_http.erl failed
69 | src/cowboy_http.erl:154: can't find include lib "cowlib/include/cow_inline.hrl"; Make sure cowlib is in your app file's 'applications' list
70 | src/cowboy_http.erl:155: can't find include lib "cowlib/include/cow_parse.hrl"; Make sure cowlib is in your app file's 'applications' list
71 | src/cowboy_http.erl:482: undefined macro 'IS_TOKEN/1'
72 | src/cowboy_http.erl:628: undefined macro 'IS_WS/1'
73 |  
74 | src/cowboy_http.erl:462: function parse_method/4 undefined
75 | src/cowboy_http.erl:616: function parse_hd_name/4 undefined
76 |  
77 | src/cowboy_http.erl:487: Warning: function parse_uri/3 is unused
78 | src/cowboy_http.erl:504: Warning: function parse_uri_authority/3 is unused
79 | src/cowboy_http.erl:508: Warning: function parse_uri_authority/5 is unused
80 | src/cowboy_http.erl:533: Warning: function parse_uri_path/5 is unused
81 | src/cowboy_http.erl:543: Warning: function parse_uri_query/6 is unused
82 | src/cowboy_http.erl:552: Warning: function skip_uri_fragment/6 is unused
83 |  
84 | ==> samhstn
85 | ** (Mix) Could not compile dependency :cowboy, "/root/.mix/rebar3 bare compile --paths /codebuild/output/src496784613/src/github.com/samhstn/samhstn/_build/test/lib/*/ebin" command failed. You can recompile this dependency with "mix deps.compile cowboy", update it with "mix deps.update cowboy" or clean it with "mix deps.clean cowboy"
86 |  
87 | [Container] 2020/11/17 13:35:49 Command did not exit successfully MIX_ENV=test mix compile exit status 1
samhstn commented

Parking this for now (will pick back up if get an answer on ElixirForum): https://elixirforum.com/t/mix-could-not-compile-dependency-cowboy-root-mix-rebar3-bare-compile/35640

  • 👍1
samhstn commented

I've come to the conclusion that codebuilds in-built cache isn't too useful for this use case.

Some things to note:

Instead looking to manually cache using s3 buckets - I think this should still result in a faster build (and will definitely be needed anyway for dialyxir).

  • 👍1
  • 😕1