heroku/base-images

Using heroku stack images for local development

Closed this issue · 19 comments

fabswt commented

Hi,

I'd like to run heroku locally to ease development, i.e.: so as to have the same environment for dev as heroku provides for staging and prod.

I'd like to use the heroku Docker images for that. This way, I wouldn't have to install every requirement manually in OS X, and would be able to juggle between different versions with the same ease as on heroku.

It's not clear yet how I can use those images.

I was able to pull the image with docker pull heroku/heroku:16, then start it with docker run -i -t heroku/heroku:16 /bin/bash.

However, I'm a bit confused as to what to do next. Could you provide some guidelines? e.g.:

How do I (Can I?) configure heroku to deploy locally to the local Docker heroku container, instead of the staging or production environments?

dmathieu commented

Hello there,

When using Docker, you don't need to use the base image we provide (though you absolutely can). You can use any base image you wish.

Docker allows you to set a base image in your Dockerfile with the FROM instruction:

FROM heroku/heroku:16

You can look at our Alpine Hello World app for a very simple example of a Dockerfile.

How do I (Can I?) configure heroku to deploy locally to the local Docker heroku container, instead of the staging or production environments?

There might be a misunderstanding here. When you use your container locally, there is no Heroku involved. You just need to build the container and start it, like any other Docker one.
Only when you wish to push it to our registry will you need to start worrying about us, as the docker push needs to push to registry.heroku.com.
Our registry CLI plugin will handle that for you see the doc.

Note that the big difference between deploying with our container registry and the standard way of using GIT push, GitHub Sync or our API is that the later will allow us to upgrade base libraries such as OpenSSL without the need for a new deployment from you.
We cannot upgrade the base image of your docker container without you triggering a new deployment.

So while we update base packages when there are security issues, we can't do that for docker containers which means that works falls back to you.

fabswt commented

Thank you, that clarifies it a bit.

So if I get that right that means that, even if I use the heroku/heroku:16 image, I still need to install the requirements myself? e.g.: PHP in the specific version I need, the webserver, etc.

Ideally, I wanted to use the exact same environment as heroku uses in production. e.g.: FPM/FastCGI rather than some other version of PHP; identify and enable the same PHP extensions; identify and install the same version of Apache or nginx…

Do I get that right, or is there a simpler way to reproduce the production environment locally?

P.S.: note I don't mean to deploy Docker to heroku. Rather the opposite: I'm using Docker locally only, to try and reproduce the heroku environment locally.

edmorley commented

Do I get that right,

That's correct.

What would be awesome would be a variant of heroku/heroku:16 that includes a buildpack loader that then clones/runs any listed buildpacks, to provide an environment that more closely matches production.

  • 👍12
edmorley commented

Though doing so would be hacky (or slow), since the Dockerfile would have no way of knowing which files the buildpacks required (eg runtimes.txt, post_compile, yarn.lock), so would either have to copy all of them in, or require the user to do so and keep the list in sync.

dmathieu commented

Ed is absolutely right.
When deploying your PHP app, we will execute the php buildpack, which will install all dependencies and make the app ready to be run (not as a docker container though. We use LXC).

Your local environment would need to mimick the execution of the buildpack, which would be tricky today.
This is definitely something on our mind though (not only for reproducing production environments locally, but also to make it easier to test buildpacks).

  • 👍2
fabswt commented

Thanks a lot, it's a lot clearer now.

There's gonna be a lot of variability of course, but I'll ask anyway: what do people usually go for in terms of local development workflow then? Would you recommend:

  • Using the heroku/heroku:16 Docker image and installing the dependencies I need.
  • Using a default Ubuntu Docker image and installing the dependencies I need.
  • To figure a way to execute the buildpack locally, maybe using LXC inside of Vagrant.
  • Taking it easy: merely installing in OS X the dependencies I need.

Sorry for the many questions. I was able to find a lot of doc in terms of deploying to Heroku, but not so much in terms of best practices for local development.

dmathieu commented

Some people run the buildpacks in the container (it's possible, the api is public).
Though I haven't seen many do so.

Even using docker will not be exactly the same thing as on our platform though. You will have the same versions, but we block sudo access and allow only storing files in /app at build time while docker allows storing files anywhere on the system.
Using LXC would bring you closer. But is it really worth the effort? You're gonna spend a lot of time setting up the environment.

For most apps, the abstraction level allows them to run the app locally, with the development environment provided by their machine.
Then, review apps allow testing in an environment much similar to the one there is in production.

The other production similarity you won't be able to achieve easily and which reinforces that idea of not trying to mimick production at 100% locally is that our router will proxy requests to your app, while with docker you will call it directly.

krainboltgreene commented

So I've gotten pretty far by creating a dockerfile that looks like this:


FROM heroku/heroku:16

ENV APPLICATION /app
ENV RAILS_ENV development
ENV RACK_ENV development
ENV PORT 3002
ENV STACK heroku-16

WORKDIR $APPLICATION

COPY Gemfile $APPLICATION/
COPY Gemfile.lock $APPLICATION/

RUN wget -q -O /heroku-buildpack-ruby-master.zip https://github.com/heroku/heroku-buildpack-ruby/archive/master.zip
RUN unzip -q /heroku-buildpack-ruby-master.zip -d /
RUN /heroku-buildpack-ruby-master/bin/detect $APPLICATION && /heroku-buildpack-ruby-master/bin/compile $APPLICATION/ /tmp

EXPOSE $PORT

CMD ["bin/rails", "server", "puma", "--binding=0.0.0.0"]

One difficulty I've encountered is that nokogiri wants gcc. I figured heroku-16 would come with gcc?

krainboltgreene commented

Once I did that it worked!

Now my blocker is that once I create a container around this build none of the gems exist (including bundler).

EDIT: This might have to do with some fuckery I tried with --deployment.

tt commented

If you take a look in $APPLICATION/.profile.d, you'll find a number of scripts created by the buildpack.

You'll likely have to source these to initialize all required environment variables.

krainboltgreene commented

So the only thing I have to figure out now is how to source right path changes.

Has anyone done this in an automated way? If not, I think all I need to do is create & COPY over a ~/.profile line, right?

krainboltgreene commented

Oh, and the other thing is that the current nodejs buildpack creates a $BUILD_DIR/.profile.d/nodejs.sh file that adds $HOME/node_modules/.bin to the path, but that directory doesn't exist since:

  1. heroku-16 supposes the directory will be /app not /root (which is the $HOME)
  2. The compile script can take a dynamic $BUILD_DIR as an argument.

What am I supposed to do about that?

ojacobson commented

It might be worth creating a user during docker build whose home is /app, and using that user. That's consistent with how Heroku dynos run - we don't run your app as root - so it will also help with reproducibility. Something like

RUN useradd --home-dir /app app
USER app

should do it. It may also be worth chowning /app to app: so that that user has permission to write to the dir.

  • 👍2
tt commented

I'm closing this issue due to inactivity and as it doesn't seem there's an action for us to take. Please reopen if I'm mistaken.

edmorley commented

If it helps for anyone else who comes across this issue looking for how to replicate heroku's dual-image pattern using Docker multi-stage builds... the following is what languages: - ruby produces when using the new heroku.yml build manifest developer preview:

FROM heroku/heroku:16-build as build
COPY . /app
WORKDIR /app
RUN mkdir -p /tmp/buildpack/ruby /tmp/build_cache /tmp/env
RUN curl https://buildpack-registry.s3.amazonaws.com/buildpacks/heroku/ruby.tgz | tar --warning=none -xz -C /tmp/buildpack/ruby
RUN STACK=heroku-16 /tmp/buildpack/ruby/bin/compile /app /tmp/build_cache /tmp/env

FROM heroku/heroku:16
COPY --from=build /app /app
ENV HOME /app
WORKDIR /app
RUN useradd -m heroku
USER heroku
CMD bundle exec puma -C config/puma.rb
  • 👍16
partounian commented

Anyone know how to get the above working with jruby? The download timeouts, but the link seems to work fine when I click it manually.

jcw- commented

I grabbed the latest multi-stage build steps for ruby like this:

Generate a rails app

rails new rails-hello-world -T -d postgresql
cd rails-hello-world
git init

Add heroku.yml

build:
  languages:
    - ruby
run:
  web: puma

Commit changes

Create heroku app and deploy (watch output for commands)

heroku create --manifest
heroku stack:set container
git push heroku master

Output

FROM heroku/heroku:18-build as build
COPY . /app
WORKDIR /app
RUN mkdir -p /tmp/buildpack/ruby /tmp/build_cache /tmp/env
RUN curl https://buildpack-registry.s3.amazonaws.com/buildpacks/heroku/ruby.tgz | tar --warning=none -xz -C /tmp/buildpack/ruby
RUN STACK=heroku-18 /tmp/buildpack/ruby/bin/compile /app /tmp/build_cache /tmp/env

FROM heroku/heroku:18
COPY --from=build /app /app
ENV HOME /app
WORKDIR /app
RUN useradd -m heroku
USER heroku
ENTRYPOINT ["/bin/bash", "-l", "-c"]
  • 👍1
partounian commented

As I'm finally becoming a decent software engineer, I'm able to contribute back to the community. I have made significant improvements to the performance of using Heroku's stacks locally.

Well almost. This is almost done, but I can't get the profile.d files to properly source.

First, make sure to set these in your shell config.

export DOCKER_BUILDKIT=1
export COMPOSE_DOCKER_CLI_BUILD=1

Dockerfile

# syntax = docker/dockerfile:1.0-experimental
# https://github.com/heroku/stack-images/issues/56#issuecomment-529177989
FROM heroku/heroku:20-build as build
RUN mkdir -p /tmp/buildpack/ruby /tmp/build_cache /tmp/env
ADD https://buildpack-registry.s3.amazonaws.com/buildpacks/heroku/ruby.tgz ruby.tgz
RUN tar --warning=none -xzf ruby.tgz -C /tmp/buildpack/ruby
ENV  BUNDLE_WITHOUT=""
ENV  BUNDLE_DEPLOYMENT=0 # this doesn't overwrite the current position :(
WORKDIR /app
COPY Gemfile .
COPY Gemfile.lock .
RUN --mount=type=cache,target=/tmp/build_cache STACK=heroku-20 /tmp/buildpack/ruby/bin/compile /app /tmp/build_cache /tmp/env

FROM heroku/heroku:20

ENV HOME /app
WORKDIR /app
RUN useradd -m heroku
USER heroku
COPY --from=build /app /app
COPY . /app

SHELL ["/bin/bash", "-l", "-c", "source /app/.profile.d/ruby.sh"]