Unable to configure for phoenix 1.3 app

[phtrivier]

Goal

I'm trying to use sqlite for a phoenix 1.3 app.
I only want the test configuration to run against an in-memory db.
The dev, and production configuration should still run against postgresql.

Context

I'm using :

• elixir 1.4.2
• phoenix 1.3.0-rc1
• ecto 2.1.4
• phoenix_ecto 3.2.3

My application (I called it myapp in the examples) is part of an umbrella.

Setup

Here is what I've done.

• Add sqlite_ecto2 as a dependency in mix.exs:

defp deps do
    [{:phoenix, "~> 1.3.0-rc"},
     {:phoenix_pubsub, "~> 1.0"},
     {:phoenix_ecto, "~> 3.0"},
     ....
     {:sqlite_ecto2, "~> 2.0.0-dev.0"}]
  end

• Add sqlite_ecto2 and ecto as applications in mix.exs:

    [mod: {MyApp, []},
     applications: [:phoenix,
                    :phoenix_pubsub,
                    :phoenix_html,
                    ...
                    :phoenix_ecto,
                    :postgrex,
                    ...
                    :sqlite_ecto2,  ## Added line
                    :ecto               ## Added line
                   ]]
  end

(Note that I still start application postgrex, I don't know if that could conflict.)

• Edit configuration from test.exs:

config :myapp, MyApp.Repo,
  adapter: Sqlite.Ecto2,
  database: "myapp_test"

• Keep Repo class as is:

defmodule MyApp.Repo do
  use Ecto.Repo,
    otp_app: :myapp
end

### Problem 1 : Compilation error

The project does not compile if I use Sqlite.Ecto2.

> mix test
I Compiling 1 file (.ex)
== Compilation error on file lib/myapp/repo.ex ==
** (ArgumentError) adapter Sqlite.Ecto2 was not compiled, ensure it is correct and it is included as a project dependency
    lib/ecto/repo/supervisor.ex:55: Ecto.Repo.Supervisor.compile_config/2
    lib/myapp/repo.ex:5: (module)
    (stdlib) erl_eval.erl:670: :erl_eval.do_apply/6

Is it normal ? Am I using a wrong version ?

However, using Sqlite.Ecto, I get things to compile, but then I run into another set of problem.

Problem 2 : SQL.Sandbox error

With test.exs:

config :myapp, MyApp.Repo,
  adapter: Sqlite.Ecto,
  database: "myapp_test"

Then I get:

> mix test
Compiling 1 file (.ex)
** (RuntimeError) cannot configure sandbox with pool DBConnection.Poolboy.
To use the SQL Sandbox, configure your repository pool as:

      pool: Ecto.Adapters.SQL.Sandbox

    (ecto) lib/ecto/adapters/sql/sandbox.ex:429: Ecto.Adapters.SQL.Sandbox.mode/2
    (elixir) lib/code.ex:370: Code.require_file/2
    (elixir) lib/enum.ex:645: Enum."-each/2-lists^foreach/1-0-"/2
    (elixir) lib/enum.ex:645: Enum.each/2
    (mix) lib/mix/tasks/test.ex:229: Mix.Tasks.Test.run/1

Is it required / impossible to use the sandbox ? Is it because of the mix test task ?

Problem 3 : Connection issues

With test.exs:

config :myapp, MyApp.Repo,
  adapter: Sqlite.Ecto,
  database: "myapp_test",
  pool: Ecto.Adapters.SQL.Sandbox

Then:

> mix test
** (CaseClauseError) no case clause matching: {:error, :invalid_message}
    lib/db_connection/ownership.ex:58: DBConnection.Ownership.ownership_checkout/2
    (ecto) lib/ecto/adapters/sql/sandbox.ex:477: Ecto.Adapters.SQL.Sandbox.checkout/2
    (ecto) lib/mix/tasks/ecto.migrate.ex:79: anonymous fn/4 in Mix.Tasks.Ecto.Migrate.run/2
    (elixir) lib/enum.ex:645: Enum."-each/2-lists^foreach/1-0-"/2
    (elixir) lib/enum.ex:645: Enum.each/2
    (mix) lib/mix/task.ex:294: Mix.Task.run_task/3
    (mix) lib/mix/task.ex:326: Mix.Task.run_alias/3
    (mix) lib/mix/task.ex:259: Mix.Task.run/2

Any idea what I'm missing ?

[phtrivier]

Just for info, as if this was not strange enough, it seems like:

• when running mix test from the umbrella's root, everything works fine
• when running mix test from the phoenix app itself, I get the errors from above.

I don't know if this could be because db_connection is not started the same way in both cases.

[scouten]

Hi Pierre, thanks for giving this a try. I'm at work today and can't really take time from my workday for this, but I hope to have time this evening or weekend to give it a deeper look.

[scouten]

Confirmed and fixed "Problem 1." Investigating the others …

[scouten]

One thing to be aware of: If you use the mix ecto.create and mix ecto.migrate commands without specifying MIX_ENV=test somewhere, the ecto.create and ecto.migrate commands will run in your dev environment. Ordinarily, this wouldn't matter, but if I understand your case, you have configured dev to use Postgres and test to use SQLite.

Given that, you will need to be explicit about using MIX_ENV=test when doing the DB setup; otherwise, you'll have no SQLite database in place to talk to, which probably explains Problem 2 above.

Note that the default Phoenix setup aliases mix test to include the ecto.create and ecto.migrate steps, so you'll need to be explicit on every mix test invocation, thus:

MIX_ENV=test mix test

You might want to revising your mix.exs file as proposed here: elixir-lang/elixir#5462 (comment) and corrected here: elixir-lang/elixir#5462 (comment)

[scouten]

I am working up a release now that will address one related issue that I uncovered while attempting to repro.

[scouten]

@phtrivier just released 2.0.0-dev.2 on hex.pm. Please give that a try and let me know what you find.

FYI I renamed the adapter to Sqlite.Ecto2 so you will need to update that in your config files.

Also, FWIW I will be out for the rest of today (US Pacific time). Should be able to follow up later tomorrow if there are new issues.

[scouten]

I set up a test repo at https://github.com/scouten/phx_sqlite_ecto2 for doing my tests; if you have any remaining issues, it would be helpful to note any important differences between my test repo and your environment. (A PR on that repo would be welcome if you want to influence how I test.)

[phtrivier]

@scouten Hi, this seems to work as I want in the test environment. However, as you mentioned, in dev environment, running mix test still uses the dev db at application startup, which breaks my solution.
I'll discuss that with my team, that leaves us with three options :

1. Expect everyone to start a postgres DB (docker works fine for that)
2. Expect everyone to run tests with MIX_ENV=test mix test
3. Use an sqlite DB in both dev and test envs, so that mix test always work.

Anyway, thanks for the help !

[scouten]

Closing this as resolved for now. Please feel free to reopen or file new issues if you have new issues.