File upload works on Local adapter, but fails on S3 adapter because uploaded file is not there for some reason

[terenceponce]

Environment

• Elixir version (elixir -v): Elixir 1.11.4 (compiled with Erlang/OTP 23)
• Waffle version (mix deps): 1.1.4
• Waffle dependencies when applicable (mix deps):
  • waffle_ecto: 0.0.10
  • ex_aws: 2.1.7
  • ex_aws_s3: 2.1.0
  • hackney: 1.17.0
  • sweet_xml: 0.6.6
• Operating system: macOS 11.2.3

I managed to make this library work locally through Waffle.Storage.Local. However, when I switch to Waffle.Storage.S3, the upload fails because for some reason, it can't find the file anymore. This is the exact error that I'm getting:

[error] Task #PID<0.674.0> started from #PID<0.478.0> terminating
** (File.Error) could not stream "/var/folders/fb/h_1lnls96m71z30s770s92800000gn/T//plug-1618/multipart-1618053108-313083717869541-5": no such file or directory
    (elixir 1.11.4) lib/file/stream.ex:83: anonymous fn/3 in Enumerable.File.Stream.reduce/3
    (elixir 1.11.4) lib/stream.ex:1407: anonymous fn/5 in Stream.resource/3
    (elixir 1.11.4) lib/stream.ex:1609: Enumerable.Stream.do_each/4
    (elixir 1.11.4) lib/task/supervised.ex:330: Task.Supervised.stream_reduce/7
    (elixir 1.11.4) lib/enum.ex:3473: Enum.reverse/1
    (elixir 1.11.4) lib/enum.ex:3066: Enum.to_list/1
    (ex_aws_s3 2.1.0) lib/ex_aws/s3/upload.ex:99: ExAws.Operation.ExAws.S3.Upload.perform/2
    (waffle 1.1.4) lib/waffle/storage/s3.ex:188: Waffle.Storage.S3.do_put/2
    (waffle 1.1.4) lib/waffle/actions/store.ex:130: Waffle.Actions.Store.put_version/3
    (elixir 1.11.4) lib/task/supervised.ex:90: Task.Supervised.invoke_mfa/2
    (elixir 1.11.4) lib/task/supervised.ex:35: Task.Supervised.reply/5
    (stdlib 3.14.1) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
Function: #Function<1.20953253/0 in Waffle.Actions.Store.async_put_version/3>
    Args: []

Also, this is what the uploaded object looks like:

%Plug.Upload{
  content_type: "image/jpeg",
  filename: "0390a195446306b1bf26840cee2388dbd719ba57_239436_420_610.jpeg",
  path: "/var/folders/fb/h_1lnls96m71z30s770s92800000gn/T//plug-1618/multipart-1618053108-313083717869541-5"
}

What's weird is that even when I'm just using the Local adapter and the upload works, the supposed file/directory doesn't even appear on the specified location all throughout the process. The plug-1618 directory just remains empty.

Here is the config I'm using for Waffle:

config :ex_aws,
  json_codec: Jason,
  access_key_id: System.fetch_env!("DO_SPACE_KEY"),
  secret_access_key: System.fetch_env!("DO_SPACE_SECRET"),
  region: System.fetch_env!("DO_SPACE_REGION")

config :ex_aws, :s3,
  schema: "https://",
  host: System.fetch_env!("DO_SPACE_ENDPOINT"),
  region: System.fetch_env!("DO_SPACE_REGION")

config :waffle,
  storage: Waffle.Storage.S3,
  bucket: System.fetch_env!("DO_SPACE_NAME")

I'm using Digital Ocean Spaces instead of S3, but it is compatible with AWS, so there shouldn't be an issue with the connection to Digital Ocean. It's just that it can't find the file that was uploaded.

[terenceponce]

Looks like the temporary file just gets deleted immediately before it gets uploaded, so I had to copy the file before it gets deleted. I don't know why this happens when I'm using the S3 adapter, but my immediate problem is solved at the very list.

[tdgs]

I am having the same issue. The upload seems to work when the file is small, but fails if the file is slightly bigger.

[JoeriDijkstra]

Maybe you can check if the file is still being uploaded to the local entry by checking the progress key in the entry map in consume files