Tests for fake Google Cloud Storage
fsouza/fake-gcs-server is a Go library providing a fake Google Cloud Storage
(GCS) API server. This project demonstrates using the Docker image with the
Python library for Google Cloud Storage, with customizations in a
derived FakeClient
.
Running
To try it, run make test
. This will:
- Build a server container from the
test
branch ofjwhitlock/gc-fake-storage
- Build a client container from the
client
folder - Run a series of commands using a modified Google Cloud Storage client to:
- Create a bucket
- List the contents of all buckets
- Upload a text file
- Download the text file
- Delete the text file
- Upload any files in
client/files
- Shut down the client and server
To keep the server running after running tests, use make test-run
.
You can then view the uploaded files at
http://storage.gcs-server.127.0.0.1.nip.io:4443, using paths printed in the
test output. You'll need to confirm a security exception for a self-signed
certificate.
History
teone/gc-fake-storage first dockerized fake-gcs-server in March 2019, and published it at matteoscandolo/gc-fake-storage.
Starting in May 2019, I started a fork jwhitlock/gc-fake-storage, to
get the latest fake-gcs-server
code and make changes to get the Python
client working for more use cases. I integrated into the Tecken
development environment in July 2019 (PR #1830).
fsouza and teone merged the two projects in July 2019 (PR #47), and the official docker images were moved to fsouza's Dockerhub.
Notes
The Python Google Cloud Storage client is not written to use URLs other than the official GCS URLs. It reads several URLs from class and module variables, and doesn't provide a clean mechanism for changing them in derived classes or instances.
To change the API URLs to the fake GCS server, the class and module variables
are updated when initialing a FakeClient
. This would affect other users
of the Python GCS libraries. The FakeClient
can be initialized as a
context manager, which will undo the changes when exiting the context.
The client/files
folder can be used to test your own files for server
upload. A file smaller than 8 MB will be uploaded using the multipart upload
strategy. A larger file will be uploaded with the resumable upload API in
multiple requests.
The server/init_buckets
folder is mounted in the server container at
/data
. The folder is used to create buckets and items in those buckets.
The files are copied to the /storage
folder in the container.
The server/storage
folder contains the internal representation of
the fake storage. It is composed of buckets as folders, and object / blobs as
files with JSON content.
Deleting buckets is not (yet) supported by fsouza/fake-gcs-server
.