Fields

A collection of commonly used fields implemented as custom Ecto types with validation, sanitising and encryption/hashing.

Why? 🤷

We found ourselves repeating code for commonly used fields on each new Phoenix project/App.
We wanted a much easier/faster way of building apps so we created a collection of pre-defined fields with built-in validation, sanitising and security. Fields makes defining Ecto Schemas faster and more precise.

What? 💭

An Elixir package that helps you add popular custom types to your Phoenix/Ecto schemas so you can build apps faster!

@dwyl we are firm believers that personal data (Personally Identifiable Information (PII)) should be encrypted "at rest" i.e. all "user" data should be encrypted before being stored in the database. This project makes hashing, encryption and decryption for secure data storage much easier for everyone.

Who? 👥

This module is for people building Elixir/Phoenix apps who want to ship simpler more maintainable code.

We've attempted to make Fields as beginner-friendly as possible.
If you get stuck using it or anything is unclear, please ask for help!

How? ✅

Start using Fields in your Phoenix App today with these 3 easy steps:

1. Add the `fields` hex package to `deps` in `mix.exs` 📦

Add the fields package to your list of dependencies in your mix.exs file:

def deps do
  [
    {:fields, "~> 2.1.0"}
  ]
end

Once you have saved the mix.exs file, run mix deps.get in your terminal to download.

2. Ensure you have the necessary environment variables 🔑

In order to use Encryption and Hashing, you will need to have environment variables defined for ENCRYPTION_KEYS and SECRET_KEY_BASE respectively.

export ENCRYPTION_KEYS='nMdayQpR0aoasLaq1g94FLba=,L+ZVX8iheoqgqb22mUpATmMDsvVGtafoAeb='
export SECRET_KEY_BASE=GLH2S6EU0eZt+GSEmb5wEtonWO847hsQ9fck0APr4VgXEdp9EKfni2WO61z0DMOF

If you need to create a secure SECRET_KEY_BASE value, please see: How to create Phoenix secret_key_base
And for ENCRYPTION_KEYS, see: How to create encryption keys

In our case we use a .env file to manage our environment variables. See: github.com/dwyl/learn-environment-variables
This allows us to securely manage our secret keys in dev without the risk of accidentally publishing them on Github.
When we deploy our Apps, we use our service provider's built-in key management service to securely store Environment Variables. e.g: Environment Variables on Heroku

3. Apply the relevant field(s) to your schema 📝

Each field can be used in place of an Ecto type when defining your schema.

An example for defining a "user" schema using Fields:

schema "users" do
  field :first_name, Fields.Name            # Length validated and encrypted
  field :email, Fields.EmailEncrypted       # Validates email then encrypts
  field :address, Fields.AddressEncrypted   # Trims address string then encrypts
  field :postcode, Fields.PostcodeEncrypted # Validates postcode then encrypts
  field :password, Fields.Password          # Hash password with argon2

  timestamps()
end

Each field is defined as an Ecto type, with the relevant callbacks. So when you call Ecto.Changeset.cast/4 in your schema's changeset function, the field will be correctly validated. For example, calling cast on the :email field will ensure it is a valid format for an email address RFC 5322.

When you load one of the fields into your database, the corresponding dump/1 callback will be called, ensuring it is inserted into the database in the correct format. In the case of Fields.EmailEncrypted, it will encrypt the email address using a given encryption key before inserting it.

Likewise, when you load a field from the database, the load/1 callback will be called, giving you the data in the format you need. Fields.EmailEncrypted will be decrypted back to plaintext. This all happens 100% transparently to the developer. It's like magic. But the kind where you can actually understand how it works! (if you're curious, read the code)

Each Field optionally defines an input_type/0 function. This will return an atom representing the Phoenix.HTML.Form input type to use for the Field. For example: Fields.DescriptionPlaintextUnlimited.input_type returns :textarea which helps us render the correct field in a form.

The fields DescriptionPlaintextUnlimited and HtmlBody uses html_sanitize_ex to remove scripts and help keep your project safe. HtmlBody is able to display basic html elements whilst DescriptionPlaintextUnlimited displays text. Remember to use raw when rendering the content of your DescriptionPlaintextUnlimited and HtmlBody fields so that symbols such as & (ampersand) and Html are rendered correctly. e.g: <p><%= raw @product.description %></p>

Available `Fields` 📖

Address - an address for a physical location. Validated and stored as a (plaintext) String.
AddressEncrypted - an address for a customer or user which should be stored encrypted for data protection.
DescriptionPlaintextUnlimited - filters any HTML/JS to avoid security issues. Perfect for blog post comments.
Encrypted - a general purpose encrypted field. converts any type of data to_string and then encrypts it.
EmailEncrypted - validate and strongly encrypt email address to ensure they are kept private and secure.
EmailHash - when an email needs to be looked up fast without decrypting. Salted and hashed with :sha256.
EmailPlaintext - when an email address is public there's no advantage to encrypting it. e.g. a customer support email.
Hash - a general-purpose hash field using :sha256, useful if you need to store the hash of a value. (one way)
HtmlBody - useful for storing HTML data e.g in a CMS.
Name - used for personal names that need to be kept private/secure. Max length 35 characters. AES Encrypted.
Password - passwords hashed using argon2.
PhoneNumberEncrypted - a phone number that should be kept private gets validated and encrypted.
PhoneNumber - when a phone number is not sensitive information and can be stored in plaintext.
Postcode - validated postcode stored as plaintext.
PostcodeEncrypted - validated and encrypted.
Url - validate a URL and store as plaintext (not encrypted) String
UrlEncrypted - validate a URL and store as AES encrypted Binary

Detailed documentation available on HexDocs: hexdocs.pm/fields

Contributing ➕

If there is a field that you need in your app that is not already in the Fields package, please open an issue so we can add it! github.com/dwyl/fields/issues

Background / Further Reading 🔗

If you want an in-depth understanding of how automatic/transparent encryption/decryption works using Ecto Types,
see: github.com/dwyl/phoenix-ecto-encryption-example

If you are rusty/new on Binaries in Elixir, take a look at this post by @blackode:
https://medium.com/blackode/playing-with-elixir-binaries-strings-dd01a40039d5

towhans/fields