/Style-AI

Implementing the Neural Style Transfer algorithm for creating artistic images and integrating it as a web service with a Flask app.

Primary LanguageJavaScript

Style-AI

StyleAI is a Flask web app, which integrates the Neural Style Transfer algorithm for generating artistic images. It allows the user to import custom style and content images used for generation and to set the initial model parameters before starting the model. After the generation is complete, the result image is saved to the user's gallery, where the image can be viewed along with the model performance statistics (content loss, style loss, etc..) and also deleted.

Image generation

Learning curves

Gallery

Main features

  • User registration and an OAuth2 authentication service through Google, Facebook and Github using the Rauth library.
  • Reset password feature using Flask-Mail as an asynchronous Celery task for seding emails in the background.
  • SQLite database for storing image-related data and creating the user gallery.
  • Tensorflow Keras-based Deep Learning model for Neural Style Transfer created on Google Colab

Open In Colab

Project structure

Project structure

Requirements

  • An NVIDIA GPU with the CUDA and CuDNN libraries installed (preferably GTX 8th gen and above) to run tensorflow's GPU version (if you don't have an NVIDIA GPU, you can switch to the CPU version by replacing tensorflow-gpu==1.13.1 with tensorflow==1.13.1 in the requirements.txt file.)
  • Python v3.5 or above, preferably in the Anaconda environment.
  • Ngrok server for creating a tunneled session, in order to run the app locally in a real domain. This is required for the OAuth authentication services, because they don't accept localhost.
  • Redis server for running background Celery workers for the email service.

Running the app

1. Clone the project from this repository to your local directory using:

git clone https://github.com/JadeBlue96/Style-AI.git 

2. If you have Anaconda installed, it would be a good idea to create a separate environment for the project, for example:

conda create -n tf-style python=3.5
conda activate tf-style

3. Install the required libraries by navigating to the st_webservice folder and using pip:

cd st_webservice 
pip install –r requirements.txt

4. Run the app in Visual Studio 2017 or from the console by setting the Flask environment variables:

set FLASK_APP=runserver.py
set FLASK_ENV=development / testing
flask run

5. Run the unit tests with the following command:

flask test --coverage

6. To reset the database, delete the db file and the migrations folder and enter:

flask db init
flask db migrate -m "custom message"

7. Testing the authentication services:

  • Run the Ngrok server on port 5000:
ngrok.exe http 5000
  • Change the URLs in layout.html to the generated one in the Forwarding section for HTTPS, for example:
$('.facebook-btn').click(function () {window.location = "https://1e498486.ngrok.io/authorize/facebook";}); 
$('.google-btn').click(function () {window.location = "https://1e498486.ngrok.io/authorize/google";}); 
$('.github-btn').click(function () {window.location = "https://1e498486.ngrok.io/authorize/github";});
  • Create a developer's app for the service you want to test:

-- For Facebook: add the Facebook Login extension to your app and set the following URLs in Valid OAuth Redirect URIs: <your-generated-url>, <your-generated-url>/callback/facebook, <your-generated-url>/authorize/facebook

-- For Github: go to your app in Settings/Developer Settings/OAuth Apps and change the homepage URL to your generated base url and the authorization callback URL to: <your-generated-url>/callback/github

-- For Google: go to Credentials/OAuth Consent Screen/Authorized Domains and add your generated base url; then in the app's settings in Authorized Redirect URIs add your base url and <your-generated-url>/callback/google

8. Testing the reset password feature:

  • Start the Redis server
  • Create a pool of Celery workers from a separate console using:

celery –A st_webservice worker –-pool=eventlet –l info

  • Set the environment variables for the mail server (example for Gmail):
set MAIL_SERVER=smtp.googlemail.com 
set MAIL_PORT=587 
set MAIL_USE_TLS=1 
set MAIL_USERNAME=<client username> 
set MAIL_PASSWORD=<client password>

9. (Update) Create your own free Amazon S3 storage bucket for serving the images to the cloud. After that, edit your .env file and/or config.py and set the environment variables concerning the credentials and the bucket url (example):

.env

AWS_ACCESS_KEY_ID=<your amazon access key id>
AWS_SECRET_ACCESS_KEY=<your amazon secret access key>
config.py

AWS_ACCESS_KEY_ID = os.environ.get('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.environ.get('AWS_SECRET_ACCESS_KEY')
FLASKS3_BUCKET_NAME = <your bucket name>
S3_LOCATION = 'https://{}.s3.amazonaws.com/'.format(FLASKS3_BUCKET_NAME)
S3_OBJECT_URL = <your object url> (example: 'https://styleai.s3.us-east-2.amazonaws.com/')

Update: Heroku Deployment

As of this point, the app has now been successfully deployed to Heroku in the cloud and is available on https://style-ai.herokuapp.com/.

However, due to the fact that I am using free tier dynos, bear in mind that it takes about 1 minute to load the server and the working pool. Heroku also does not provide GPU support and has limited CPU memory, so the model will inevitably run significantly slower in the cloud.

With that said, here is how to configure this app to run locally through Heroku with GPU support:

1. Create a .env file in the main directory with your environment variables:

.env

FLASK_APP=runserver.py
FLASK_CONFIG=heroku
MAIL_USERNAME=<your mail username>
MAIL_PASSWORD=<your mail password>
REDIS_URL=redis://
AWS_ACCESS_KEY_ID=<your amazon access key id>
AWS_SECRET_ACCESS_KEY=<your amazon secret access key>

2. Make sure that config.py has the url for the local Redis server specified:

CELERY_BROKER_URL = 'redis://localhost:6379/0' #os.environ['REDIS_URL'] for cloud version
CELERY_RESULT_BACKEND = 'redis://localhost:6379/0' #os.environ['REDIS_URL']

3. Make sure that the Procfile has the local port set for the waitress server:

web: waitress-serve --port=5000 runserver:app 
worker: celery -A st_webservice worker --pool=eventlet -l info

4. Start your Redis server and initialize the heroku server using:

heroku local:run flask deploy
heroku local

If you want to test the cloud deployment version make sure you switch to the deployment branch. The correct environment variables should be specified there. After making a change to the cloud version, commit your changes and push them to Heroku using:

git push heroku deployment:master

Afterwards, you might want to restart the working dynos or run an upgrade to the database depending on your changes:

heroku run flask deploy
heroku restart

Acknowledgements

Big thanks to Miguel Grinberg for his Flask Mega-Tutorial series, as well as his Flask Web Development book.