Sentiment Analysis

How to start

Download the dataset files stored at gs://sentiment-analysis-240222 with the following make target:

make pull

Install poetry to manage the dependencies and virtual environment, so install the packages by the following:

poetry install

Then, enter into the virtual environment shell:

poetry shell

Compute engine deployment

The following steps presents how to deploy a virtual machine as a development environment for this application. First, you must install Terraform and provide you Google Cloud credentials. Then, change the project name and region in the gcp/compute/variables.tf:

variable "project" {
  default = "weather-297102"
}

variable "region" {
  default = "us-central1"
}

variable "zone" {
  default = "us-central1-c"
}

So, create a bucket to be used to store the states generated by Terraform:

make tf-backend

Finally, init, plan and apply the Terraform resources as the following:

cd ./gcp/compute
terraform init
terraform plan
terraform apply

The external IP of the virtual machine created can be found by listing the compute instances of your project, as the following command:

gcloud compute instances list

To continue with the deployment, you must enter into the shell command line of the virtual machine:

gcloud compute ssh sentiment-analysis-instance

With the virtual machine up, you can clone this repository and run the scripts/install-dependencies.sh script. It will install jupyter lab and their dependencies. Also, a service is created to auto start the jupyter lab server at port 80:

git clone https://github.com/himewel/sentiment-analysis
cd sentiment-analysis
sudo bash ./scripts/install-dependencies.sh

Then, you can repeat the step at the How to start section and then start the application server:

sudo python3 src/application.py

Kubernetes engine deployment

Similar to the first steps at Compute Engine Deployment, you have to declare project, region and credentials to apply the Terraform modules. So, you will be able to apply the gcp/kubernetes modules and create a GKE cluster, a VPC network where the nodes will get an address, and a Container Registry to store the Docker image of the project, as the following:

cd ./gcp/kubernetes
terraform init
terraform plan
terraform apply

In a few minutes you can check if your cluster was created with:

gcloud container clusters list

Then, build and push a Docker image of the project to a repository visible to GKE. You can do this as the following:

# get the credentials to push the image
gcloud auth configure-docker
# so build and push the image
docker build . --tag gcr.io/weather-297102/sentiment-analysis
docker push gcr.io/weather-297102/sentiment-analysis

Now you can create the pods where your application will be deployed in the Kubernetes cluster with kubectl:

# get the credentials to manage you cluster
gcloud container clusters get-credentials sentiment-analysis-gke
# create the services declared in ./templates
kubectl apply -f templates

Finally, check the services to get the endpoint where the API will be provided:

kubectl get services