google-cloud-aiplatform==1.36.4
redis==5.0.1
twilio==8.10.2
Install the Vertex AI SDK for Python with this link. Ensure that the Python version (Python 3.10 or 3.11 as of 11/19/23) for the virtual environment fits the python requirement for Vertex AI SDK. Then, set up the Application Default Credentials for your Google Cloud Services. Finally, ensure to do source ~/.bashrc
at the end.
Install Redis for Python with this link or using pip install redis
. Please refer to the Redis Documentation for more help.
Install and setup Twilio for Python using this link or using pip install twilio
.
Create library to use in other files to create the same Redis instance
Initiates conversation with Vertex AI and sends/receives messages from a Redis Database.
Initializes Google API Platform
Initializes Redis Database
Initializes a Vertex AI ChatModel using a pre-trained model named "chat-bison"
Retrieves context and input message from the Redis database
Starts a chat with the Vertex AI model using the provided context
Sends the input message to the model and receives a response
Tries to establish a connection to another Redis database to verify connectivity.
Stores the model's response back into the Redis database for the user
Use Vertex AI image text model to answer a series of questions about baby’s situation.
Initialize Google Cloud and Vertex AI Model
Load the input image
Generate the answers given the series of questions
Initialize the Redis database
Formulate and publish the answers as the context for
the Conversationer
Contains SMS interactions with Parent
Initializes Redis Database
Create a Twilio Client
Retrieve and publish the Parent's Message
for Conversationer
Retrieve Conversationer's Message and send to the Parent
Takes image from laptops' webcam
Open a connection to camera
Once every 10 second has passed since the last capture,
capture a new image
- Navigate to
first_demo
folder - Ensure that Google Application Default Credentials are set up
- Confirm that Twilio’s Account SID, Authentication Code, and Phone Numbers are correct
- Learn the virtual environment that you have created with the main dependencies
- Execute
capture_image.py
on a terminal - Execute
ngrok http 5000
on a different terminal and copy ngrok URL into Twilio Webhook⚠️ MacOS needs to turn off Airplay Receiver in Settings
- Execute Twilio Flask App
twilio_app.py
on another terminal, resulting in three different running terminals - Text to Your Designated Baby Face AI!
Our vision extends to integrating long-term video monitoring devices like cameras or specialized baby monitoring devices. We aim to refine our AI model, providing more tailored solutions and fostering more engaging conversations to better serve parents and their babies.