Howdy Howdy Lets get Rowdy!

Welcome to the NPCGPT github

Team: Eduardo Antonio Trevino, Saran Chowdari m, Christopher Placencio, Nicholas "Andres" Escobar

Here we are going to outline the entire project

The Project Managment Plan was done by: Eduardo Trevino

Preface of project Outline: Motivations for effort

The goal of this project was to extend the capabilities of Open AI's GPT family onto a fresh virtual environment interface. Virtual Reality / Mixed Reality technologies continue to offer humans the ability to expand their creativity while allowing for experiences that would otherwise not be possible. Therefore, extending GPT to access virtual interactions by humans is essential for its continued growth towards Artifical General Intelligence

Breakdown and reason for using Frameworks/Plugins/Technology:

For this Project I decided we should build it using OpenXR. OpenXR allows for future integration of XR (Virtual and Augmented Reality) improvements to upgrade the user interaction as the technology begins to diffuse.

For the Game Engine we decided on Unreal Engine 5.1. Unreal Engine has seen a lot of use in the Digital Twin (IOT) space, immersive simulation training and gaming, and for generating synthetic data for machine learning models/environments for reinforcement learning AI models. This presents an opportunity to learn a high demanding technology with near-infinite B2B, B2C market opportunities.

VoiceSDK

MetaHuman & MetaHumanSDK

Azure Cognitive services

Http REST API

STEP 1: EU5 Client (occulus quest 2)

STEP 2: VR Headset Mic Audio

STEP 3: Generate speech using human speech

Step 4: Azure Speech plugin: will be used for Azure speech to text and text to speech vice versa

Azure Cognitive Services is a suite of APIs and services that enable developers to build intelligent applications. In this project, the following Cognitive Services are utilized:

STT (Speech to Text): Converts spoken language into written text. TTS (Text to Speech): Generates natural-sounding speech from text.

Step 5: Http plugin: Send text to OpenAI GPT

Step 6: OpenAI GPT API generates text

Step 7: HTTP Gpt plugin receives text from GPT

Step 8: Azure Speech plugin: Azure text to speech

TAP SPEECH FROM BUTTOJN PRESS IS OPTIMAL

                          END OF PART 1

                          Engineering feat 1:

Microphone Audio -------> Convert to Text ---------> Print text to terminal (CONSCE)

                          END OF ENGINEERING FEAT 1
                                                   
                         
                         DESIGN:

NPC, GITHUB REPO DOCUMENTATION, VR WORLD/ LEVEL

                         Engineering feat 2: Research

ilkeroralkasim/Project-VR-NPCGPT