/navlead

The client-side of Navlead. Navlead is an deep-learning assistant chatbot for navigation in 3D enviroments.

Primary LanguageTypeScriptMIT LicenseMIT

Navlead

AI-powered assistant chatbot for navigation in 3D enviroments.

The chatbot was trained on real-world navigation assistance dialogs to give natural responses.

The 3D enviroments were reconstructed form real-world houses.

Navlead simulation iPhone mockup Navlead simulation iPad mockup Navlead simulation iPhone mockup

Navlead gif

Overview of NavLead

Navlead is an AI-powered assistant chatbot for navigation in 3D enviroments. The simulation, the chatbot, and their deployment online are all part of this project. (Note: this repo contains the client-side section only)

The simulation has custom 3D assets, a chat dialog box, motion controls and game logic. The chatbot has a custom ML NLP model that has a seq2seq architecture and was trained on CVDN dataset. The client-host loads the simulation and mediates its interactions with the API server. The API server authenticates the request and runs the ML NLP model.

Navlead system architecture diagram

Motivation

Inspired by the rise of VR and AI, this project aims to combine both into a navigation assistance system that can run through the browser.

Navlead questin sample Navlead target sample Navlead location sample

NavleadNet main diagram

Navlead answer sample

Tech stack of Navlead

  • Client-side
    • Simulation
      • Blender (3.1)
      • Unity (2020.3 LTS)
    • UI
      • React (Next ts)
  • Server-side (not on this repo)
    • API server
      • Django
    • ML service
      • Pytorch

In-depth explanation

For an in-depth explanation of this project, please, see Navlead's About page

Navlead's about page image quilt

Live website

See Navlead.

Project setup

npm install

Compiles and hot-reloads

npm run dev

Compiles and minifies for production

npm run build

Lints and fixes files

npm run lint

Acknowledgements

Vision-and-Dialog Navigation

https://arxiv.org/abs/1907.04957
https://github.com/mmurray/cvdn