ChoHyoungSeo/posenet-react-native

This is a react native app with a posenet model running on the camera image. It currently outputs the predictions to the console. When running an app it will display the overall confidence in the model at the bottom of the screen.

PoseNet Camera example

PoseNet model in realtime, modified from with-tfjs-camera

Identify poses with @tensorflow/tfjs, expo-camera, and expo-gl (for native acceleration).

🚀 How to use

1. run this command:

npx create-react-native-app my-app -t with-tfjs-camera

2. change directory to my-app:

cd my-app

3. check that the with-tfjs-camera model works:

• Run expo start, open on a native device (simulator, emulator, and browser are not supported).

4. run this command:

npm i @tensorflow-models/posenet

5. clone this repository into a new and different folder:

git clone https://github.com/jude253/posenet-react-native

6. copy the files ModelView.js and PredictionList.js from posenet-react-native/src to my-app/src and overwrite the files in my-app/src

7. change directories to my-app:

cd my-app

8. check that the posenet model works:

• Run expo start, open on a native device (simulator, emulator, and browser are not supported).

9. stop server:

• hit control + c in terminal

10. convert expo managed project to expo bare workflow:

• in the directory my-app run command expo eject and enter in the expo-free app and package names

• to run the app now, use npm run web or some variant of npm run

• You can swap out @tensorflow-models/posenet for another TensorFlow model to achieve different results.

📝 Notes

• the number at the bottom of the camera UI is the confince the posenetmodel has in its predictions

• the prediction output of the model is logged to the console.

• the predictions are generated in ModelView.js

• PredictionList.js is where the predictions are turned into a UI element.

• TFJS Expo API reference

• @tensorflow/tfjs-react-native package

• Expo Camera docs