Friend: Open-Source AI Wearable with 24h+ on single charge

Homepage | Documentation

Friend is an AI wearable device that records everything you say, gives you proactive feedback and advice.

Features

Real-Time AI Audio Processing: Leverage powerful on-device AI capabilities for real-time audio analysis.
Low-powered Bluetooth: Capture audio for 24h+ on a small button battery
Open-Source Software: Access and contribute to the pin's software stack, designed with openness and community collaboration in mind.
Wearable Design: Experience unparalleled convenience with ergonomic and lightweight design, perfect for everyday wear.

How it works

graph TD;
   A[Device] -- Streams Audio --> B[Phone App];
   B -- Transmits --> C[Deepgram];
   C -- Returns Transcript --> D[Phone App];
   D -- Saves Transcript --> E[Phone Storage];

classDef lightMode fill:#FFFFFF, stroke:#333333, color:#333333;
classDef darkMode fill:#333333, stroke:#FFFFFF, color:#FFFFFF;

classDef lightModeLinks stroke:#333333;
classDef darkModeLinks stroke:#FFFFFF;

class A,B,C,D,E lightMode
class A,B,C,D,E darkMode

linkStyle 0 stroke:#FF4136, stroke-width:2px
linkStyle 1 stroke:#1ABC9C, stroke-width:2px
linkStyle 2 stroke:#FFCC00, stroke-width:2px
linkStyle 3 stroke:#2ECC40, stroke-width:2px

Structure

There are 3 different apps in these repositories located in different branches and folders. Our goal is to merge them into one big project.

Folder "AppStandalone": Standalone version of the app that doesn't require any hardware to use it.

Folder "AppWithWearable": Wearable-connected version of the app that requires the "Friend necklace" to use it.

Folder "AppWithWearableReactNative": Wearable-connected version of the app that is built in React native

Hardware Buying Guide (<$20 total)

Board: Seeed Studio XIAO nRF52840 Sense | $15 on Seedstudio and $24 on Amazon and also link for europe and India | <$15/unit
Rechargeable Battery on Amazon $7
Slider switch | $7 for 30+ units on Amazon and also link for europe but you only need one | <$1/unit
Wires. I didn't try these ones but they should work
3D-print design of the case is located here

Getting Started

Follow these steps to get started with your Friend.

Install the app

Clone the repo git clone https://github.com/BasedHardware/friend.git
Choose which version of the app you want to install (see Structure).
- Don't have the device? run cd apps/AppStandalone in terminal
- Have the device/NRF Board? run cd apps/AppWithWearable in terminal
Install Flutter and CocoaPods
Install your environment variables
- For AppWithWearable, open file ble_receive_w_a_v.dart located in apps/AppWithWearable/lib/custom_code/actions/ Find "DEEPGRAM_API_KEY" and provide your own api-key for Deepgram for transcriptions to work
then, go to apps/AppWithWearable/lib/custom_code/actions and in the "stream_api_response" file, add your openai key instead of ""
- For AppStandalone, update variables in in .env.template file
iOS: Install XCode then navigate to the iOS folder. You might need to launch Xcode to select a team and specify a bundle identifier. Android: Download/install android Studio then navigate to the Android folder Don't run in web/simulator: Bluetooth will not work
Run flutter clean then flutter pub get then pod install
When everything is installed, run flutter run , this should run your app on a selected device

No-Code Alternative:

Don't have the device? Clone this Flutterflow Project
Have the wearable device? Copy this Flutterflow Project

Install Firmware

Follow these steps to install the firmware:

Set up nRF Connect by following the tutorial in this video: https://youtu.be/EAJdOqsL9m8
In the nRF Connect Extension inside your VS Code, click "Open an existing application" and open the firmware folder from the root of this repo.
In the application panel of the extension, click the "Add Build Configuration" icon.
Choose the board as "xiao_ble_sense" and select the configuration as "prj.conf". Then, click "Build Configuration".
Once the build succeeds, you will find the zephyr.uf2 file in the firmware/build/zephyr directory.
Double-click on the reset button of the device. The device will appear on your computer as a disk. Drag and drop the zephyr.uf2 file into it.

Note: On a Mac, you might see an error message after dropping the file, indicating that the process did not complete. This is just a Mac-specific error; the firmware is successfully uploaded.

That's it! You have successfully installed the firmware on your device.

Testing Audio Recording on Your Computer

Follow these steps to test audio recording on your computer using a Python script:

Open your terminal and navigate to the project's root directory.
Change to the "test" folder:
```
cd test
```
Install the required Python modules:
```
pip install -r requirements.txt
```
Run the Python script:
```
python local_laptop_client.py
```

The recorded audio files will be stored periodically in the test/recordings directory.

That's it! You have now set up and tested audio recording on your computer.

Assemble the device

Step 0: Make sure you have bought everything from the buying guide above

Step 1: You need to design the case using 3D printer. Find .stl file here. If you don't know how to do it, send this file to someone who has a 3d printer

Step 2: Solder everything together like on the picture below. using a soldering kit. Don't have it? buy this one for $9

Step 3: Fit everything in the case. Biggest hole is for the usb port. In my example, I put the battery first, then the board and then the switch, however it's not an ideal design. If you will figure out a better solution, please contribute!

Step 4: Use hot glue to attach the lid to the case. You can also use a scotch tape first for testing purposes. Last, on the USB-port side, you'll find 2 small round holes. This is where the thread should go through.

Congratulations! you now have a fully working and assembled device!

Contributing

Join our Discord! We welcome contributions from the community! If you're interested in improving Friend, our current biggest goal is to combine both apps together (AppStandalone with AppWithWearable).

Standalone App brings great prompts and rich structure
AppWithWearable brings simple bluetooth connecting functionality

Support

For open-source support, please open an issue on GitHub and/or ask in our Discord Community. For commercial support, license inquiries, or any other questions, please contact us directly at team@whomane.com.

Disclaimer

Please note that the Friend is a prototype project and is provided "as is", without warranty of any kind. Use of the device should comply with all local laws and regulations concerning privacy and data protection.

Thank you for your interest in Friend, the open-source AI wearable. We're excited to see what you'll build with it!

Licensing

Friend is available under MIT License

coinspiracynut/Friend