This project is a PhD research demonstrates a proof-of-concept for using raw touchpad input data to control 3D navigation within a WebView2-embedded web page. It leverages C# (WPF) to capture touchpad data, communicate with the embedded web page, and visualize touch points in real time. The original RawInput.Touchpad source code by emoacht and Extended from my research banyapon.
- Enhanced Multitouch Support: Builds upon the RawInput.Touchpad foundation to enable more robust multitouch interactions.
- Research-Oriented: Designed for academic exploration and experimentation with multitouch technologies.
- Windows Compatibility: Specifically tailored for the Windows operating system.
- Raw Touchpad Input: Captures detailed touch data from precision touchpads (including contact ID, X/Y coordinates).
- C#-JavaScript Communication: Utilizes WebView2's PostWebMessageAsString to send touchpad data to the web page.
- Real-time Visualization: Renders touch points as circles on an HTML canvas within the WebView2 control.
- MainWindow.xaml: Defines the WPF window layout, including the WebView2 control.
- MainWindow.xaml.cs: Contains the C# code for handling touchpad input, communicating with the web page, and visualizing the data.
- index.html: The embedded HTML page with the JavaScript code to display the touch points on a canvas.
- TouchpadHelper.cs: (Assumed) Helper class for touchpad interaction (not included in this example).
- Touchpad Data Capture: The C# code uses the Raw Input API (through TouchpadHelper) to receive raw touchpad input events.
- Data Formatting: The captured data is formatted into a string representing contact points (e.g., "Contact ID:0 Point:879,507").
- Communication: The formatted data is sent to the index.html page via PostWebMessageAsString.
- Web Page Visualization: The JavaScript code in index.html parses the string, extracts the touch point coordinates, and dynamically draws circles on the canvas to represent the touch positions.
Before using this project, it's crucial to disable the default Windows touchpad gestures in your system settings. This ensures accurate and unhindered multitouch data collection.
Here's a visualization of 5 simultaneous touch points being recognized and processed by the project:
Prerequisites: Visual Studio: Ensure you have Visual Studio installed with the necessary WPF development tools and the WebView2 SDK. Microsoft Edge (Chromium): WebView2 relies on the Microsoft Edge (Chromium) runtime. Make sure it's installed and up-to-date. Clone the Repository:
git clone https://github.com/banyapon/RawInputWebViewJSONTouch.git- Open and Build: Open the solution in Visual Studio and build the project.
- Run: Run the application. You should see a window with an embedded web page. Interacting with your touchpad will display touch points as circles on the canvas.
- 3D Navigation Integration: Implement actual 3D navigation controls based on the touchpad input.
- Gesture Recognition: Incorporate machine learning or gesture recognition algorithms to interpret more complex touchpad interactions.
- Customization: Allow for customization of the visualization (e.g., color, size of the circles) and the types of touchpad events to capture.
This project serves as a foundation for exploring novel touchpad-based 3D navigation techniques. Future research could focus on:
- Intuitive Gesture Design: Designing user-friendly gestures for 3D manipulation.
- Machine Learning Models: Training models to recognize complex gestures and map them to 3D actions.
- User Studies: Evaluating the effectiveness and usability of the new gestures.
Contributions are welcome! Feel free to open issues, submit pull requests, or share your ideas for extending this project. Let me know if you'd like any adjustments or further details in this README file!
This project is part of the ongoing research conducted at the School of Information Technology, King Mongkut's Institute of Technology Ladkrabang.
This project is licensed under the MIT License.