/ESP32MouseTo

Arduino ESP32 S3 library for moving mouse pointer to absolute screen coordinates

Primary LanguageC++MIT LicenseMIT

ESP32MouseTo

Library for Arduino ESP32 S3 for moving the mouse pointer to absolute screen coordinates.

Table of contents

Installation

  • Download the most recent version of ESP32MouseTo here: https://github.com/Youngv/ESP32MouseTo/archive/master.zip
  • Using Arduino IDE 1.0.x:
    • Sketch > Import Library... > Add Library... > select the downloaded file > Open
  • Using Arduino IDE 1.5+:
    • Sketch > Include Library > Add ZIP Library... > select the downloaded file > Open
  • Running the example sketches:
    • File > Examples > ESP32MouseTo
    • Upload to device

Usage

See the example sketches at File > Examples > ESP32MouseTo for demonstration of library usage.

#include <ESP32MouseTo.h>

Allow access to the functions of the ESP32MouseTo library.

ESP32MouseTo.setTarget(targetX, targetY[, homeFirst])

Move mouse pointer to absolute screen coordinates. Note that screen coordinates start from the upper left corner.

  • Parameter: targetX - X screen coordinate to move to.
    • Type: int
  • Parameter: targetY - Y screen coordinate to move to.
    • Type: int
  • Parameter(optional): homeFirst - Whether to home the mouse pointer before moving to the target. At a minimum the pointer must be homed before the first use of ESP32MouseTo in your code. If homing is not done after that time the accuracy of the mouse pointer movements might suffer, for example if the regular mouse connected to the computer is moved. The default value is true.
    • Type: boolean
  • Returns: none
ESP32MouseTo.getTargetX()

Get the last X coordinate set by ESP32MouseTo.setTarget().

  • Returns: Target X coordinate.
    • Type: int
ESP32MouseTo.getTargetY()

Get the last Y coordinate set by ESP32MouseTo.setTarget().

  • Returns: Target Y coordinate.
    • Type: int
ESP32MouseTo.move()

Move mouse pointer to the target coordinates. Multiple calls to ESP32MouseTo.move() may be required before the target coordinates are reached.

  • Returns: true = target coordinates reached, false = target coordinates not yet reached
    • Type: boolean
ESP32MouseTo.setScreenResolution(x, y)

When ESP32MouseTo.move() homes it assumes that the mouse pointer is at the lower right corner. The default values are set to home correctly even with very high resolution(4K UHD). By setting the ESP32MouseTo screen resolution value to your actual resolution you can reduce the time ESP32MouseTo.move() takes to reach the target coordinates.

  • Parameter: x - X screen resolution. The default value is 3840.
    • Type: int
  • Parameter: y - Y screen coordinate to move to. The default value is 2160.
    • Type: int
  • Returns: none
ESP32MouseTo.getScreenResolutionX()
  • Returns: X screen resolution.
    • Type: int
ESP32MouseTo.getScreenResolutionY()
  • Returns: Y screen resolution.
    • Type: int
ESP32MouseTo.setCorrectionFactor(correctionFactorInput)

The screen distance moved is different from the distance arguments passed to Mouse.move(). In order for ESP32MouseTo.move() coordinates to correspond with the screen coordinates you must set a correction factor. The correct value can be determined with File > Examples > ESP32MouseTo > BasicUsage.

  • Parameter: correctionFactorInput - The default value is 1(no correction).
    • Type: float
  • Returns: none
ESP32MouseTo.getCorrectionFactor()
  • Returns: Correction factor.
    • Type: float
ESP32MouseTo.setMaxJump(maxJumpDistanceInput)

Mouse.move() only allows a maximum move of 127 pixels in each axis so longer mouse pointer movements require multiple calls. Smaller values will require more calls to ESP32MouseTo.move() before the target coordinates are reached, larger values may lead to less accuracy. You must recalculate the correction factor whenever this is changed.

  • Parameter: maxJumpDistanceInput - The maximum distance(x and y) the mouse pointer can move on each call to ESP32MouseTo.move(). Allowable values are 1-127. The default value is 10.
    • Type: int8_t
  • Returns: none
ESP32MouseTo.getMaxJump()
  • Returns: Maximum jump distance.
    • Type: int8_t
ESP32MouseTo.home()

Re-home the mouse before continuing to the target coordinates. This is useful for cases when ESP32MouseTo.move() completion was delayed long enough that the mouse pointer may have been moved.

  • Returns: none

ESP32MouseTo.move() Process

  • Each call of ESP32MouseTo.move() moves the pointer up to the maximum jump distance(set by ESP32MouseTo.setMaxJump()) in the x and y axes.
  • Home mouse pointer to (0, 0).
  • Once the mouse pointer is homed start moving to the target coordinates.
  • Once the target coordinates are reached return true.

Mouse coordinate display software

For calibration and usage, it is helpful to have a program that displays the current mouse pointer coordinates. Here are some options:

  • MousePosition.html (any OS): This file is included with the library in the extras folder. Open it with your browser and the mouse coordinates will be displayed there. It will work on any OS. It only works when the mouse pointer is over the browser window. You can set the browser to full screen, move the mouse to the desired position in another window, then use the keyboard to switch back to the browser window (e.g. Alt+Tab) to check the coordinates.
  • MousePointer console application (Windows OS): This program displays the coordinates even when the mouse pointer is not over the application window.
  • watch -t -n 0.01 xdotool getmouselocation (Linux or macOS) - Install xdotool and then run this command from the terminal. The decimal point in 0.01 is locale-dependent so you may need to change it to 0,01.
  • Android developer tools (Android) - Android has a built-in option to display the pointer coordinates. When using this for calibration, note that it only displays the coordinates while a mouse button is pressed, so you will need to use Mouse.press() in your code for this purpose.
    • Settings > Developer options
    • Set Developer options to on.
    • Set Pointer location to on.

Alternatives

Contributing

Pull requests or issue reports are welcome! Please see the contribution rules for instructions.