nut.js (Native UI Toolkit)

	GitHub Actions
Master
Develop

Please visit

nutjs.dev

for detailed documentation and tutorials

About

nut.js is a cross-platform native UI automation / testing tool.

It allows for native UI interactions via keyboard and / or mouse, but additionally gives you the possibility to navigate the screen based on image matching.

Sponsoring

nut.js is developed with community in mind.

A huge "Thank you!" goes out to all sponsors who make open source a bit more sustainable!

Demo

Check out this demo video to get a first impression of what nut.js is capable of.

Tutorials

Please consult the project website at nutjs.dev for in-depth tutorials

Examples

nut-tree/trailmix contains a set of ready to use examples which demo the usage ot nut.js.

API Docs

nut.js provides public API documentation auto-generated by TypeDoc.

Community

Feel free to join our Discord community

Modules

This list gives an overview on currently implemented and planned functionality. It's work in progress and will undergo constant modification.

Clipboard

Copy text to clipboard
Paste text from clipboard

Keyboard

Support for standard US keyboard layout
Support for multimedia keys

Mouse

Support for mouse movement
Support for mouse scroll
Configurable movement speed
Mouse drag

Window

List all windows
Retrieve active window
Retrieve window title
Retrieve window size and position

Screen

Find an image on screen (requires an additional provider package like e.g. nut-tree/template-matcher)
Find all image occurrences on screen
Wait for an image to appear on screen (requires an additional provider package like e.g. nut-tree/template-matcher)
Retrieve RGBA color information on screen
Hooks to trigger actions based on images (requires an additional provider package like e.g. nut-tree/template-matcher)
Highlighting screen regions

Integration

Jest
Electron

Sample

The following snippet shows a valid nut.js example:

"use strict";

const {
  mouse,
  left,
  right,
  up,
  down,
  straightTo,
  centerOf,
  Region,
} = require("@nut-tree/nut-js");

const square = async () => {
  await mouse.move(right(500));
  await mouse.move(down(500));
  await mouse.move(left(500));
  await mouse.move(up(500));
};

(async () => {
  await square();
  await mouse.move(straightTo(centerOf(new Region(100, 100, 200, 300))));
})();

Installation

Prerequisites

This section lists runtime requirements for nut.js on the respective target platform.

Windows

In case you're running Windows 10 N and want to use ImageFinder plugins, please make sure to have the Media Feature Pack installed.

macOS

On macOS, Xcode command line tools are required. You can install them by running

xcode-select --install

Permissions:

nut.js requires the executing application, e.g. your terminal, to be given both Accessibility and Screen Recording permissions.

Starting with release 2.3.0, nut.js will check for and request these permissions automatically:

It will also give you a subtle hint in case permissions are lacking:

Accessibility: ##### WARNING! The application running this script is not a trusted process! Please visit https://github.com/nut-tree/nut.js#macos #####
Screen Recording: ##### WARNING! The application running this script is not allowed to capture screen content! Please visit https://github.com/nut-tree/nut.js#macos #####

Attention:

Prior to release 2.3.0 you'll have to grant these permissions manually.

Settings -> Security & Privacy -> Privacy tab -> Accessibility -> Add...

For example, if you want to execute your node script in e.g. iTerm2, you'd have to add iTerm.app to the list. When running your script from a built-in terminal in e.g. VSCode or IntelliJ, you'd have to add the respective IDE.

Linux

Depending on your distribution, Linux setups may differ.

In general, nut.js requires

libXtst

Installation on *buntu distributions:

sudo apt-get install libxtst-dev

Setups on other distributions might differ.

Install `nut.js`

Running

npm i @nut-tree/nut-js

yarn add @nut-tree/nut-js

will install nut.js and its required dependencies.

Snapshot releases

nut.js also provides snapshot releases which allows to test upcoming features.