intellij-ui-test-robot: A Kotlin repository from nizienko

This library allows you to write and execute UI tests among IntelliJ Idea. You can test your plugin.

If you have any questions you are welcome to our Slack Channel

Quick Start

First we need to launch the IDE. Because the runIdeForUiTests task is blocking, we can run it as an asynchronous process:

./gradlew ui-test-example:clean ui-test-example:runIdeForUiTests &

Next, we can start the tests. Because they run locally, you must be sure the Welcome Frame is visible on the screen:

./gradlew ui-test-example:test

Or, just run all tasks together with one command:

./gradlew ui-test-example:clean ui-test-example:runIdeForUiTests & ./gradlew ui-test-example:test

Remote-Robot

The Remote-Robot library is inspired by Selenium WebDriver. It supports IntelliJ IDEA since version 2018.3.

It consists of a remote-robot client and a robot-server plugin:

remote-robot - is a client (test) side library used to send commands to the robot-server plugin.
robot-server - is an IDEA plugin that should run with the plugin you are developing.

The easiest way to start the test system is to execute the runIdeForUiTests task. (See Quick Start section above.) When IDEA is initialized, the robot-server plugin starts listening for commands from the UI test client.

The remote-robot library communicates with the robot-server plugin via HTTP protocol. This connection means you can launch IDEA on remote machines or in docker containers to check your plugin within different test environments.

Setup

The last version of the Remote-Robot is 0.11.16.

In the test project:

repositories {
    maven { url = "https://packages.jetbrains.team/maven/p/ij/intellij-dependencies" }
}
dependencies {
    testImplementation("com.intellij.remoterobot:remote-robot:REMOTE-ROBOT_VERSION")
}

In the plugin project:

runIdeForUiTests {
    systemProperty "robot-server.port", "8082" // default port 8580
}

downloadRobotServerPlugin {
    version = REMOTE-ROBOT_VERSION
}

By default, the port is local, so it could not be reached from another host. In case you need to make it public, you can add system property in the runIdeForUiTests task:

runIdeForUiTests {
    // ......
    systemProperty "robot-server.host.public", "true" // port is public
}

Of course, you can write UI tests in the plugin project.

Launching

There are two ways of launching Idea and UI tests:

Using Intellij gradle plugin

First we need to launch the IDE. Because the runIdeForUiTests task is blocking, we can run it as an asynchronous process:

./gradlew ui-test-example:clean ui-test-example:runIdeForUiTests &

Next, we can start the tests. Because they run locally, you must be sure the Welcome Frame is visible on the screen:

./gradlew ui-test-example:test

Or, just run all tasks together with one command:

./gradlew ui-test-example:clean ui-test-example:runIdeForUiTests & ./gradlew ui-test-example:test

Check this project as an example

Using ide-launcher

Using ide-launcher we can control Idea execution from the test. For that we have to add one more dependency in our project:

dependencies {
    testImplementation("com.intellij.remoterobot:ide-launcher:REMOTE-ROBOT_VERSION")
}

Next, we can use IdeLauncher to start Idea:

final OkHttpClient client=new OkHttpClient();
final IdeDownloader ideDownloader=new IdeDownloader(client);
     ideaProcess = IdeLauncher.INSTANCE.launchIde(
     ideDownloader.downloadAndExtract(Ide.IDEA_COMMUNITY, tmpDir),
     Map.of("robot-server.port",8082),
     List.of(),
     List.of(ideDownloader.downloadRobotPlugin(tmpDir), pathToOurPlugin),
     tmpDir
);

Check Java and Kotlin examples.

Useful launch properties

Property	Value	Description
jb.consents.confirmation.enabled	false	Disable consent dialog
eap.require.license	true	EAP version requires license the same way as Release version. That could help to avoid EAP login on CI
ide.mac.message.dialogs.as.sheets	false	Disable Sheet dialogs on Mac, they are not recognizable by Java Robot
ide.mac.file.chooser.native	false	Disable Mac native file chooser, it is not recognizable by Java Robot
jbScreenMenuBar.enabled + apple.laf.useScreenMenuBar	false + false	Disable Mac native menu, it is not recognizable by Java Robot
idea.trust.all.projects	true	Disable Thrust Project dialog when project is opened
ide.show.tips.on.startup.default.value	false	Disable Tips Of the Day dialog on startup

Create RemoteRobot

In the UI test project:

RemoteRobot remoteRobot = new RemoteRobot("http://127.0.0.1:8082");

Searching Components

We use the XPath query language to find components. Once IDEA with robot-server has started, you can open http://ROBOT-SERVER:PORT link. The page shows the IDEA UI components hierarchy in HTML format. You can find the component of interest and write an XPath to it, similar to Selenium WebDriver. There is also a simple XPath generator, which can help write and test your XPaths.

For example: Define a locator:

Locator loginToGitHubLocator = byXpath("//div[@class='MainButton' and @text='Log in to GitHub...']");

Find one component:

ComponentFixture loginToGitHub = remoteRobot.find(ComponentFixture.class,loginToGitHubLocator);

Find many components:

List<ContainterFixture> dialogs = remoteRobot.findAll(
    ComponentFixture.class,
    byXpath("//div[@class='MyDialog']")
);

Fixtures

Fixtures support the PageObject pattern. There are two basic fixtures:

ComponentFixture is the simplest representation of a real any component with basic methods;
ContainerFixture extends ComponentFixture and allows searching other components within it.

You can create your own fixtures:

@DefaultXpath(by = "FlatWelcomeFrame type", xpath = "//div[@class='FlatWelcomeFrame']")
@FixtureName(name = "Welcome Frame")
public class WelcomeFrameFixture extends ContainerFixture {
    public WelcomeFrameFixture(@NotNull RemoteRobot remoteRobot, @NotNull RemoteComponent remoteComponent) {
        super(remoteRobot, remoteComponent);
    }

    // Create New Project 
    public ComponentFixture createNewProjectLink() {
        return find(ComponentFixture.class, byXpath("//div[@text='Create New Project' and @class='ActionLink']"));
    }

    // Import Project
    public ComponentFixture importProjectLink() {
        return find(ComponentFixture.class, byXpath("//div[@text='Import Project' and @class='ActionLink']"));
    }
}

// find the custom fixture by its default XPath
WelcomeFrameFixture welcomeFrame=remoteRobot.find(WelcomeFrameFixture.class);
welcomeFrame.createNewProjectLink().click();

Remote-Fixtures

We have prepared some basic fixtures:

dependencies {
    testImplementation("com.intellij.remoterobot:remote-fixtures:REMOTE-ROBOT_VERSION")
}

The library contains Fixtures for most basic UI components. Please check this package to learn more. In case you want to add missing basic Fixtures you are welcome to PR or create an issue

Getting Data From a Real Component

We use the JavaScript rhino engine to work with components on the IDEA side.

For example, retrieving text from ActionLink component:

public class ActionLinkFixture extends ComponentFixture {
    public ActionLinkFixture(@NotNull RemoteRobot remoteRobot, @NotNull RemoteComponent remoteComponent) {
        super(remoteRobot, remoteComponent);
    }

    public String text() {
        return callJs("component.getText();");
    }
}

We can retrieve data using RemoteRobot with the callJs method. In this case, there is a robot var in the context of JavaScript execution. The robot is an instance of extending the org.assertj.swing.core.Robot class.

When you use the callJs() method of a fixture object, the component argument represents the actual UI component found (see Searching Components) and used to initialize the ComponentFixture.

The runJs method works the same way without any return value:

public void click() {
    runJs("const offset = component.getHeight()/2;"+
        "robot.click("+
        "component, "+
        "new Point(offset, offset), "+
        "MouseButton.LEFT_BUTTON, 1);"
    );
}

We import some packages to the context before the script is executed:

 java.awt
 org.assertj.swing.core
 org.assertj.swing.fixture

You can add other packages or classes with js methods:

importClass(java.io.File);
importPackage(java.io);

Or just use the full path:

Boolean isDumbMode=ideaFtame.callJs(
    "com.intellij.openapi.project.DumbService.isDumb(component.project);"
);

Store data between `runJs/callJs` requests

There are global and local Map<String, Object> variables available in the js context. global is a single map for the whole Ide. local map is defined on a per-fixture basis. Please check GlobalAndLocalMapExamples

In case you made robot-server-plugin port public, you may want to enable encryption for JavaScript code:

runIdeForUiTests {
    systemProperty "robot.encryption.enabled", "true"
    systemProperty "robot.encryption.password", "secret"
}

test {
    systemProperty "robot.encryption.password", "secret"
}

Text

Sometimes you may not want to dig through the whole component to determine which field contains the text you need to reach. If you need to check whether some text is present on the component, or you need to click on the text, you can use fixture methods:

welcomeFrame.findText("Create New Project").click();
assert(welcomeFrame.hasText(startsWith("Version 20")));
List<String> renderedText=welcomeFrame.findAllText()
    .stream()
    .map(RemoteText::getText)
    .collect(Collectors.toList());

Instead of looking for text inside the component structure, we render it on a fake Graphics to collect text data and its points.

Screenshots

There are two ways to get the screenshot.

Get shot of whole screen (method of RemoteRobot object)

remoteRobot.getScreenshot()

Get component screenshot (Method of Fixture object).
isPaintingMode parameter allows returning a new render of the component false by default).
It might be helpful when you don't have a complete set of desktop environments or when any other component covers the component of your interest.

someFixture.getScreenshot();
someFixture.getScreenshot(true);

In both cases, you will get BufferedImage object specified as .png

Kotlin

If you already familiar with Kotlin, please take a look at the kotlin example . You may find it easier to read and use.

Steps Logging

We use the step wrapper method to make test logs easy to read. The StepLogger example shows how useful it can be. For instance, by implementing your own StepProcessor, you can extend the steps workflow and connect to the allure report framework.

nizienko/intellij-ui-test-robot

Quick Start

Remote-Robot

Setup

Launching

Using Intellij gradle plugin

Using ide-launcher

Useful launch properties

Create RemoteRobot

Searching Components

Fixtures

Remote-Fixtures

Getting Data From a Real Component

Store data between `runJs/callJs` requests

Text

Screenshots

Kotlin

Steps Logging

FAQ

nizienko/intellij-ui-test-robot

Quick Start

Remote-Robot

Setup

Launching

Using Intellij gradle plugin

Using ide-launcher

Useful launch properties

Create RemoteRobot

Searching Components

Fixtures

Remote-Fixtures

Getting Data From a Real Component

Store data between runJs/callJs requests

Text

Screenshots

Kotlin

Steps Logging

FAQ

Store data between `runJs/callJs` requests