hamzahjamad/testmybot

Automated Testing for Chatbots

Test My Bot

TestMyBot is a test automation library for chatbots. Test cases are recorded by Capture & Replay tools and can be run against your chatbot implementation automatically over and over again. It is meant to be included in your continuous integration pipeline, just as with your unit tests.

Documentation

Please navigate to our Wiki for documentation.

Samples

There are a couple of samples available, showing possible scenarios to use TestMyBot. For all samples, you these commands to run them:

> npm install
> npm run test

To run the console emulator (text input and output), run this command in the sample directory:

> npm install
> npm run emulator

In case you want to see some more output, enable debug logging:

> export DEBUG=testmybot*,botium*

Loading ready-to-use Chatbot Test Cases and Utterances from "Botium Utterances" library

Link

• Demonstrates usage of the free utterances and chatbot test cases library "Botium Utterances"
• Loads the typical "Tell me a Joke" test suite and runs it agains a Dialogflow chatbot

Chatbot developed with Microsoft Bot Framework

Link

• Uses one of the sample chatbots included in the Bot Framework Samples Repository
• Loads chatbot code from Github automatically
• Uses "docker" mode to place chatbot in docker container

Chatbot developed with Botkit

Link

• Wires Botkit with TestMyBot

Loading a Facebook chatbot from Git

Link

• Uses a calculator sample
• Loads chatbot code from Github automatically
• Uses "docker" mode to place chatbot in docker container

(no test cases contained in this sample)

Chatbot developed in Clojure

Link

• Shows a chatbot developed in another programming language than TestMyBot
• Uses a chatbot developed with Clojure from the Lemmings incubator
• Downloads a docker image ("clojure.lein") and placing the chatbot in a docker container

{
  "botium": {
    "Capabilities": {
      "PROJECTNAME": "testmybot-sample-clojure",
      "FACEBOOK_API": true,
      "FACEBOOK_WEBHOOK_PORT": 3000,
      "FACEBOOK_WEBHOOK_PATH": "webhook",
      "CLEANUPTEMPDIR": false,
	    "DOCKERIMAGE": "clojure:lein",
      "STARTCMD": "lein ring server-headless"
    },
    "Sources": {
      "GITURL": "https://github.com/lemmings-io/02-facebook-example.git"
    }
  }
}

Sample Facebook Chatbot (Node.js)

Link

• Shows how to include TestMyBot into your build pipeline for continuous testing
• docker container

Connecting to Chatbot running in Facebook Messenger

Link

• Can be used to automate testing of a chatbot already running in facebook messenger
• Usefull for regression testing, not for continous testing (takes to much time to run)
• See sample directory for instructions

Using Mocha

Link

• Mocha can be used instead of Jasmine as well
• docker container

Chatbot developed in PHP

Link

• Chatbot developed in PHP
• See sample directory for instructions

Slack Chatbot

Link

• Sample how to test a Slack chatbot

IBM Watson Chatbot

Link

• Connecting to Watson conversation service to run your convos
• Useful for regression testing
• Workspace can be copied automatically before running the test cases (to not pollute your workspace)

Voice recognition and speech synthesis

Link

• Uses the Web speech api to run a conversation with Alexa
• Requires a real browser supporting the Web Speech API

Custom REST/HTTP interface and using Excel test cases

Link

• Contacting a custom REST/HTTP interface, as usual for chatbots running on websites
• Test cases are extracted from an Excel sheet instead of text files