- Glossary
- Introduction
- Quick Start
- Installation
- Configuration
- API
- Mocking responses from the bot
- Examples
- ChangeLog
- script or conversation spec - array of objects. Script is a step-by-step specification for conversation between a human and an application;
- filter function - a custom function that will be called by Library. Used to inject async data, the function should return a Promise;
- MBF - Microsoft Bot Framework, Node.JS version;
The Library still in an active development, so don't hesitate to propose changes and new features! Backward compatibility of API not guaranteed before 1.0 release.
This is a test framework for chatbots developed with Microsoft Bot Framework for Node.JS. Supports both unit and functional tests. In background of MBF
Emulates a conversation between user and bot. Provides input for bot and validates response. Each test requires a script - array of steps, where every step represents of next entities:
- Validator for Bot response;
- An Input for bot;
- Conversation State validator or modifier;
- Custom action, with logic injected by test developer;;
Test fails if conversation deviates from specified in script.
- before and after attributes are not supported anymore, use custom steps instead;
- Bot instance removed from filter functions arguments, as it is useless, as it never used;
I apologize for the inconvenience.
-
Support bots (for functional tests) and dialogs (unit testing);
-
Built-in validators:
- text : message text. Could be an exact phrase or regular expression by equality or by regular expression;
- prompts;
- conversation endings and typing indicators;
- validation of suggested actions;
- attachment and richcard validation;
-
session state management;
-
active dialog and default params management;
-
timeout validation;
-
custom validation login as a step of the script
-
configurable reporting;
-
mocking of responses from a bot, use it if you need only to prototype conversation;
npm install --save-dev botbuilder-unit
File "test-script.js":
const unit = require('botbuilder-unit');
const builder = require('botbuilder');
// This array, also called a script. It will be used to validate conversation with user
let script = [
{
"user": "hi"
},
{
"bot": "How should I call you?"
},
{
"user": "Timmy"
},
{
"bot": "Nice to meet you, \"Timmy\"!"
}
];
// Setting up a bot
bot = new builder.UniversalBot(connector);
bot.dialog('/', [
session => builder.Prompts.text(session, 'How should I call you?'),
(session, response) => session.endDialog(`Nice to meet you, ${JSON.stringify(response.response)}!`)
]);
// Executing test
unit(bot, script, {
title: 'Your first test script',
reporter : new unit.BeautyLogReporter() // Display log in messenger-like style, with colors
}).then(() => {
// If test finished successfully
console.log('Script passed');
}, (err) => {
console.error(err);
})node ./test-script.js
At the end you will see next result:
npm install --save-dev botbuilder-unit
As already mentioned, all steps are divided into groups of activities, each step is key-value object. Each activity defines a set of unique keys. So, the Library differentiates steps key used in a step configuration.
Each key represents a concrete action. Some keys could used together, in one step. If key name marked as standalone, that means the key always alone in a step. It is not possible to mix key of different activities.
Full list of supported step attributes:
-
Bot Response Validation. :
- bot:(string|filter) - message text, received from bot;
- suggestedActions: array - validates suggestedActions of received message ;
- attachmentLayout: string - used to identify layout, a way how attachments ordered and displayed. Usually one of list or carousel.
- attachments: array - each item represents an attachment, meta-data and content;
- standalone *endConversation: null, use it to specify that conversation will be finished;
- standalone typing: null, use it, if you want to validate that typing indicator sent;
-
Input from User:
- user: (string|filter) text message or instance of Botbuilder.Message
-
Session Management:
- session (object|filter) - used to set value of session or to validate it (see examples below);
-
Dialog Management:
-
dialog, sets current dialog for bot instance, could be used together with args option
-
args, set default arguments for bot instance, could be used together with dialog option
If the message is from the user, than message object should look like this:
{
"user" : "Hey there!"
}Or, specify a filter function. The Library will pass an argument to the function - an instance of bot being tested. A function must return a Promise. An resolved value of Promise (supposed to be a string) will be passed to the bot:
{
"user": function () {
return Promise.resolve('Hello world!');
}
}In case, if the message is from the bot, than:
{
"bot" : "Hello, %username%"
}It is possible to validate bot messages with RegExps:
{
"bot" : /^Hello/
}or with a filter function. The Library will pass two arguments into the function:
- bot, a bot instance itself,
- receivedMessage, a message object received from bot
Next example presents usage of filter function, that validates if chatbot returned a Number in range of 0 to 100:
{
"bot" : function ( receivedMessage) {
let value = parseInt(receivedMessage.text);
if (( value >= 0 ) && (value <=100)) {
return Promise.resolve('success');
} else {
return Promise.reject('failure');
}
}
}If you want to validate suggested actions of the message:
{
"bot" : "Hello world!",
"suggestedActions" : [
botbuilder.CardAction.imBack(null, "add", "Add"),
botbuilder.CardAction.imBack(null, "settings", "Settings")
]
}You could use filter function to validate suggested actions:
{
"bot" : function ( receivedMessage ) {
if ( receviedMessage.suggestedActions.length == 2 ) {
return Promise.resolve('success');
} else {
return Promise.reject('fail');
}
}
}It is possible to validate attachments and richcards returned by a bot. To validate attachment body you need to specify a step with attachments attribute, to validate an attachment layout you need to specify attachmentLayout attribute.
It is possible to combine these parameters together.
{
"attachmentLayout": "carousel",
"attachments": [
{
"contentType": "application/vnd.microsoft.card.hero",
"content": {
"title": "My Title",
"subtitle": "My Subtitle",
"images": [
{
"url": "Some Url"
},
{
"url": "Another Url"
}
]
}
}
]
},Or with bot attribute:
{
"bot" : "Hello World!",
"attachmentLayout": "carousel",
"attachments": [
{
"contentType": "application/vnd.microsoft.card.hero",
"content": {
"title": "My Title",
"subtitle": "My Subtitle",
"images": [
{
"url": "Some Url"
},
{
"url": "Another Url"
}
]
}
}
]
},Inside attachments body you could freely use filter functions. The function will receive a part of attachments object body for validation:
{
"bot" : "World!",
"attachmentLayout": function ( value ) {
return "carousel" == value ? Promise.resolve() : Promise.reject();
},
"attachments": [
{
"contentType": "application/vnd.microsoft.card.hero",
"content": {
"title": function ( value ) {
return "My Title" == value ? Promise.resolve() : Promise.reject('Wrong title');
},
"subtitle": function (value ) {
return "My Subtitle" == value ? Promise.resolve() : Promise.reject('Wrong subtitle');
},
"images": function ( value ) {
return 2 == value.length ? Promise.resolve() : Promise.reject('Wrong images count');
}
}
}
]
}It is possible to setup a state for session:
{
"session" : {userData: {userName : 'Joe'}}
}Another option is to specify a filter function. Current session object will be passed as a first argument into the filter:
{
"session" : function (session) {
session.userDta.userName = 'Joe';
return Promise.resolve(session);
}
}Example:
{
"endConversation" : true
}More about ending conversation you could find in official documentation.
Example:
{
"typing": true
}
It is possible to inject a custom step into the script. Such step contains a user-defined filter function (in attribute custom)_ that MUST return a Promise object.
Once, when the Promise will appear resolved/rejected state:
- rejected state will stop active test execution with error returned by the Promise
- fulfilled (resolved) state will continue the script execution
Example:
{
"custom": function () {
if ( someValidationFunc() ) {
return Promise.resolve();
} else {
return Promise.reject('Custom validation failed');
}
}
}
You could specify a dialog id that will be set as active or default dialog for the bot. In case, if you call that function in the middle of conversation (when session conversation object already created) this message will replace See an Example:
It is also possible to specify an arguments to the function:
Important! the message produces a side effect as the function manipulates with attributes: settings.defaultDialogId and settings.defaultDialogArgs
You also could specify a filter function. The Library will pass current instance of bot as argument into the function:
Global options will be applied to every script that will be processed by library. There are two ways to setup a global option:
- as an environment variable, Library will autoload it during startup;
- as an attribute of config object exposed in module.exports section of The Library;
- BOTBUILDERUNIT_TEST_TIMEOUT - timeout for script execution in milliseconds
- BOTBUILDERUNIT_REPORTER - logging style, supported values: empty, plain and beauty
For example, to run script with 10 seconds timeout and beautified output you need to execute something like:
export BOTBUILDERUNIT_REPORTER=beauty; export BOTBUILDERUNIT_TEST_TIMEOUT=10000; npm test
The library exposes config object in module.exports. Properties of an object:
- timeout - timeout for script execution in milliseconds
- reporter - the instance of reporting class. Default value: new PlainLogReporter(). Classes provided by library:
- PlainLogReporter, default, will output text messages in console
- EmptyLogReporter, nothing will be sent to output
- PlainLogReporter, colored and styled output, useful for long scripts
-
unit(bot , script, options) or unit(dialog, script, options) - Tests given bot instance or dialog with script. Returns a Promise that will be resolved in case of success. Method arguments:
- bot | dialog UniversalBot | function | Array, Bot instance or a dialog. Dialogs wrapped by Library into empty UniversalBot instance.
- script Array, List of test steps, each step represents one piece of conversation between a user and a bot;
- (optional) options object - a key-value object.
| key | description |
|---|---|
| title | String or null, value that represents test title. Will be rendered in test report |
| timeout | Integer, allowed execution time for a test |
| reporter | intance of BaseLogReporter, an log reporter instance, overwrites default log reporter |
Library provides an ConversationMock class with purpose to mock responses from the chatbot. Possible use cases for such feature are:
- you want to test a recognizer;
- you want to "prototype" a conversation flow and see how it looks and feels, without real implementation of the dialogs
Where steps is an array of standard waterfall dialog functions. Each step will be executed only once. The Library will pass standard arguments: session, arguments,next into step.
- getListener() returns a listener for WaterfallDialog. Once the listener executed the first step will be executed, and will move internal pointer to a next step. Second call will execute second step callback and so on...
-
sendMessagesStep( messages, afterFunc) - Creates a step for waterfall dialog. Arguments:
- messages argument is an array of strings, these messages will be sent to user;
- afterFunc(session, args, next) is a callback, will be called after messages will be sent.
- Base Example
- Session Management Example
- Startup Dialog Example
- Rich Card Validation Example
- Testing Standalone Dialog Example
- 0.7.1 - code coverage tool integrated into CI pipeline, finally. Improved attachment validation errors.
- 0.7.0 - new method - dialog(waterfallDialog, script), MemoryConnector, methods for testing standalone dialog or middleware,
- 0.6.5 - Switched license to LGPL, more tests for suggested actions;
- 0.6.4 - Better error messages;
- 0.6.3 - Fix for attachment validation by callback. Better error messages. Smoke tests for log reporters;
- 0.6.2 - Better error messages;
- 0.6.1 - Rich card and attachments validation. Warning! Bot instance removed from filter functions arguments, as it is useless, as it never used;
- 0.6.0 - support for custom steps, refactoring of steps execution flow. Warning! before and after attributes are not supported anymore, use custom steps instead
- 0.5.5 - documentation updates & fixes;
- 0.5.4 - TOC added into documentation, basic test for proactive messages, basic code coverage report added;
- 0.5.3 - fixes, examples;
- 0.5.2 - missed setDialogMessageSpec specification;
- 0.5.1 - documentation updates, support for startup dialog, refactorings;
- 0.5.0 - support for session messages
- 0.4.7 - support for suggestActions, minor fixes
- 0.4.2 - new static method for ConversationMock class - sendMessagesStep, minor fixes
- 0.4.0 - new output log, global options support
- 0.3.0 - timeout support, minor fixes
- 0.2.3 - fixed error with case then multiple messages from users awaited
- 0.2.2 - updated error messages in case if current message in script does not matching pattern for a bot's message
- 0.2.0 - removed ambiguity with user and bot messages, using "user" and "bot" instead of "out" and "in"
- 0.1.0 - initial version