This is the repository for Hackolade plugins. Plugins let anyone create support so new NoSQL databases can leverage the Hackolade data modeling engine, if they're not already natively supported in Hackolade. Several plugins have been created by Hackolade for leading NoSQL databases, but you may build your own and have it listed in the DB Target Plugin Manager of the application.
The plugin architecture of Hackolade lets you create your own NoSQL database ‘targets’, respecting its characteristics, through a combination of configuration and custom JavaScript code. Custom JS code is only necessary for Reverse- and Forward-Engineering.
The process includes 3 main steps:
This guide walks you through the steps to create your own plugin in your own GitHub repository. Once you’re ready to make your plugin public, you may create a pull request for your entry in the registry file in this repository.
Each NoSQL document database has its own personality: terminology, storage approach, primary keys, data types, indexing, partitioning/sharding, API, query language, etc... At Hackolade, after adapting our engine to a couple of rather different NoSQL vendors, we quickly realized that we were going to have a hard time keeping up with the frequent appearance of new databases on the market. So, in order to unleash the power of our data modeling engine, we decided to rewrite the application and open up our features through a plugin architecture.
With the customization of the properties pane, you're able to control attributes specific to each DB at the following levels: model, container, entity, field/attribute, indexing, sharding, etc...
The customization is performed through the configuration of the following modules:
- localization
- data types
- properties panes
- connection and authentication parameters
- reverse-engineering parameters for document sampling and schema inference
- forward-engineering if the target DB supports it (TBA)
You first need to carefully define the hierarchy for the database, and its related terminology. Consider some differences between the MongoDB and the Couchbase hierarchies, for example:
Next, you need to define data types. Many databases simply support the standard JSON data types. Others have additional data types with special behaviors.
You may specify the order in which the data types appear.
Each data type can inherit default properties and constraints from the standard Hackolade configurations. But you may want to create additional properties and constraints for some data types. To define these properties and controls, you may choose among different controls:
The same controls can be used to define information in different properties panes and lower tabs, for example for indexing, sharding, etc..
In order for reverse-engineering to be performed, it is first necessary to connect to the database. You need to define the protocol, connection parameters, and authentication parameters.
For Reverse-Engineer to be performed, it is required to write custom JavaScript code.
This code will connect to the database according to the connection parameters defined above. It will also invoke queries to discover the database objects so the user can select what needs to be reverse-engineered. Finally, the code will query the database to obtain a sample and pass it on to Hackolade's engine for schema inference.
The easiest way to create a new plugin is to copy an existing one as a starting point, and to edit it progressively, verifying along the way that the desired behavior gets reflected in the application. When choosing an existing plugin, make sure to take one with a storage model as close as possible to yours.
It is strongly suggested to operate carefully, progressing with small steps, and testing very frequently. The application requires to be restarted to load the plugin configuration modifications. If the plugin does not appear in the target selection dialog when creating a new a model, that's generally a sign that there are JSON syntax errors or conflicts.
Plugins are stored in the %userprofile%/.hackolade/plugins directory. The plugin folder name must be the same as the name property in package.json file of your plugin.
The creation process follows these steps:
- package.json file
- logo file
- localization file
- data types
- properties panes
- input controls
- reverse-engineering
- forward-engineering
The package.json file is the plugin entry point. This file is required and contains initial info about plugin such as name, version, author, target name etc.. It must remain in the root of your plugin folder.
The mapping below shows how the information is used throughout the application.
name (string) - required; unique plugin name
version (string) - required; used to compare versions and provide possibility to update plugin
versionDate (string) - optional; for internal use only
author (string) - required; defines plugin author
engines (object) - required; used to define the lowest Hackolade version supporting this plugin
Example:
"engines": {
"hackolade": "1.9.x",
}
contributes (object) - required; provides information about the database target and available features
Example:
"contributes": {
"target": {
"applicationTarget": "ELASTICSEARCH",
"title": "Elasticsearch",
"versions": [
"v6.0"
]
},
"features": {
"nestedCollections": true
}
}
target (object) - provides information about target name for which the plugin was created; applicationTarget value must be uppercase
features (object) - optional; object which defines what features will be enabled or disabled; Key is the name of feature. Value - true or false;
Available features:
- nestedCollections (boolean) - optional: if true, enables parent-child entities [default: false]
- disablePatternField (boolean) - optional: if false, disables the ability to define attributes as pattern fields [default: false]
description - optional; this data will be displayed in the list of plugins
disabled - optional; used to mark the plugin as disabled [default: false]
logo.jpg - the plugin logo image to be displayed in the list of plugins. Name and extension of the file cannot be changed. The file is placed in the root folder next to the package.json file. If not set - the first letter from name property in package.json file will be used instead of a logo.
The localization folder contains files for language / target localization of the target database. Language localization can be implemented by creating *.json files inside localization folder. File name for localization should correspond to ISO naming convention (e.g. en for English localization)
Each localized item inside *.json file is represented by a key/value pair e.g.:
"MAIN_MENU___ADD_COLLECTION": "Add Collection"
If you delete any predefined property from this file - default text will be displayed.
There are 7 base types in Hackolade:
- String
- Boolean
- Numeric
- Document
- Array
- Binary
- Null
If you don’t need to change name and behavior of basic types - you can skip this configuration block. If you want to customize basic type or create a custom type - you can create your own configuration. The only condition is that new type must be INHERITED from the basic types above.
For example in order to create custom type <list> using the <array> type as a basis, it is necessary to perform the following steps:
a. Create list.json file inside types sub-folder of the plugin folder (notice that file name should be the same as type name)
b. Edit list.json to set the name, parent type (one out of list of 7 base types) and abbreviations for your custom type in the properties object inside list.json file:
{
"name": "list",
"erdAbbreviation": "<list>",
"dtdAbbreviation": "[...]",
"parentType": "array"
}
Some data datatypes have many subtypes. For example a numeric datatype may allow: integer, smallint, bigint, float, double, etc... In such case, we advise to only define one datatype (numeric) and put the subtypes in a dropdown mode.
The syntax works as follows: name (string) - required; type name
erdAbbreviation (string) - optional; abbreviation that is used to display in the ERD table
dtdAbbreviation (string) - optional; abbreviation used to define field visually on DTD
parentType (string) - optional; one out of 7 base types (used if custom type should be created)
sample (string) - optional; value that is used to display in the corresponding input in Property Pane by default
useSample (boolean) - optional; defines whether to use default value for sample or not
mode (array) - required for numeric fields; there are 4 base modes for numeric types:
- integer32
- integer64
- decimal128
- double
It is possible to define own mode with custom name that inherits base mode behaviour (esp. validation). For example here’s how a new mode int will be created:
{
"name": "int"
"parentType": "integer32",
"sample": 12
}
defaultValues (array) - optional; used to define properties in PP; can be different and custom for each type
There are multiple levels in the Hackolade hierarchy:
- model: each Hackolade file represents one model for a given DB target
- container: a database, a namespace, a bucket, a keyspace, or a region
- entity: a collection, a table, an index
- field: an attribute
- view (if applicable)
The properties pane can be configured for each level. And each level can have multiple properties pane tabs.
All configurations are stored in the properties_pane sub-folder of the plugin folder. Each level has its own sub-folder with corresponding naming:
- model_level
- container_level
- entity_level
- field_level
- view_level
Inside each folder there’s a [LEVELNAME]LevelConfig.json file which contains configurations for its properties pane tabs.
You can add defaultData.json file to a properties_pane folder to define default values to be displayed in the properties pane.
Each *.json file represents an array with objects. Each object has configurations for a particular tab. Tab name is defined in lowerTab property. All properties are defined as an array and have the same structure across all properties pane configurations. Here’s an example of details tab with one property comments:
[
{
"lowerTab": "Details",
"structure": [
{
"propertyName": "Comments",
"propertyKeyword": "comments",
"shouldValidate": false,
"propertyTooltip": "comments",
"propertyType": "details",
"propertyDefault": "Some comment",
"template": "textarea",
"valueType": "string"
}
]
}
]
lowerTab is used to define the lower tab for the hierarchical level, such as details, indexes, sharding, etc...
structure property represents a collection of objects, each describing field type and behavior.
Container Level Keys - fields which are the keys at the container level and automatically added to entities when they are created. They are defining at containerLevelConfig by property containerLevelKeys. For example:
{
"containerLevelKeys": [...],
"structure": [...],
"lowerTab": "..."
}
Setting container level keys has the similar structure to Property controls. For example key with name "Index":
{
"containerLevelKeys": [{
"labelName": "Index",
"defaultName": "_index",
"propertyPrimaryKey": true,
"typeName": "Data type",
"typeOptions": ["string"],
"defaultType": "string",
"disabledFieldOption": true,
"sampleGen": "&containerName",
"propertyName": "Index",
"propertyKeyword": "index",
"propertyType": "text"
}]
}
labelName (string) - required; name of property in properties pane
propertyPrimaryKey (boolean) - required; whether the field is a primary key
typeName (string) - required; label name of selector type
typeOptions (array) - required; variant of types for field
defaultType (string) - optional; default field type
defaultName (string) - optional; default name of field
sampleGen (string) - optional; generate sample for field in preview, can contains next variants:
- &random - will generate random value
- &entityName - link to entity name
- &containerName - link to container name
- constant - some constant value
disabledFieldOption (boolean) - optional; disable options of field
Different input controls can be defined for each property:
- simple text
- text area (pop-up text)
- dropdown selection
- numeric
- checkbox (boolean)
- properties group
- field list
- field list with dropdown of attributes
Each control is further described below. The control is defined using the following syntax:
propertyName (string) - required; used to display label in the Properties
Panes for a property.
propertyKeyword (string) - required; used in the main code as a keyword; no whitespaces are allowed.
propertyType (string) - required; field types; possible values are:
- simple text: text
- text area (pop-up text): details
- dropdown selection: select
- numeric: numeric
- checkbox (boolean): checkbox
- properties group: group
- field list: tagInput
template (string) - optional; template used in the modal window if propertyType is details; possible value is: details: textarea, or tagInput: collectiontree
options (array) - optional; used to define options in the select input if propertyType is select.
propertyTooltip (string) - optional; only taken into account for propertyTypes: text area, dropdown, properties group, and field list
valueType (string) - optional; type (one out of 7 basic types) to define validation rules.
shouldValidate (string) - optional; defines whether field should be validated or any value is allowed. Validation rules are defined in validationRegularExpressions.json;
propertyDefault (string|number|boolean) - optional; contains default value of property. Type of value dependent on propertyType.
For your properties, you may choose among a number of input controls:
{
"propertyName": "Simple text",
"propertyKeyword": "simpletextProp",
"propertyType": "text",
}
Used to add a description or comments with defined template property
{
"propertyName": "Text area",
"propertyKeyword": "textareaProp",
"propertyTooltip": "Popup for multi-line text entry",
"propertyType": "details",
"template": "textarea"
}
{
"propertyName": "Dropdown selection",
"propertyKeyword": "dropdownProp",
"shouldValidate": false,
"propertyTooltip": "Select from list of options",
"propertyType": "select",
"options": [
"Option 1",
"Option 2",
"Option 3",
"Option 4"
]
}
{
"propertyName": "Numeric",
"propertyKeyword": "numericProp",
"propertyType": "numeric",
"valueType": "number",
"allowNegative": false,
"propertyValidate": true
},
{
"propertyName": "Checkbox",
"propertyKeyword": "checkboxProp",
"propertyType": "checkbox"
},
{
"propertyName": "Group",
"propertyType": "group",
"propertyKeyword": "grpProp",
"propertyTooltip": "",
"structure": [
{
"propertyName": "Simple Grp Text",
"propertyKeyword": "simpleGrpText",
"propertyType": "text"
},
{
"propertyName": "Group Number",
"propertyKeyword": "grpNumber",
"propertyValidate": true,
"propertyType": "numeric",
"valueType": "number",
"allowNegative": false
}
]
}
{
"propertyName": "Field List",
"propertyKeyword": "keyList",
"propertyType": "tagInput",
"template": "collectiontree"
}
{
"propertyName": "Field List w/ dropdown",
"propertyKeyword": "keyListOrder",
"propertyType": "tagInput",
"template": "collectiontree",
"types": [
"ascending",
"descending"
]
}
All configurations for Reverse Engineering are stored in folder reverse_engineering.
The reverse-engineering structure is represented by several blocks and include:
- configuration for connection settings modal and authentication
- API program to access the DB and perform reverse-engineering
- error messages
- package file with npm modules dependencies
Connection and authentication params and connection settings modal configuration are defining in the connectionSettingsModalConfig.json file.
[
{
"lowerTab": "Connection",
"structure": [
{
"inputLabel": "Disable SSl Authentication",
"inputKeyword": "disableSSL",
"inputType": "checkbox",
"inputTooltip": "Disable SSl Authentication for local instance"
}
]
}
]
This config contain a list of templates (tabs) for different part of connection settings split by categories and has the next structure:
lowerTab (string) - required; tab name in connection settings modal
structure (array) - required; list of inputs for connection settings modal
inputLabel (string) - required; connection settings input label
inputKeyword (string) - required; keyword for connection settings param which is used in the RE process
inputType (string) - required; type of connection settings input. Available input types to use when customizing connection settings template are:
- text
- checkbox
- password
inputTooltip (string) - optional; connection settings input tooltip message
inputPlaceholder (string) - optional; placeholder for connection settings input
isHiddenKey (string) - optional; flag that defines values which would be replaced by ****** in the log file
defaultValue (string) - optional; default value for connection settings param
The file api.js is an adapter between the Hackolade application and the target database that allows you to perform reverse-engineering and process data using the API methods:
- connect()
- disconnect()
- testConnection()
- getDatabases()
- getDocumentKinds()
- getDbCollectionsNames()
- getDbCollectionsData()
The file config.js consists of error message list and property
errors (object) - list of error messages
excludeDocKind (array) - list of properties which would be excluded from select docType modal
{
"errors": {
"NO_DATABASES": "There are no databases in CosmosDB instance",
"WRONG_CONNECTION": "Can not connect to CosmosDB instance"
},
"excludeDocKind": ["id"]
}
The file package.json contains a list of dependencies that are required to execute RE via api.js
In order for your plugin to appear as downloadable within the Hackolade DB Target Plugin Manager, you need to submit a Pull Request for a modification commit on https://github.com/hackolade/plugins/blob/master/pluginRegistry.json
Add a record with the following structure:
{
"name": "your-plugin-name",
"target": "application-target",
"author": "you",
"authorRepo": "https://github.com/<you>",
"description": "Target plugin description",
"repository": "https://github.com/<you>/<your-plugin-name>"
},
Send an email to info@hackolade.com and we'll review your submission, and release the plugin if all goes well.