OpenCV autoit udf

Do you want to use OpenCV v4+ in AutoIt v3 ?
If yes, then this udf might be for you.

In fact, the dll being a Component Object Model (COM) dll, it can be used in any client dans can use COM components. For example Office VBA, PowerShell.

Prerequisites
Usage
Running examples
How to translate python/c++ code to the UDF
Developpement
Breaking changes from v1
History

Prerequisites

Download and extract opencv-4.9.0-windows.exe into a folder
Download and extract autoit-opencv-4.9.0-com-v2.6.0.7z into a folder

Usage

AutoIt

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_UseX64=y
#AutoIt3Wrapper_Change2CUI=y
#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6
#AutoIt3Wrapper_AU3Check_Stop_OnWarning=y
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****

#include "autoit-opencv-com\udf\opencv_udf_utils.au3"

_OpenCV_Open("opencv-4.9.0-windows\opencv\build\x64\vc16\bin\opencv_world490.dll", "autoit-opencv-com\autoit_opencv_com490.dll")
OnAutoItExitRegister("_OnAutoItExit")
Example()

Func Example()
  Local $cv = _OpenCV_get()
  If Not IsObj($cv) Then Return

  Local $img = _OpenCV_imread_and_check($cv.samples.findFile("lena.jpg"))
  $cv.imshow("Image", $img)
  $cv.waitKey()
  $cv.destroyAllWindows()
EndFunc   ;==>Example

Func _OnAutoItExit()
  _OpenCV_Close()
EndFunc   ;==>_OnAutoItExit

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_UseX64=y
#AutoIt3Wrapper_Change2CUI=y
#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6
#AutoIt3Wrapper_AU3Check_Stop_OnWarning=y
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****

#include "autoit-opencv-com\udf\opencv_udf_utils.au3"
#include <GUIConstantsEx.au3>

_OpenCV_Open("opencv-4.9.0-windows\opencv\build\x64\vc16\bin\opencv_world490.dll", "autoit-opencv-com\autoit_opencv_com490.dll")
OnAutoItExitRegister("_OnAutoItExit")
Example()

Func Example()
  Local $cv = _OpenCV_get()
  If Not IsObj($cv) Then Return

  #Region ### START Koda GUI section ### Form=
  Local $FormGUI = GUICreate("show image in autoit gui", 400, 400, 200, 200)
  Local $Pic = GUICtrlCreatePic("", 0, 0, 400, 400)
  GUISetState(@SW_SHOW)
  #EndRegion ### END Koda GUI section ###

  Local $img = _OpenCV_imread_and_check($cv.samples.findFile("lena.jpg"))

  _OpenCV_imshow_ControlPic($img, $FormGUI, $Pic)

  Local $nMsg
  While 1
    $nMsg = GUIGetMsg()
    Switch $nMsg
      Case $GUI_EVENT_CLOSE
        ExitLoop
    EndSwitch
  WEnd

  $cv.destroyAllWindows()
EndFunc   ;==>Example

Func _OnAutoItExit()
  _OpenCV_Close()
EndFunc   ;==>_OnAutoItExit

PowerShell

powershell.exe -ExecutionPolicy UnRestricted -File example.ps1

#requires -version 5.0

Import-Module .\autoit-opencv-com\dotnet\opencv_utils.psm1

function Example() {
    $cv = [OpenCvComInterop]::ObjCreate("cv")
    $img = $cv.imread(( _OpenCV_FindFile "samples\data\lena.jpg" ))
    $cv.imshow("Image", $img)
    $cv.waitKey() | Out-Null
    $cv.destroyAllWindows()
}

[OpenCvComInterop]::DllOpen("opencv-4.9.0-windows\opencv\build\x64\vc16\bin\opencv_world490.dll", "autoit-opencv-com\autoit_opencv_com490.dll")

Example

[OpenCvComInterop]::DllClose()

csharp

Open x64 Native Tools Command Prompt for VS 2022

Runtime example

csc.exe example-runtime.cs autoit-opencv-com\dotnet\OpenCvComInterop.cs && example-runtime.exe

using System;
using System.ComponentModel;
using System.Runtime.InteropServices;

public static class Test
{
    private static void Example()
    {
        var cv = OpenCvComInterop.ObjCreate("cv");
        if (ReferenceEquals(cv, null))
        {
            throw new Win32Exception("Failed to create cv object");
        }

        var img = cv.imread(OpenCvComInterop.FindFile("samples\\data\\lena.jpg"));
        cv.imshow("image", img);
        cv.waitKey();
        cv.destroyAllWindows();
    }

    static void Main(String[] args)
    {
        OpenCvComInterop.DllOpen(
            "opencv-4.9.0-windows\\opencv\\build\\x64\\vc16\\bin\\opencv_world490.dll",
            "autoit-opencv-com\\autoit_opencv_com490.dll"
        );

        Example();
        OpenCvComInterop.DllClose();
    }
}

Compile time example

csc.exe example-compile.cs /link:autoit-opencv-com\dotnet\OpenCV.InteropServices.dll autoit-opencv-com\dotnet\OpenCvComInterop.cs && example-compile.exe

using System;
using System.ComponentModel;
using System.Runtime.InteropServices;
using OpenCV.InteropServices;

public static class Test
{
    private static void Example()
    {
        ICv_Object cv = new Cv_Object();

        var img = cv.imread(OpenCvComInterop.FindFile("samples\\data\\lena.jpg"));
        cv.imshow("image", img);
        cv.waitKey();
        cv.destroyAllWindows();
    }

    static void Main(String[] args)
    {
        OpenCvComInterop.DllOpen(
            "opencv-4.9.0-windows\\opencv\\build\\x64\\vc16\\bin\\opencv_world490.dll",
            "autoit-opencv-com\\autoit_opencv_com490.dll"
        );

        // Enable Registration-Free COM
        OpenCvComInterop.DllActivateManifest();

        Example();

        // Disable Registration-Free COM
        OpenCvComInterop.DllDeactivateActCtx();

        OpenCvComInterop.DllClose();
    }
}

Running examples

Prerequisites

Install 7-zip and add the 7-zip folder to your system PATH environment variable

Then, in Git Bash, execute the following commands

# go to the folder of your choice
# cd ...

# download autoit-opencv-4.9.0-com-v2.6.0.7z
curl -L 'https://github.com/smbape/node-autoit-opencv-com/releases/download/v2.6.0/autoit-opencv-4.9.0-com-v2.6.0.7z' -o autoit-opencv-4.9.0-com-v2.6.0.7z

# extract the content of autoit-opencv-4.9.0-com-v2.6.0.7z into a folder named autoit-opencv-com
7z x autoit-opencv-4.9.0-com-v2.6.0.7z -aoa -oautoit-opencv-com

# download opencv-4.9.0-windows.exe
curl -L 'https://github.com/opencv/opencv/releases/download/4.9.0/opencv-4.9.0-windows.exe' -o opencv-4.9.0-windows.exe

# extract the content of opencv-4.9.0-windows.exe into a folder named opencv-4.9.0-windows
./opencv-4.9.0-windows.exe -oopencv-4.9.0-windows -y

# download autoit-opencv-4.9.0-com-v2.6.0-src.zip
curl -L 'https://github.com/smbape/node-autoit-opencv-com/archive/refs/tags/v2.6.0.zip' -o autoit-opencv-4.9.0-com-v2.6.0-src.zip

# extract the autoit-addon and samples folders of autoit-opencv-4.9.0-com-v2.6.0-src.zip
7z x autoit-opencv-4.9.0-com-v2.6.0-src.zip -aoa 'node-autoit-opencv-com-2.6.0\autoit-addon\*' 'node-autoit-opencv-com-2.6.0\samples\*'
cp -rf node-autoit-opencv-com-2.6.0/* ./
rm -rf node-autoit-opencv-com-2.6.0

autoit

Now you can run any .au3 file in the samples folder.

powershell

To run a .ps1 file in the samples\dotnet use the following command

powershell.exe -ExecutionPolicy UnRestricted -File <path to the .ps1 file>

csharp

To run a .cs file in the samples\dotnet use the following command

Open x64 Native Tools Command Prompt for VS 2022

# samples\dotnet\csrun.bat <path to the .cs file> <arguments>
samples\dotnet\csrun.bat samples\dotnet\01-show-image.cs

[optional] Build the addon dll

This shows how to put performance critical tasks in c++ functions, export those functions in a dll and then use them in autoit.

Look at samples\tutorial_code\Histograms_Matching\calcHist_Demo.au3 for an example of usage.

Prerequisite

Install CMAKE >= 3.19
Install visual studio >= 10

Building

Run build.bat script located in the autoit-addon folder.

How to translate python/c++ code to the UDF

The transformation will usually be straight from python.

The translation usually involves 2 steps:

Finding the functions/constants names.
Transform the parameter types according to the UDF parameter. This step might involve looking at the opencv documentation.

Finding the functions/constants names

For a function named foo, there is usually a function named foo

For a constant FOO, there is usually a Global Const ending with _FOO and starting with $CV_.

Look into cv_enums.au3 to find and cv_interface.au3 to search for constants.

Transform the parameter types

For cv::Point, cv::Range, cv::Rect, cv::Scalar and cv::Size types,
there are _OpenCV_Point, _OpenCV_Range, _OpenCV_Rect, _OpenCV_Scalar and _OpenCV_Size functions to convert those parameters.

For cv::ScalarAll, there is _OpenCV_ScalarAll function.

Types which are *OfArrays, like InputArrayOfArrays, are harder to translate because, in AutoIt they are all Arrays or VARIANT.
It is always safe to use a VectorOfMat for those types.

However, if you really need to, you can transform an Array into a typed Array with the corresponding VectorOf constructor.
For example, to transform an Array of Int into a VectorOfInt, do

Local $aInt[] = [1, 2, 3]
Local $oVectorOfInt = ObjCreate("OpenCV.VectorOfInt").create($aInt)

Python translation example

Let's translate the following python code

blurred = cv2.GaussianBlur(image, (3, 3), 0)
T, thresh_img = cv2.threshold(blurred, 215, 255, cv2.THRESH_BINARY)
cnts, _ = cv2.findContours(thresh_img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

First line

blurred = cv2.GaussianBlur(image, (3, 3), 0)

The GaussianBlur documentation gives the following information

void cv::GaussianBlur   (   InputArray    src,
    OutputArray   dst,
    Size    ksize,
    double    sigmaX,
    double    sigmaY = 0,
    int   borderType = BORDER_DEFAULT 
  )     
Python:
  cv.GaussianBlur(  src, ksize, sigmaX[, dst[, sigmaY[, borderType]]] ) ->  dst

dst is an output array. It will be therefore put at the end and returned.

The AutoIt version of the function is

AutoIt:
  cv.GaussianBlur(  src, ksize, sigmaX[, dst[, sigmaY[, borderType]]] ) ->  dst

The python code will therefore become

$blurred = $cv.GaussianBlur($image, _OpenCV_Size(3, 3), 0)

Second line

T, thresh_img = cv2.threshold(blurred, 215, 255, cv2.THRESH_BINARY)

The threshold documentation gives the following information

double cv::threshold  (   InputArray    src,
    OutputArray   dst,
    double    thresh,
    double    maxval,
    int   type 
  )     
Python:
  cv.threshold( src, thresh, maxval, type[, dst]  ) ->  retval, dst

The AutoIt version of the function is

AutoIt:
  cv.threshold( src, thresh, maxval, type[, dst]  ) ->  retval, dst

Applying the same steps leads to

$T = $cv.threshold($blurred, 215, 255, $CV_THRESH_BINARY)
$thresh_img = $cv.extended[1]

; Or

$thresh_img = ObjCreate("OpenCV.cv.Mat")
$T = $cv.threshold($blurred, 215, 255, $CV_THRESH_BINARY, $thresh_img)

; Or

$cv.threshold($blurred, 215, 255, $CV_THRESH_BINARY)
$T = $cv.extended[0]
$thresh_img = $cv.extended[1]

Third line

cnts, _ = cv2.findContours(thresh_img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

The findContours documentation gives the following information

void cv::findContours   (   InputArray    image,
    OutputArrayOfArrays   contours,
    OutputArray   hierarchy,
    int   mode,
    int   method,
    Point   offset = Point() 
  )     
Python:
  cv.findContours(  image, mode, method[, contours[, hierarchy[, offset]]]  ) ->  contours, hierarchy

The AutoIt version of the function is

AutoIt:
  cv.findContours(  image, mode, method[, contours[, hierarchy[, offset]]]  ) ->  contours, hierarchy

The python code will become

$cnts = $cv.findContours($thresh_img, $CV_RETR_EXTERNAL, $CV_CHAIN_APPROX_SIMPLE)
$_ = $cv.extended[1]

Final result

Python

blurred = cv2.GaussianBlur(image, (3, 3), 0)
T, thresh_img = cv2.threshold(blurred, 215, 255, cv2.THRESH_BINARY)
cnts, _ = cv2.findContours(thresh_img, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

AutoIt

$blurred = $cv.GaussianBlur($image, _OpenCV_Size(3, 3), 0)

$T = $cv.threshold($blurred, 215, 255, $CV_THRESH_BINARY)
$thresh_img = $cv.extended[1]

$cnts = $cv.findContours($thresh_img, $CV_RETR_EXTERNAL, $CV_CHAIN_APPROX_SIMPLE)
$_ = $cv.extended[1]

Developpement

Prerequisites

Install CMAKE >= 3.19
Install visual studio >= 2017
Install Git for Windows
Install nodejs
Install Python >= 3.8

Environment

In Git BASH, excute the following commands

# get the source files
git clone https://github.com/smbape/node-autoit-opencv-com
cd node-autoit-opencv-com

# Install nodejs dependencies
npm ci

Generate the UDF files

cmd.exe //c 'autoit-opencv-com\build.bat'

Breaking changes from v1

Parameters order is now the same as in python. It changes the signature of the following functions, thus making them incompatible between v1 and v2 if you used the output parameters, or parameters with default values

cv::add
cv::addWeighted
cv::batchDistance
cv::bilateralFilter
cv::bitwise_and
cv::bitwise_not
cv::bitwise_or
cv::bitwise_xor
cv::blur
cv::boxFilter
cv::buildOpticalFlowPyramid
cv::calcCovarMatrix
cv::calcHist
cv::calcOpticalFlowPyrLK
cv::calibrateCamera
cv::calibrateCameraExtended
cv::calibrateCameraRO
cv::calibrateCameraROExtended
cv::calibrateHandEye
cv::calibrateRobotWorldHandEye
cv::Canny
cv::cartToPolar
cv::colorChange
cv::connectedComponents
cv::connectedComponentsWithStats
cv::convertMaps
cv::convertScaleAbs
cv::convexHull
cv::copyMakeBorder
cv::cornerEigenValsAndVecs
cv::cornerHarris
cv::cornerMinEigenVal
cv::cvtColor
cv::dct
cv::demosaicing
cv::detailEnhance
cv::dft
cv::dilate
cv::distanceTransform
cv::distanceTransformWithLabels
cv::divide
cv::divSpectrums
cv::edgePreservingFilter
cv::erode
cv::estimateAffine2D
cv::estimateAffine3D
cv::estimateAffinePartial2D
cv::estimateTranslation3D
cv::fastNlMeansDenoising
cv::fastNlMeansDenoisingColored
cv::fastNlMeansDenoisingColoredMulti
cv::fastNlMeansDenoisingMulti
cv::filter2D
cv::filterHomographyDecompByVisibleRefpoints
cv::findChessboardCorners
cv::findChessboardCornersSB
cv::findCirclesGrid
cv::findContours
cv::findHomography
cv::GaussianBlur
cv::gemm
cv::getDerivKernels
cv::getRectSubPix
cv::goodFeaturesToTrack
cv::goodFeaturesToTrackWithQuality
cv::HoughCircles
cv::HoughLines
cv::HoughLinesP
cv::HoughLinesWithAccumulator
cv::idct
cv::idft
cv::illuminationChange
cv::integral
cv::integral2
cv::integral3
cv::intersectConvexConvex
cv::invert
cv::Laplacian
cv::matchTemplate
cv::matchTemplateParallel
cv::meanStdDev
cv::morphologyEx
cv::mulSpectrums
cv::multiply
cv::mulTransposed
cv::PCACompute
cv::PCACompute2
cv::pencilSketch
cv::phase
cv::polarToCart
cv::preCornerDetect
cv::projectPoints
cv::pyrDown
cv::pyrMeanShiftFiltering
cv::pyrUp
cv::recoverPose
cv::reduce
cv::reduceArgMax
cv::reduceArgMin
cv::remap
cv::reprojectImageTo3D
cv::resize
cv::Scharr
cv::searchTemplate
cv::sepFilter2D
cv::Sobel
cv::solve
cv::solvePnP
cv::solvePnPGeneric
cv::solvePnPRansac
cv::solvePoly
cv::spatialGradient
cv::sqrBoxFilter
cv::stereoCalibrate
cv::stereoCalibrateExtended
cv::stereoRectify
cv::stereoRectifyUncalibrated
cv::stylization
cv::subtract
cv::SVDecomp
cv::textureFlattening
cv::undistort
cv::undistortPoints
cv::warpAffine
cv::warpPerspective
cv::AffineFeature::detectAndCompute
cv::AgastFeatureDetector::detectAndCompute
cv::AKAZE::detectAndCompute
cv::BRISK::detectAndCompute
cv::FastFeatureDetector::detectAndCompute
cv::Feature2D::detectAndCompute
cv::GFTTDetector::detectAndCompute
cv::KAZE::detectAndCompute
cv::MSER::detectAndCompute
cv::ORB::detectAndCompute
cv::SIFT::detectAndCompute
cv::SimpleBlobDetector::detectAndCompute
cv::BackgroundSubtractor::apply
cv::BackgroundSubtractorKNN::apply
cv::BackgroundSubtractorMOG2::apply
cv::cuda::GpuMat::convertTo
cv::dnn::Net::forward
cv::fisheye::calibrate
cv::fisheye::distortPoints
cv::fisheye::estimateNewCameraMatrixForUndistortRectify
cv::fisheye::projectPoints
cv::fisheye::stereoCalibrate
cv::fisheye::stereoRectify
cv::fisheye::undistortImage
cv::fisheye::undistortPoints
cv::flann::Index::knnSearch
cv::flann::Index::radiusSearch
cv::Mat::convertTo
cv::ml::ANN_MLP::predict
cv::ml::Boost::predict
cv::ml::DTrees::predict
cv::ml::EM::predict
cv::ml::KNearest::predict
cv::ml::LogisticRegression::predict
cv::ml::NormalBayesClassifier::predict
cv::ml::NormalBayesClassifier::predictProb
cv::ml::RTrees::predict
cv::ml::StatModel::predict
cv::ml::SVM::predict
cv::ml::SVMSGD::predict
cv::QRCodeDetector::decodeMulti
cv::QRCodeDetector::detectAndDecodeMulti
cv::segmentation::IntelligentScissorsMB::getContour
cv::VideoCapture::retrieve

History

A previous attempt to bring OpenCV usage to AutoIt was functionnal but not practical.
The user has to know too much information before correctly use the UDF.

This is an attempt to make the usage of OpenCV less painfull in AutoIt

smbape/node-autoit-opencv-com

OpenCV autoit udf

Table Of Contents

Prerequisites

Usage

AutoIt

PowerShell

csharp

Runtime example

Compile time example

Running examples

Prerequisites

autoit

powershell

csharp

[optional] Build the addon dll

Prerequisite

Building

How to translate python/c++ code to the UDF

Finding the functions/constants names

Transform the parameter types

Python translation example

First line

Second line

Third line

Final result

Developpement

Prerequisites

Environment

Generate the UDF files

Breaking changes from v1

History