inqlik/inqlik-tools

Not recognising qdf variables

Opened this issue · 3 comments

veglar commented

The documentation aroud using the QlikView script syntax checker from inqlik CLI needs an overview.
I tried running the syntax checker while using QDF and run into trouble. See error below.

------------------------------
 $(must_Include=$(vG.LocalePath)2.Swe.qvs);
>>>>> Parse error. File: "C:\QDF\1.Production\100.System\2.Extract.Excel\3.Include\99.Extract.Excel\Main.qvs", line: 4 col: 1 message: Variable vG.LocalePath not defined
------------------------------

I notice the following in the settings file, but no documentation on how to use it.

// Implicit include file (use it for example to declare predefined variables of QlikView Deployment Framework) 
  "qv_script_check_syntax_impicit_include_file": "default_inqlude.qvs"

(I have managed to run scripts with qdf variables before without any modification, but I don't know why it does not work all the time)

vadimtsushko commented

Hi Vegar.
What is your current settings?
Is "qv_script_check_syntax" parameter set do default false value?
In #5 I proposed to change "qv_script_use_cli" to true, and leave other parameters intact. That should emulate previous behaviour of plugin, namely to run load script but not check syntax with embedded parser.

veglar commented

Yes,
I am aware of this syntax. When qv_script_check_syntax is set to false I have no problems. What I am suggesting is it to put a bit more detail around the documentation of the "qv_script_check_syntax_impicit_include_file" as I got the impression that it could solve the QDF-problem. Am I right?

vadimtsushko commented

OK, now I understand the problem.
You are absolutely right. I definitely should document all parser related features somehow.
It is not so easy though. Frankly speaking I'm not so sure that the parser is ready for external usage. We use it internally and it helps us significantly, but the setup process was not so easy and intuitive.
Setup description should probably describe our approach to QDF process too. I have a plan to write an article about it, probably with a sample of our QDF project, with some internal utilities and so on, but so far have not enough time for it.

I can try to answer your immediate question though.
One of the fundamental things which are checked by parser is that all variables used in the load script are defined in the script before first usage. In QDF projects, QDF bootstrapping code create all internal variables dynamically - before parser kicks in.

All our ETL qvw applications have identical internal script:

// First Base Variable Include ,Use relative path to 1.init.qvs
// Contains base searchpath in this container
let vG.BasePath=; 
// Basic search for 1.Init.qvs
$(Include=..\..\..\..\3.include\1.basevariable\1.init.qvs);
$(Include=..\..\..\3.include\1.basevariable\1.init.qvs);
$(Include=..\..\3.include\1.basevariable\1.init.qvs);
$(Include=..\3.include\1.basevariable\1.init.qvs);
Exit script when '$(vG.BasePath)'= '';

$(must_include=$(vG.SubPath)RunMainApp.qvs);

So inside QVW file we have only QDF bootstrapping code plus invoking of RunMainApp.qvs

Typical RunMainApp.qvs looks like:

// Include and load QVS script corresponding to QVW file

call LoadContainerGlobalVariables('Src');
call LoadContainerGlobalVariables('App','QVD');

LET vLoadStartTime = now();
LET vScriptStartTime = replace(Num(now()),',','_');

LET include_file = '$(vG.CustomPath)' & Upper(replace(DocumentName(), '.qvw', '.qvs'));

$(must_include=$(vG.BasePath)..\99.Shared_Folders\3.Include\1.BaseVariable\InitUniversalVariables.qvs);

$(must_include=$(vG.SubPath)InQlik.qvs);

$(must_include=$(include_file));

So, basically for each qvw application we have corresponding qvs script with main logic (Invent.qvs for Invent.qvs for example). Same that script is checked by the parser.
To facilitate such scenario parser checks additional qvs file (by default default_include.qvs immediately near the main qvs file) If such file exists, it is invoked before execution of main script, as if main script has the include directive with that file. So we use default_include.qvs "to explicitly define" all internal QDF variables.

It would be tedious and error-prone to create such default_include.qvs file manually, so we have vb script, executed at project initiation phase, or lately - create_default_include.qvs

Dim objFSO
Dim objQV
Set objQV=CreateObject("QlikTech.QlikView")
Set objFSO = CreateObject("Scripting.FileSystemObject")

rootPath = objFSO.GetParentFolderName(objFSO.GetParentFolderName(objFSO.GetParentFolderName(objFSO.GetParentFolderName(Wscript.ScriptFullName))))


CreateVarFile "2.Transformation"
CreateVarFile "3.Source"
CreateVarFile "1.App"


Set FSO = Nothing

Sub CreateVarFile(targetDir)
    targetPath = rootPath & "\" & targetDir
    if targetDir = "1.App" then
        qvwFile = targetPath & "\4.Mart\_NewFileTemplate.qvw"
    else
        qvwFile = targetPath & "\1.Application\_NewFileTemplate.qvw"
    end if 
    varFile = targetPath & "\3.Include\6.Custom\default_include.qvs"
''    WScript.Echo qvwFile & " " & varFile
    if not objFSO.FileExists(qvwFile) then
        if not objFSO.FileExists("_NewFileTemplate.qvw") then
            WScript.Echo "Cannot find file _NewFileTemplate.qvw"
            WScript.Exit 1
        end if
        objFSO.CopyFile "_NewFileTemplate.qvw" ,qvwFile
    end if     
    Dim folder, file
    Dim objSource
    Dim objVars, varcontent, objTempVar, varname, i
    Dim str, fl
    Set objSource = objQV.OpenDoc(qvwFile)
    set objVars = objSource.GetVariableDescriptions
    for i = 0 to objVars.Count - 1
        set objTempVar = objVars.Item(i)
        varname=Trim(objTempVar.Name)
        objSource.RemoveVariable(varname)
    next 'end of loop
    objSource.Reload    
    set objVars = objSource.GetVariableDescriptions
''    Set fl = objFSO.OpenTextFile(varFile, 2, True, -1) 
    Set fl = CreateObject("ADODB.Stream")
    fl.Open
    fl.CharSet = "utf-8"
    for i = 0 to objVars.Count - 1
        set objTempVar = objVars.Item(i)
        varname=Trim(objTempVar.Name)
        Set objSourceVar=objSource.Variables(varname)
        varcontent=objSourceVar.GetRawContent
        fl.WriteText "LET " & varname & " = '" & varcontent & "';" & vbNewline
        objSource.RemoveVariable(varname)
    next 'end of loop

    fl.SaveToFile varFile, 2
    Set fl = Nothing
    WScript.Echo varFile & " created"  
    objSource.Save
    objSource.CloseDoc                  
End Sub

Sub Save2File (sText, sFile)
    Dim oStream
    Set oStream = CreateObject("ADODB.Stream")
    With oStream
        .Open
        .CharSet = "utf-8"
        .WriteText sText
        .SaveToFile sFile, 2
    End With
    Set oStream = Nothing
End Sub