Build ADF Code Started Giving Errors
SravanDodda opened this issue · 17 comments
Hello,
We have been using Build ADF Code for quite some time and was working great until about 4 weeks ago. It started giving error - "The property 'globalParameters' cannot be found on this object. Verify that the property exists.". We tried to resolve it modifying ARM template files to remove reference to globalParameters but to no avail
Interestingly, our ADF pipelines are NOT using any global parameters. Not sure why this error and why now
Here is the log:
2022-12-01T16:25:50.7037626Z === Validating other rules ...
2022-12-01T16:26:01.3577512Z ##[debug]Leaving D:\a_tasks\BuildADFTask_8a00f62d-c46d-4019-9e11-c05d88821db8\1.14.1173\BuildADF.ps1.
2022-12-01T16:26:01.4093112Z ##[debug]Caught exception from task script.
2022-12-01T16:26:01.4159581Z ##[debug]Error record:
2022-12-01T16:26:01.4754218Z ##[debug]Test-AdfCode : The property 'globalParameters' cannot be found on this object. Verify that the property exists.
2022-12-01T16:26:01.4768873Z ##[debug]At D:\a_tasks\BuildADFTask_8a00f62d-c46d-4019-9e11-c05d88821db8\1.14.1173\BuildADF.ps1:46 char:14
2022-12-01T16:26:01.4782374Z ##[debug]+ ... $r = Test-AdfCode -RootFolder "$RootFolder" -ConfigPath $Confi ...
2022-12-01T16:26:01.4796068Z ##[debug]+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2022-12-01T16:26:01.4809370Z ##[debug] + CategoryInfo : NotSpecified: (:) [Test-AdfCode], PropertyNotFoundException
2022-12-01T16:26:01.4823061Z ##[debug] + FullyQualifiedErrorId : PropertyNotFoundStrict,Test-AdfCode
2022-12-01T16:26:01.4835915Z ##[debug]
2022-12-01T16:26:01.4857109Z ##[debug]Script stack trace:
2022-12-01T16:26:01.4894256Z ##[debug]at Test-AdfCode, D:\a_tasks\BuildADFTask_8a00f62d-c46d-4019-9e11-c05d88821db8\1.14.1173\ps_modules\azure.datafactory.tools\public\Test-AdfCode.ps1: line 96
2022-12-01T16:26:01.4907588Z ##[debug]at , D:\a_tasks\BuildADFTask_8a00f62d-c46d-4019-9e11-c05d88821db8\1.14.1173\BuildADF.ps1: line 46
2022-12-01T16:26:01.4920666Z ##[debug]at , : line 1
2022-12-01T16:26:01.4933671Z ##[debug]at , : line 22
2022-12-01T16:26:01.4946693Z ##[debug]at , : line 18
2022-12-01T16:26:01.4959914Z ##[debug]at , : line 1
2022-12-01T16:26:01.4979792Z ##[debug]Exception:
2022-12-01T16:26:01.5083785Z ##[debug]System.Management.Automation.PropertyNotFoundException: The property 'globalParameters' cannot be found on this object. Verify that the property exists.
2022-12-01T16:26:01.5097213Z ##[debug] at System.Management.Automation.ExceptionHandlingOps.CheckActionPreference(FunctionContext funcContext, Exception exception)
2022-12-01T16:26:01.5110801Z ##[debug] at System.Management.Automation.Interpreter.ActionCallInstruction2.Run(InterpretedFrame frame) 2022-12-01T16:26:01.5123815Z ##[debug] at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame) 2022-12-01T16:26:01.5137240Z ##[debug] at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame) 2022-12-01T16:26:01.5149921Z ##[debug] at System.Management.Automation.Interpreter.Interpreter.Run(InterpretedFrame frame) 2022-12-01T16:26:01.5162298Z ##[debug] at System.Management.Automation.Interpreter.LightLambda.RunVoid1[T0](T0 arg0) 2022-12-01T16:26:01.5175288Z ##[debug] at System.Management.Automation.PSScriptCmdlet.RunClause(Action1 clause, Object dollarUnderbar, Object inputToProcess)
2022-12-01T16:26:01.5188388Z ##[debug] at System.Management.Automation.PSScriptCmdlet.DoEndProcessing()
2022-12-01T16:26:01.5201562Z ##[debug] at System.Management.Automation.CommandProcessorBase.Complete()
2022-12-01T16:26:01.5263290Z ##[error]The property 'globalParameters' cannot be found on this object. Verify that the property exists.
2022-12-01T16:26:01.5265312Z ##[debug]Processed: ##vso[task.logissue type=error]The property 'globalParameters' cannot be found on this object. Verify that the property exists.
2022-12-01T16:26:01.5284130Z ##[debug]Processed: ##vso[task.complete result=Failed]
2022-12-01T16:26:01.5801971Z ##[section]Finishing: Validate ADF files
Pipeline design is attached
Could you help?
I test the task in my test environment and everything looks good:
Can you provide ADF code in order to replicate the issue?
I test the task in my test environment and everything looks good:
Thanks for your help
Which one should I test?
Which one should I test?
ADF-BIGroupFacts01-Dev
I am sorry, it should be "Azure ADF". When I publish ADF, ARM templates are getting updated in "ADF-BIGroupFacts01-Dev" and pipelines are getting updated in "Azure ADF". "ADF-BIGroupFacts01-Dev" is our development ADF name. Git is configured in ADF for collaboration and deploy branch to "Dev" branch in Azure DevOps
Does it make sense?
One other note, in the ADF-BIGroupFacts01-Dev repo, there were no folders other than "globalParameters" and one more (don't remember). I have copied the folders (dataflow, pipeline and so on) from "Azure ADF" to ADF-BIGroupFacts01-Dev repo to see if that works
For both folders the same issue raises:
Yes, this duplication thing is new. Even before the duplication thing, "globalParameters" issue was happening.
The duplication is because you have the same name for a pipeline as well as dataflow.
But that does not cause the error.
Correct. I just realized that while going through Data Flows and Pipelines
So, you are not able to reproduce the 'globalParameter" error?
I reproduced the error thanks to the attached code.
It will be released in PS module first (ver.0.110), and in DevOps extension afterwards (tonight).
Fixed in ver.1.15.1239, just released.
Fixed in ver.1.15.1239, just released.
Thank you. Appreciate your help
How do I install the new version by replacing the existing one? Could you provide some documentation of how to do that?
It should be installed automatically.
It should be installed automatically.
Thank you. Now I get this and failing. Could you help?
You can not point ARM Template parameter file as a config.
This task is not for deploying from ARM Template but from adf code directly.
Read more in documentation and watch video on my YouTube channel:
https://sqlplayer.net/adftools
You can not point ARM Template parameter file as a config. This task is not for deploying from ARM Template but from adf code directly. Read more in documentation and watch video on my YouTube channel: https://sqlplayer.net/adftools
Got it. Thank you so much for your great help and taking care of this in no time. Really appreciate it and am a big follower on YouTube as well