dxatscale/sfpowerscripts

Validate do not run test for source package

alanjaouen opened this issue · 5 comments

alanjaouen commented

Describe the bug
during a validate, source package are deployed without running test

To Reproduce
Steps to reproduce the behavior:
package descriptor:

    {
      "package": "be-legacy",
      "type": "source",
      "path": "./src/be-legacy",
      "preDeploymentScript": "scripts/core/pre.sh",
      "isOptimizedDeployment": false,
      "skipDeployOnOrgs": ["prod-fr", "st-fr", "prod-de", "st-de"],
      "versionNumber": "1.5.8.NEXT",
      "dependencies": [
        {
          "package": "Tilkee@2.65.0.3"
        },
        {
          "package": "Asperato ONE@2.19.0.8"
        },
        {
          "package": "Nebula Logger@v4.11.2"
        }
      ]

Expected behavior
during validate, metadata should be deploy and test should be launched

Screenshots
If applicable, add screenshots to help explain your problem.

Platform Details (please complete the following information):

  • OS: docker ghcr.io/dxatscale/sfpowerscripts:24.5.0-6246479439
  • Version sfp -- The DX@Scale CLI -Version:24.5.0 -Release:August 23
  • Salesforce CLI(sfdx cli) Version:
  • CI Platform:

Additional context
complete log:

------------------------------------Installing Package------------------------------------
Name: be-legacy
Type: source
Version Number: 1.5.8.1
Optimized Deployment: false
Contains Apex Classes/Triggers: true
Metadata to be deployed: 4723
------------------------------------------------------------------------------------------
Installing package dependencies of this be-legacy in test-idouyvt4fj7h@example.com


Resolved external package dependencies:
                                                                   
 Order   Package                 Version   Subscriber Version Id 
 0       Tilkee@2.65.0.3         N/A       04t3Z000000LdW6QAK    
 1       Asperato ONE@2.19.0.8   N/A       04t1n000001pI4AAAU    
 2       Nebula Logger@v4.11.2   N/A       04t5Y000001TsZAQA0    
                                                                   



Checking whether package  Tilkee@2.65.0.3 with ID 04t3Z000000LdW6QAKis installed in test-idouyvt4fj7h@example.com
Package to be installed was found in the target org test-idouyvt4fj7h@example.com
Skipping Installing of package Tilkee@2.65.0.3 in test-idouyvt4fj7h@example.com

Checking whether package  Asperato ONE@2.19.0.8 with ID 04t1n000001pI4AAAUis installed in test-idouyvt4fj7h@example.com
Package to be installed was found in the target org test-idouyvt4fj7h@example.com
Skipping Installing of package Asperato ONE@2.19.0.8 in test-idouyvt4fj7h@example.com

Checking whether package  Nebula Logger@v4.11.2 with ID 04t5Y000001TsZAQA0is installed in test-idouyvt4fj7h@example.com
Package to be installed was found in the target org test-idouyvt4fj7h@example.com
Skipping Installing of package Nebula Logger@v4.11.2 in test-idouyvt4fj7h@example.com


Successfully completed external dependencies of this be-legacy in test-idouyvt4fj7h@example.com
Checking status of permission sets group..
Proceeding with deployment, as no PermissionSetGroups are being updated
Executing preDeployment script
Executing command.. sh -e .sfpowerscripts/unzippedArtifacts/nSiRQ7Xf/be-legacy_sfpowerscripts_artifact/source/scripts/preDeployment be-legacy test-idouyvt4fj7h@example.com test-idouyvt4fj7h@example.com .sfpowerscripts/unzippedArtifacts/nSiRQ7Xf/be-legacy_sfpowerscripts_artifact/source ./src/be-legacy
Executing Pre Deployers
Executing Field History Tracking Analyzer
Executing Feed Tracking Analyzer
Executing Picklist Analyzer
Pre Deployer Picklist Enabler skipped or not enabled
Attempt to delete components mentioned in destructive manifest
Deploying Destructive Changes with ID 0Af7Y00002BTFpxSAH to test-idouyvt4fj7h@example.com
Polling for Deployment Status
Polling for Deployment Status
We attempted a deletion of components, However we are not successful.             Either the components are already deleted or there are components which             have dependency to components in the manifest.             Please check whether this manifest works!             Acutal Error Observed:             --------------------------------------             Unable to deploy the Destructive Changes: [...]
The following metadata will be deployed:
                                                                                                                                      
[...]                                                                            
                                                                                                                                      
Deploying to test-idouyvt4fj7h@example.com with id:0Af7Y00002BTFs8SAH
Status: Pending Progress: 0/0
Status: InProgress Progress: 700/4432
Status: InProgress Progress: 700/4432
Status: InProgress Progress: 701/4432
Status: InProgress Progress: 1799/4432
Status: InProgress Progress: 3009/4432
Status: InProgress Progress: 3130/4432
Status: InProgress Progress: 3457/4432
Status: InProgress Progress: 3576/4432
Status: InProgress Progress: 3659/4432
Status: InProgress Progress: 3659/4432
Status: InProgress Progress: 3659/4432
Status: InProgress Progress: 4203/4432
Status: InProgress Progress: 4264/4432
Status: InProgress Progress: 4264/4432
Status: InProgress Progress: 4424/4432
Succesfully Deployed 4386 components in 00:08:15.590
Executing Post Deployers
Post Deployer Field History Tracking Enabler skipped or not enabled
Post Deployer Feed Tracking Enabler skipped or not enabled
Existing artifact record id for be-legacy in Org for 1.5.8.1: [...]
Updated Org with new Artifact be-legacy 1.5.8.1 [...]
Package be-legacy installation took 00:08:50.779
azlam-abdulsalam commented

@alanjaouen Its is an expected behaviour, tests are run after deployment in validate, Isnt if running in that mod

alanjaouen commented

@azlam-abdulsalam the validate end with success right after the deployment, without running tests for this package

previously (in 21.1.0) for source package with isOptimizedDeployment to false, tests where run during deployment

  • 👍1
alanjaouen commented

I tried to find what changed between 21.1.0 related and current, but didn't find anything relevant :/
https://github.com/dxatscale/sfpowerscripts/compare/@dxatscale/sfpowerscripts@21.1.0...@dxatscale/sfpowerscripts@25.0.6

nabondance commented

After deep diving in the code, I found something:
This is the line specifying to not run the apex tests when optimized deployment is false

sfpowerscripts/packages/sfpowerscripts-cli/src/impl/validate/ApexTestValidator.ts

Lines 38 to 43 in ab1a369

if (this.sfpPackage.packageDescriptor.isOptimizedDeployment == false)
return {
id: null,
result: true,
message: "Tests would have already run",
};

But the tests doesn't run previously because the only moment they can run is when the package is deployed
In the DeployImpl, it specify to not skip the tests during a validate of a source package without optimized deployment:

sfpowerscripts/packages/sfpowerscripts-cli/src/impl/deploy/DeployImpl.ts

Lines 747 to 762 in ab1a369

//During validate, if optimizeDeploymentMode is false, use full local tests to validate
//but respect skipTesting #issue 1075
//During Prepare (push), dont trigger tests
if (this.props.currentStage == Stage.VALIDATE) {
//Always enable skipTest as the default installation option during validate
//as test are run subsequently
installationOptions.skipTesting = true;
if (!this.isOptimizedDeploymentForSourcePackage(pkgDescriptor)) {
if (sfpPackage.packageDescriptor.skipTesting)
installationOptions.skipTesting = sfpPackage.packageDescriptor.skipTesting;
else installationOptions.skipTesting = false;
}
} else if (this.props.currentStage === Stage.PREPARE) {
installationOptions.optimizeDeployment = false;
installationOptions.skipTesting = true;
}

Then it regenerate of the settings of the deploy here:

sfpowerscripts/packages/core/src/package/packageInstallers/InstallSourcePackageImpl.ts

Lines 70 to 77 in ab1a369

//Construct Deploy Command for actual payload
deploymentOptions = await this.generateDeploymentOptions(
this.options.waitTime,
this.options.optimizeDeployment,
this.options.skipTesting,
this.sfpOrg.getUsername(),
this.options.apiVersion
);

And then, when setting up the deploy, it specify that there is no tests to run in the case of a source package deployed on a Scratch Org for validation:
We go in this else because this.options.isInstallingForValidation == true :

sfpowerscripts/packages/core/src/package/packageInstallers/InstallPackage.ts

Lines 471 to 478 in ab1a369

} else {
if (orgDetails.isSandbox) {
deploymentOptions.testLevel = TestLevel.RunNoTests;
} else {
deploymentOptions.testLevel = TestLevel.RunSpecifiedTests;
deploymentOptions.specifiedTests = 'skip';
}
}

And because we deploy in SO, we have orgDetails.isSandbox == true

This is the explanation of the issue I was able to find
I can't understand why it was working previously

Possible ways to fix:

  1. Remove the if (this.sfpPackage.packageDescriptor.isOptimizedDeployment == false) , probably not the best solution
  2. In DeployImpl.installPackage : specify the testLevel and pass it through all the way down
  3. In InstallPackage.generateDeploymentOptions : add a new if for this specific usecase
  • 👀1
azlam-abdulsalam commented

wow! great investigation @nabondance . Very helpful! Let me have a look into it