Job run succeeds when Person record is not inserted - bug?

[aleksa-krolls]

Overview

The job MOH513-Enroll-Household-in-SF.js upserts Households and Persons in Salesforce when a CHW registers new households. However, the client has reported that sometimes Person records are missing from Salesforce even though they are present in CommCare.

Current Behavior

MOH513-Enroll-Household-in-SF.js job runs succeed and show green checks even when not all operations in the job are successfull.
See run 6879421 as an example where OpenFn marks the run as successful, but the last Person (see L270 in run log) in the run log was not successfully upload to Salesforce (no result returned in the log).

Upserting Person__c with externalId CommCare_ID__c : { CommCare_ID__c: 'd215cbb0-531a-45a7-adf5-4361536e2207',
  CommCare_HH_Code__c: '8d05b920-cff4-4b6a-a155-aaa1c19614c5',
  RecordType: { Name: 'Youth' },
  Name: 'BXXXXXXXXXX',

Request

Please troubleshoot and make the appropriate changes so that:

1. OpenFn will not return a false "success" message when one of the job operations failed
2. Benny's Person__c record is inserted correctly

Resources

state.json

Test with Lwala Salesforce sandbox. See LP for creds.
Use sample_data/state_personB.json to test the above scenario.

adaptor

Use language-salesforce and make use of this LP's FakeAdaptor for offline testing.

expression.js

Related to job MOH513-Enroll-Household-in-SF.js

Toggl

Log your time to Lwala Cleanup 2020

[lakhassane]

@aleksa-krolls There is an invalid value in field Disability_c.

I don't have access to the salesforce platform but my guess is there is an invalid value in a picklist for this field.

In the sample_data/state_personB.json Benny's Person__c is the only one with 2 values for that field.

[aleksa-krolls]

@lakhassane Nice work on the logs! To resolve this mapping issue, can you please add a transformation to the job that returns null if CommCare disability=='none'?

[aleksa-krolls]

@lakhassane I deployed the disability change, but what did you implement in your logs to troubleshoot this issue? Can you add enhanced logging to this job? I want to avoid a future scenario where this happens again (the run succeeds when actually one of the operations fails... if this happens, it should fail in OpenFn).

Ideally we should implement this enhanced logging across all jobs (which is why I was wondering if this should be a change to language-salesforce). But please share what you did and we can evaluate from there! cc @taylordowns2000

[aleksa-krolls]

@lakhassane if this will turn into another task re: the LP, feel free to close this issue and link to a new one.

[lakhassane]

@taylordowns2000 what's your call on this? Should we open one about refactoring language-salesforce so it catches error, with a callback error.

So far it only has callbacks success

[taylordowns2000]

Please close this if it's delivered and open an issue on the salesforce adaptor to discuss over there! Will need to take a look, but is it that there's no "reject(...)", or that errors are somehow actually being swallowed? (Please copy this question over to the salesforce issue.)

…

On Thu, Sep 10, 2020 at 5:34 PM lakhassane ***@***.***> wrote: @taylordowns2000 <https://github.com/taylordowns2000> what's your call on this? Should we open one about refactoring language-salesforce so it catches error, with a callback error. So far it only has callbacks success — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#23 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACCUBLNKH7LJFCMS4GZIO2TSFFA75ANCNFSM4RCFPSKA> .

-- Taylor Downs | Head of Product Open Function Group <https://www.openfn.org/> | How Data Integration Makes ICT4D Work <https://youtu.be/GVB9voGp6T4>

[aleksa-krolls]

@lakhassane Can you please analyze these other jobs to see if this combine() --> alterState change needs to be applied as well?

1. MOH513-Enroll-Person-in-SF.js
2. MOH514-Update-Person-in-SF-Production.js