lyndsey-ferguson/fastlane-plugin-test_center

Setting option disable_xcpretty to true crashes multi_scan's retry functionality

Closed this issue · 11 comments

jambudi commented

All tests are executing as per expectation and at the end, We are seeing this error

+----------```
--------------------------------------+---------------------------------------------------------------------------------------+
| Summary for scan 2.153.1 |
+------------------------------------------------+---------------------------------------------------------------------------------------+
| workspace | /.xcworkspace |
| only_testing | ["//"] |
| app_identifier | // |
| scheme | // |
| devices | ["iPhone 11 Pro (13.3)"] |
| disable_concurrent_testing | true |
| disable_xcpretty | true |
| output_types | junit |
| output_files | report.junit |
| code_coverage | false |
| derived_data_path | ./fastlane/DerivedData |
| reinstall_app | true |
| reset_simulator | true |
| fail_build | false |
| xcargs | -quiet |
| buildlog_path | ./fastlane/test_output/smoke_results |
| output_directory | ./fastlane/test_output/smoke_results |
| skip_detect_devices | false |
| force_quit_simulator | false |
| disable_slide_to_type | true |
| clean | false |
| open_report | false |
| should_zip_build_products | false |
| use_clang_report_name | false |
| build_for_testing | true |
| slack_use_webhook_configured_username_and_icon | false |
| slack_username | fastlane |
| slack_icon_url | https://fastlane.tools/assets/img/fastlane_icon.png |
| skip_slack | false |
| slack_only_on_failure | false |
| xcodebuild_command | env NSUnbufferedIO=YES xcodebuild |
| include_simulator_logs | false |
| skip_build | false |
| xcode_path | /Applications/Xcode.app |
+------------------------------------------------+---------------------------------------------------------------------------------------+


DEBUG [2020-08-23 10:56:17.14]: Before building, removing pre-existing xctestrun files: ["./fastlane/DerivedData/Build/Products//****_iphonesimulator13.2-x86_64.xctestrun"]
DEBUG [2020-08-23 10:56:17.14]: Fetching available simulator devices
INFO [2020-08-23 10:56:17.36]: Resetting iPhone 11 Pro
DEBUG [2020-08-23 10:56:17.63]: Fetching available simulator devices
INFO [2020-08-23 10:56:17.86]: Resetting iPhone 11 Pro
DEBUG [2020-08-23 10:56:18.08]: Fetching available simulator devices
INFO [2020-08-23 10:56:18.35]: Disabling 'Slide to Type' iPhone 11 Pro
INFO [2020-08-23 10:56:18.35]: $ /usr/libexec/PlistBuddy -c "Add :KeyboardContinuousPathEnabled bool false" /Users//****//Library/Developer/CoreSimulator/Devices/4620252A-4855-4102-AD4C-327F98256330/data/Library/Preferences/com.apple.keyboard.ContinuousPath.plist >/dev/null 2>&1
DEBUG [2020-08-23 10:56:18.36]: Fetching available simulator devices
INFO [2020-08-23 10:56:18.62]: Disabling 'Slide to Type' iPhone 11 Pro
INFO [2020-08-23 10:56:18.62]: $ /usr/libexec/PlistBuddy -c "Add :KeyboardContinuousPathEnabled bool false" /Users//****//Library/Developer/CoreSimulator/Devices/D1A6E91A-0761-40C6-A806-09D7DBF5AD29/data/Library/Preferences/com.apple.keyboard.ContinuousPath.plist >/dev/null 2>&1
DEBUG [2020-08-23 10:56:18.63]: Uninstalling app '****' from iPhone 11 Pro...
INFO [2020-08-23 10:56:18.63]: Launch Simulator iPhone 11 Pro
INFO [2020-08-23 10:56:18.63]: $ xcrun instruments -w 4620252A-4855-4102-AD4C-327F98256330 &> /dev/null
INFO [2020-08-23 10:56:25.97]: Uninstall application ****
INFO [2020-08-23 10:56:25.97]: $ xcrun simctl uninstall 4620252A-4855-4102-AD4C-327F98256330 **** &> /dev/null
DEBUG [2020-08-23 10:57:08.65]: Uninstalling app '****' from iPhone 11 Pro...
INFO [2020-08-23 10:57:08.65]: Launch Simulator iPhone 11 Pro
INFO [2020-08-23 10:57:08.65]: $ xcrun instruments -w D1A6E91A-0761-40C6-A806-09D7DBF5AD29 &> /dev/null
INFO [2020-08-23 10:57:16.77]: Uninstall application ****
INFO [2020-08-23 10:57:16.77]: $ xcrun simctl uninstall D1A6E91A-0761-40C6-A806-09D7DBF5AD29 **** &> /dev/null
INFO [2020-08-23 10:57:41.75]: $ set -o pipefail && env NSUnbufferedIO=YES xcodebuild -workspace /Users/***.xcworkspace -scheme *** -destination 'platform=iOS Simulator,id=4620252A-4855-4102-AD4C-327F98256330' -destination 'platform=iOS Simulator,id=D1A6E91A-0761-40C6-A806-09D7DBF5AD29' -derivedDataPath './fastlane/DerivedData' -disable-concurrent-testing -enableCodeCoverage NO -quiet -only-testing:*** build-for-testing | tee '***.log'


10:36:37  [worker 3] [22:06:11]: ------------------------------
10:36:37  [worker 3] [22:06:11]: --- Collecting system logs ---
10:36:37  [worker 3] [22:06:11]: ------------------------------
10:36:37  [worker 3] [22:06:11]: $ xcrun simctl spawn D2427E9D-B344-4208-A5F2-32CA90333544 log collect --output /***/fastlane/test_output/smoke_results/***-batch-3/system_logs-iPhone\ 11\ Pro\ Clone\ 3\ TestCenter::Helper::MultiScanManager::SimulatorHelper\<70206711797300\>_iOS_13.3.logarchive 2>/dev/null

10:36:37  [worker 3] [22:06:13]: Error setting value '***/fastlane/test_output/smoke_results/ApplicationUITests-batch-3/report.junit' for option 'junit'

10:36:37  [worker 3] Error: cannot find the junit xml report file '***/fastlane/test_output/smoke_results/ApplicationUITests-batch-3/report.junit'

10:36:37  [worker 3] ["***/vendor/bundle/ruby/2.6.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/config_item.rb:224:in `rescue in valid?'", "/private/var/jenkins/workspace/***

2028/vendor/bundle/ruby/2.6.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/config_item.rb:220:in `valid?'", "/private/var/jenkins/workspace/***/vendor/bundle/ruby/2.6.0/gems/fastlane-

2.153.1/fastlane_core/lib/fastlane_core/configuration/config_item.rb:182:in `verify!'", "/private/var/jenkins/workspace/***vendor/bundle/ruby/2.6.0/gems/fastlane-

2.153.1/fastlane_core/lib/fastlane_core/configuration/configuration.rb:87:in `block in verify_value_exists'", "/private/var/jenkins/workspace/***/vendor/bundle/ruby/2.6.0/gems/fastlane-


**At end getting this error** 

10:36:37  +------------------+-----------------+
10:36:37  |            Lane Context            |
10:36:37  +------------------+-----------------+
10:36:37  | DEFAULT_PLATFORM | ios             |
10:36:37  | PLATFORM_NAME    | ios             |
10:36:37  | LANE_NAME        | ios smoke_tests |
10:36:37  +------------------+-----------------+
10:36:37  [22:06:14]: No such file or directory @ rb_file_s_stat - /private/var/jenkins/workspace/***/fastlane/test_output/smoke_results/ApplicationUITests/.
jambudi commented
  1. error :

10:36:37 [worker 3] [22:06:13]: Error setting value '***/fastlane/test_output/smoke_results/ApplicationUITests-batch-3/report.junit' for option 'junit'

No such file or directory @ rb_file_s_stat - /private/var/jenkins/workspace/***/fastlane/test_output/smoke_results/ApplicationUITests/.

jambudi commented 
12:09:43  [worker 3] [01:40:15]: $ xcrun simctl spawn E046AFCE-FC53-4DA6-8C1C-6C6CF0D69DDD log collect --output /private/var/jenkins/workspace/********/iOS/Apps/********/fastlane/test_output/ApplicationUITests-batch-3/system_logs-iPhone\ 11\ Pro\ Clone\ 3\ TestCenter::Helper::MultiScanManager::SimulatorHelper\<70201303692320\>_iOS_13.3.logarchive 2>/dev/null
12:09:43  [worker 3] [01:40:18]: Error setting value '/private/var/jenkins/workspace/********/iOS/Apps/********/fastlane/test_output/ApplicationUITests-batch-3/report.junit' for option 'junit'
12:09:43  [worker 3] Error: cannot find the junit xml report file '/private/var/jenkins/workspace/********/iOS/Apps/********/fastlane/test_output/ApplicationUITests-batch-3/report.junit'
12:09:43  [worker 3] ["/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/config_item.rb:224:in `rescue in valid?'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/config_item.rb:220:in `valid?'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/config_item.rb:182:in `verify!'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/configuration.rb:87:in `block in verify_value_exists'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/configuration.rb:82:in `each'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/configuration.rb:82:in `verify_value_exists'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/configuration.rb:66:in `initialize'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/configuration.rb:36:in `new'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/configuration.rb:36:in `create'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/retrying_scan_helper.rb:199:in `failure_details'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/retrying_scan_helper.rb:178:in `send_callback_testrun_info'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/retrying_scan_helper.rb:149:in `handle_success'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/retrying_scan_helper.rb:143:in `after_testrun'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/retrying_scan.rb:74:in `run'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/retrying_scan.rb:59:in `run'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/test_batch_worker.rb:18:in `run'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/parallel_test_batch_worker.rb:49:in `block in run'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/parallel_test_batch_worker.rb:45:in `fork'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/parallel_test_batch_worker.rb:45:in `run'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:203:in `block in run_test_batches'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:200:in `each'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:200:in `each_with_index'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:200:in `run_test_batches'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:99:in `run'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/actions/multi_scan.rb:38:in `run'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/runner.rb:263:in `block (2 levels) in execute_action'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/actions/actions_helper.rb:50:in `execute_action'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/runner.rb:255:in `block in execute_action'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/runner.rb:229:in `chdir'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/runner.rb:229:in `execute_action'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/runner.rb:157:in `trigger_action_by_name'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/fast_file.rb:159:in `method_missing'", "Fastfile:33:in `block (2 levels) in parsing_binding'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/lane.rb:33:in `call'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/runner.rb:49:in `block in execute'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/runner.rb:45:in `chdir'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/runner.rb:45:in `execute'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/lane_manager.rb:47:in `cruise_lane'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/command_line_handler.rb:36:in `handle'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/commands_generator.rb:108:in `block (2 levels) in run'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/commander-fastlane-4.4.6/lib/commander/command.rb:178:in `call'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/commander-fastlane-4.4.6/lib/commander/command.rb:153:in `run'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/commander-fastlane-4.4.6/lib/commander/runner.rb:476:in `run_active_command'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/ui/fastlane_runner.rb:76:in `run!'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/commander-fastlane-4.4.6/lib/commander/delegates.rb:15:in `run!'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/commands_generator.rb:352:in `run'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/commands_generator.rb:41:in `start'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/cli_tools_distributor.rb:119:in `take_off'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/bin/fastlane:23:in `<top (required)>'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/bin/fastlane:23:in `load'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/bin/fastlane:23:in `<top (required)>'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/cli/exec.rb:74:in `load'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/cli/exec.rb:74:in `kernel_load'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/cli/exec.rb:28:in `run'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/cli.rb:465:in `exec'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/cli.rb:27:in `dispatch'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/cli.rb:18:in `start'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/exe/bundle:30:in `block in <top (required)>'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/friendly_errors.rb:124:in `with_friendly_errors'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/exe/bundle:22:in `<top (required)>'", "/private/var/jenkins/.rbenv/versions/2.4.2/bin/bundle:23:in `load'", "/private/var/jenkins/.rbenv/versions/2.4.2/bin/bundle:23:in `<main>'"]
12:09:43  [01:40:18]: $ xcrun xcresulttool merge /private/var/jenkins/workspace/********/iOS/Apps/********/fastlane/test_output/ApplicationUITests-batch-2/report.xcresult /private/var/jenkins/workspace/********/iOS/Apps/********/fastlane/test_output/ApplicationUITests-batch-1/report.xcresult /private/var/jenkins/workspace/********/iOS/Apps/********/fastlane/test_output/ApplicationUITests-batch-3/report.xcresult --output-path /var/folders/s2/1j280s4d74zcb06dbqjx7nxw0000gq/T/collated_result_20200823-10840-16ggnmu.xcresult
12:09:43  [01:40:20]: ▸ [v3] Merged to: /var/folders/s2/1j280s4d74zcb06dbqjx7nxw0000gq/T/collated_result_20200823-10840-16ggnmu.xcresult
12:09:43  +------------------+-----------------+
lyndsey-ferguson commented

@jambudi I am not able to determine when this is crashing, nor why. Are you providing all the logs? I believe that you are using the --verbose flag, but I would expect to see more; I would expect to see more logs before the issue, and more of a crash log.

If these are the logs, I need to see how you called multi_scan to see if I can reproduce the issue.

jambudi commented

@lyndsey-ferguson I am providing all logs. Here are things happening after finish tests. It is only happening after running retry logic and merging two results

12:09:43 [worker 3] [01:40:15]: $ xcrun simctl spawn E046AFCE-FC53-4DA6-8C1C-6C6CF0D69DDD log collect --output /private/var/jenkins/workspace//iOS/Apps//fastlane/test_output/ApplicationUITests-batch-3/system_logs-iPhone\ 11\ Pro\ Clone\ 3\ TestCenter::Helper::MultiScanManager::SimulatorHelper<70201303692320>_iOS_13.3.logarchive 2>/dev/null
12:09:43 [worker 3] [01:40:18]: Error setting value '/private/var/jenkins/workspace//iOS/Apps//fastlane/test_output/ApplicationUITests-batch-3/report.junit' for option 'junit'
12:09:43 [worker 3] Error: cannot find the junit xml report file '/private/var/jenkins/workspace//iOS/Apps//fastlane/test_output/ApplicationUITests-batch-3/report.junit'
12:09:43 [worker 3] ["/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/config_item.rb:224:in rescue in valid?'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/config_item.rb:220:in valid?'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/config_item.rb:182:in verify!'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/configuration.rb:87:in block in verify_value_exists'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/configuration.rb:82:in each'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/configuration.rb:82:in verify_value_exists'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/configuration.rb:66:in initialize'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/configuration.rb:36:in new'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/configuration/configuration.rb:36:in create'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/retrying_scan_helper.rb:199:in failure_details'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/retrying_scan_helper.rb:178:in send_callback_testrun_info'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/retrying_scan_helper.rb:149:in handle_success'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/retrying_scan_helper.rb:143:in after_testrun'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/retrying_scan.rb:74:in run'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/retrying_scan.rb:59:in run'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/test_batch_worker.rb:18:in run'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/parallel_test_batch_worker.rb:49:in block in run'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/parallel_test_batch_worker.rb:45:in fork'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/parallel_test_batch_worker.rb:45:in run'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:203:in block in run_test_batches'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:200:in each'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:200:in each_with_index'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:200:in run_test_batches'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:99:in run'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-plugin-test_center-3.14.1/lib/fastlane/plugin/test_center/actions/multi_scan.rb:38:in run'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/runner.rb:263:in block (2 levels) in execute_action'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/actions/actions_helper.rb:50:in execute_action'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/runner.rb:255:in block in execute_action'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/runner.rb:229:in chdir'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/runner.rb:229:in execute_action'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/runner.rb:157:in trigger_action_by_name'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/fast_file.rb:159:in method_missing'", "Fastfile:33:in block (2 levels) in parsing_binding'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/lane.rb:33:in call'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/runner.rb:49:in block in execute'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/runner.rb:45:in chdir'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/runner.rb:45:in execute'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/lane_manager.rb:47:in cruise_lane'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/command_line_handler.rb:36:in handle'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/commands_generator.rb:108:in block (2 levels) in run'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/commander-fastlane-4.4.6/lib/commander/command.rb:178:in call'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/commander-fastlane-4.4.6/lib/commander/command.rb:153:in run'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/commander-fastlane-4.4.6/lib/commander/runner.rb:476:in run_active_command'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane_core/lib/fastlane_core/ui/fastlane_runner.rb:76:in run!'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/commander-fastlane-4.4.6/lib/commander/delegates.rb:15:in run!'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/commands_generator.rb:352:in run'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/commands_generator.rb:41:in start'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/fastlane/lib/fastlane/cli_tools_distributor.rb:119:in take_off'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/gems/fastlane-2.153.1/bin/fastlane:23:in <top (required)>'", "/private/var/jenkins/workspace/********/vendor/bundle/ruby/2.4.0/bin/fastlane:23:in load'", "/private/var/jenkins/workspace//vendor/bundle/ruby/2.4.0/bin/fastlane:23:in <top (required)>'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/cli/exec.rb:74:in load'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/cli/exec.rb:74:in kernel_load'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/cli/exec.rb:28:in run'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/cli.rb:465:in exec'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/vendor/thor/lib/thor/command.rb:27:in run'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in invoke_command'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/vendor/thor/lib/thor.rb:387:in dispatch'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/cli.rb:27:in dispatch'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/vendor/thor/lib/thor/base.rb:466:in start'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/cli.rb:18:in start'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/exe/bundle:30:in block in <top (required)>'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/lib/bundler/friendly_errors.rb:124:in with_friendly_errors'", "/var/jenkins/.rbenv/versions/2.4.2/lib/ruby/gems/2.4.0/gems/bundler-2.0.2/exe/bundle:22:in <top (required)>'", "/private/var/jenkins/.rbenv/versions/2.4.2/bin/bundle:23:in load'", "/private/var/jenkins/.rbenv/versions/2.4.2/bin/bundle:23:in

'"]
12:09:43 [01:40:18]: $ xcrun xcresulttool merge /private/var/jenkins/workspace//iOS/Apps//fastlane/test_output/ApplicationUITests-batch-2/report.xcresult /private/var/jenkins/workspace//iOS/Apps//fastlane/test_output/ApplicationUITests-batch-1/report.xcresult /private/var/jenkins/workspace//iOS/Apps/********/fastlane/test_output/ApplicationUITests-batch-3/report.xcresult --output-path /var/folders/s2/1j280s4d74zcb06dbqjx7nxw0000gq/T/collated_result_20200823-10840-16ggnmu.xcresult
12:09:43 [01:40:20]: ▸ [v3] Merged to: /var/folders/s2/1j280s4d74zcb06dbqjx7nxw0000gq/T/collated_result_20200823-10840-16ggnmu.xcresult
12:09:43 +------------------+-----------------+

jambudi commented 
Here is multiscan 

	multi_scan(
			workspace: File.absolute_path("../****.xcworkspace"),
			only_testing: ["ApplicationUITests"],
			app_identifier: "****",
			scheme: "*****",
			devices: ["iPhone 11 Pro (13.3)"],
			try_count: 3, # retry _failing_ tests up to three times.
		        testrun_completed_block: test_run_block,
			include_simulator_logs: true,
			disable_concurrent_testing: true,
			disable_xcpretty: true,
			output_types: "junit",
  			output_files: "report.junit",
			code_coverage: false,
			derived_data_path: "./fastlane/DerivedData",
			reinstall_app: true,
			reset_simulator: true,
			parallel_testrun_count: 3,
			fail_build: false,
			result_bundle: true,
	  		xcargs: "-quiet",
			buildlog_path: "./fastlane/test_output/smoke_results",
		    output_directory: "./fastlane/test_output/smoke_results"
		)
lyndsey-ferguson commented

Can you change disable_xcpretty: true to disable_xcpretty: false? multi_scan relies on junits to know what has passed or not.

  • 👍1
jambudi commented

After disable_xcpretty: false, Jenkins's console output is not formatted. Do you have any suggestions on how to print console output formatted?

lyndsey-ferguson commented

@jambudi can you elaborate? I never use that flag and the Jenkins console output seems normal (?). What are the differences that you see?

jambudi commented

@lyndsey-ferguson I think everything looks good. After changing from true to false for disable_xcpretty: false. All test results are merged and working as expected. thank you

lyndsey-ferguson commented

Great, thank you! I will change this Issue title so that I prevent that from happening in the action in itself