undefined method `value' for nil:NilClass
Opened this issue · 14 comments
New Issue Checklist
- Updated
fastlane-plugin-test_centerto the latest version - I read the README.md
- I reviewed the example(s) for the action(s) I am using
- I have removed any sensitive data such as passwords, authentication tokens, or anything else I do not want to world to see
- I have reviewed the Discussions forum to see my question has already been addressed.
Issue Description
multi_scan crashes when it encounters a junit file that does not include a "name" in its root xml object. I don't know why such a junit file was generated in the first place, I have attached it to this document. So multi_scan crashes when collating the tests. My guess is there are two things to do: 1) figure out why a "wrong" junit is generated in the first place and 2) make collate_junit_reports.rb more robust to ignore such junits in case they do occur.
Complete output when running fastlane, including the stack trace and command used
I'm not gonna include the full output but the parts I think are relevant, but can provide more if required:
Tail of the output:
[0;36;49m[worker 4] �[0mDEBUG [2021-05-15 11:16:30.23]: ReportCollator collating
�[0;36;49m[worker 4] �[0mDEBUG [2021-05-15 11:16:30.23]: All tests passed for batch 4? true
DEBUG [2021-05-15 11:19:13.01]: Collating results for all batches
DEBUG [2021-05-15 11:19:13.01]: ReportCollator collating
DEBUG [2021-05-15 11:19:13.01]: Collating junit report files ["/Volumes/extra/jenkins/workspace/re_IOS-3692-testing-improvements/fastlane/test_output/BikemapTests-batch-2/report.junit", "/Volumes/extra/jenkins/workspace/re_IOS-3692-testing-improvements/fastlane/test_output/BikemapTests-batch-4/report.junit", "/Volumes/extra/jenkins/workspace/re_IOS-3692-testing-improvements/fastlane/test_output/BikemapTests-batch-3/report.junit", "/Volumes/extra/jenkins/workspace/re_IOS-3692-testing-improvements/fastlane/test_output/BikemapTests-batch-1/report.junit"]
DEBUG [2021-05-15 11:19:13.01]: collate_junit_reports with ["/Volumes/extra/jenkins/workspace/re_IOS-3692-testing-improvements/fastlane/test_output/BikemapTests-batch-2/report.junit", "/Volumes/extra/jenkins/workspace/re_IOS-3692-testing-improvements/fastlane/test_output/BikemapTests-batch-4/report.junit", "/Volumes/extra/jenkins/workspace/re_IOS-3692-testing-improvements/fastlane/test_output/BikemapTests-batch-3/report.junit", "/Volumes/extra/jenkins/workspace/re_IOS-3692-testing-improvements/fastlane/test_output/BikemapTests-batch-1/report.junit"]
WARN [2021-05-15 11:19:13.02]: �[0;33;49mLane Context:�[0m
INFO [2021-05-15 11:19:13.02]: {:DEFAULT_PLATFORM=>:ios, :PLATFORM_NAME=>:ios, :LANE_NAME=>"ios sweep"}
INFO [2021-05-15 11:19:13.02]: �[0;32;49mSuccessfully generated documentation at path '/Volumes/extra/jenkins/workspace/re_IOS-3692-testing-improvements/fastlane/README.md'�[0m
+------+----------------------------+-------------+
| �[0;32;49mfastlane summary�[0m |
+------+----------------------------+-------------+
| Step | Action | Time (in s) |
+------+----------------------------+-------------+
| 1 | Verifying fastlane version | 0 |
| 2 | default_platform | 0 |
| 💥 | �[0;31;49mmulti_scan�[0m | 638 |
+------+----------------------------+-------------+
DEBUG [2021-05-15 11:19:13.06]: All plugins are up to date
ERROR [2021-05-15 11:19:13.06]: �[0;31;49mfastlane finished with errors�[0m
Looking for related GitHub issues on fastlane/fastlane...
Search query: undefined method `value' for nil
URL: https://api.github.com/search/issues?q=undefined%20method%20%60value%27%20for%20nil+repo:fastlane/fastlane
➡️ �[0;33;49mfastlane multi_scan: undefined method `length' for nil:NilClass�[0m
https://github.com/fastlane/fastlane/issues/18639 [�[0;31;49mopen�[0m] 0 💬
a week ago
➡️ �[0;33;49mUndefined method `status' for nil:NilClass (NoMethodError)�[0m
https://github.com/fastlane/fastlane/issues/17910 [�[0;31;49mopen�[0m] 33 💬
3 weeks ago
➡️ �[0;33;49mfastlane deliver init fails with undefined method `get_app_info_localizations' for nil:NilClass�[0m
https://github.com/fastlane/fastlane/issues/18565 [�[0;31;49mopen�[0m] 1 💬
3 weeks ago
and 329 more at: https://github.com/fastlane/fastlane/search?q=undefined%20method%20%60value%27%20for%20nil&type=Issues&utf8=✓
🔗 You can ⌘ + double-click on links to open them directly in your browser.
DEBUG [2021-05-15 11:19:14.86]: All plugins are up to date
INFO [2021-05-15 11:19:14.86]: Deleting iPhone 8 Clone 1 TestCenter::Helper::MultiScanManager::SimulatorHelper<70244558031000>
INFO [2021-05-15 11:19:14.97]: Deleting iPhone 8 Clone 2 TestCenter::Helper::MultiScanManager::SimulatorHelper<70244558031000>
INFO [2021-05-15 11:19:15.08]: Deleting iPhone 8 Clone 3 TestCenter::Helper::MultiScanManager::SimulatorHelper<70244558031000>
INFO [2021-05-15 11:19:15.19]: Deleting iPhone 8 Clone 4 TestCenter::Helper::MultiScanManager::SimulatorHelper<70244558031000>
INFO [2021-05-15 11:19:15.30]: Shutting down ED6B82AD-C9F5-4EBC-9AEA-94D98BBB17C7
/Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/actions/collate_junit_reports.rb:13:in block in run': �[0;31;49m[!] undefined method value' for nil:NilClass�[0m (NoMethodError)
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/actions/collate_junit_reports.rb:13:in map' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/actions/collate_junit_reports.rb:13:in run'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/report_collator.rb:66:in collate_junit_reports' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/report_collator.rb:28:in collate'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:395:in collate_batched_reports_for_testable' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:281:in block in collate_batched_reports'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:280:in each' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:280:in collate_batched_reports'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:252:in run_test_batches' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:110:in block in run'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:137:in block in setup_run_tests_for_each_device' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:126:in each'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:126:in each_with_index' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:126:in setup_run_tests_for_each_device'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:108:in run' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/actions/multi_scan.rb:38:in run'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/runner.rb:263:in block (2 levels) in execute_action' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/actions/actions_helper.rb:69:in execute_action'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/runner.rb:255:in block in execute_action' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/runner.rb:229:in chdir'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/runner.rb:229:in execute_action' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/runner.rb:157:in trigger_action_by_name'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/fast_file.rb:159:in method_missing' from Fastfile:156:in block (2 levels) in parsing_binding'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/lane.rb:33:in call' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/runner.rb:49:in block in execute'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/runner.rb:45:in chdir' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/runner.rb:45:in execute'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/lane_manager.rb:47:in cruise_lane' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/command_line_handler.rb:36:in handle'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/commands_generator.rb:109:in block (2 levels) in run' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/commander-4.6.0/lib/commander/command.rb:187:in call'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/commander-4.6.0/lib/commander/command.rb:157:in run' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/commander-4.6.0/lib/commander/runner.rb:444:in run_active_command'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane_core/lib/fastlane_core/ui/fastlane_runner.rb:76:in run!' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/commander-4.6.0/lib/commander/delegates.rb:18:in run!'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/commands_generator.rb:353:in run' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/commands_generator.rb:42:in start'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/cli_tools_distributor.rb:122:in take_off' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/bin/fastlane:23:in <top (required)>'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/bin/fastlane:23:in load' from /Users/bikemap/.rvm/gems/ruby-2.6.5/bin/fastlane:23:in
'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/bin/ruby_executable_hooks:24:in eval' from /Users/bikemap/.rvm/gems/ruby-2.6.5/bin/ruby_executable_hooks:24:in '
/Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/actions/collate_junit_reports.rb:13:in block in run': undefined method value' for nil:NilClass (NoMethodError)
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/actions/collate_junit_reports.rb:13:in map' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/actions/collate_junit_reports.rb:13:in run'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/report_collator.rb:66:in collate_junit_reports' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/report_collator.rb:28:in collate'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:395:in collate_batched_reports_for_testable' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:281:in block in collate_batched_reports'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:280:in each' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:280:in collate_batched_reports'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:252:in run_test_batches' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:110:in block in run'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:137:in block in setup_run_tests_for_each_device' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:126:in each'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:126:in each_with_index' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:126:in setup_run_tests_for_each_device'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/helper/multi_scan_manager/runner.rb:108:in run' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-plugin-test_center-3.15.3/lib/fastlane/plugin/test_center/actions/multi_scan.rb:38:in run'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/runner.rb:263:in block (2 levels) in execute_action' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/actions/actions_helper.rb:69:in execute_action'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/runner.rb:255:in block in execute_action' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/runner.rb:229:in chdir'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/runner.rb:229:in execute_action' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/runner.rb:157:in trigger_action_by_name'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/fast_file.rb:159:in method_missing' from Fastfile:156:in block (2 levels) in parsing_binding'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/lane.rb:33:in call' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/runner.rb:49:in block in execute'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/runner.rb:45:in chdir' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/runner.rb:45:in execute'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/lane_manager.rb:47:in cruise_lane' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/command_line_handler.rb:36:in handle'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/commands_generator.rb:109:in block (2 levels) in run' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/commander-4.6.0/lib/commander/command.rb:187:in call'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/commander-4.6.0/lib/commander/command.rb:157:in run' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/commander-4.6.0/lib/commander/runner.rb:444:in run_active_command'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane_core/lib/fastlane_core/ui/fastlane_runner.rb:76:in run!' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/commander-4.6.0/lib/commander/delegates.rb:18:in run!'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/commands_generator.rb:353:in run' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/commands_generator.rb:42:in start'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/fastlane/lib/fastlane/cli_tools_distributor.rb:122:in take_off' from /Users/bikemap/.rvm/gems/ruby-2.6.5/gems/fastlane-2.183.0/bin/fastlane:23:in <top (required)>'
from /Users/bikemap/.rvm/gems/ruby-2.6.5/bin/fastlane:23:in load' from /Users/bikemap/.rvm/gems/ruby-2.6.5/bin/fastlane:23:in '
from /Users/bikemap/.rvm/gems/ruby-2.6.5/bin/ruby_executable_hooks:24:in eval' from /Users/bikemap/.rvm/gems/ruby-2.6.5/bin/ruby_executable_hooks:24:in '
multi_scan settings:
+-------------------------+--------------------------------------------------+ | �[0;32;49mSummary for multi_scan (test_center v3.15.3)�[0m | +-------------------------+--------------------------------------------------+ | try_count | 3 | | testrun_completed_block | # | | parallel_testrun_count | 4 | +-------------------------+--------------------------------------------------+ERROR [2021-05-15 11:08:34.86]: �[1;34;49mUsing deprecated option: '--custom_report_file_name' (Use
--output_filesinstead)�[0m
WARN [2021-05-15 11:08:34.87]: �[0;33;49mScanfile found: overriding multi_scan options with it's values.�[0m
DEBUG [2021-05-15 11:08:34.87]: Building the project in preparation for multi_scan testing
ERROR [2021-05-15 11:08:34.88]: �[1;34;49mUsing deprecated option: '--custom_report_file_name' (Use--output_filesinstead)�[0m
INFO [2021-05-15 11:08:34.89]: �[0;32;49mSuccessfully loaded '/Volumes/extra/jenkins/workspace/re_IOS-3692-testing-improvements/fastlane/Scanfile' 📄�[0m+-------------------+------------------------+
| Detected Values from './fastlane/Scanfile' |
+-------------------+------------------------+
| scheme | Bikemap |
| open_report | false |
| devices | ["iPhone 8"] |
| clean | false |
| derived_data_path | build/DerivedData |
| output_types | junit |
| xcargs | -UseNewBuildSystem=YES |
+-------------------+------------------------+
How multi_scan is called:
lane :sweep do
test_run_block = lambda do |testrun_info|
sh("set-simulator-location -c 51.510279 -0.138947")
failed_test_count = testrun_info[:failed].size
if failed_test_count > 0
UI.important('The run of tests would finish with failures due to fragile tests here.')
try_attempt = testrun_info[:try_count]
if try_attempt < 3
UI.header("Using :multi_scan - rerunning failing tests #{try_attempt}... ")
end
end
end
result = multi_scan(
workspace: MY_XCODE_WORKSPACE_FILEPATH,
try_count: 3,
fail_build: true,
scheme: 'Bikemap',
testrun_completed_block: test_run_block,
parallel_testrun_count: 4,
)
unless result[:failed_testcount].zero?
UI.message("There are #{result[:failed_testcount]} legitimate failing tests")
end
end
The report.junit the plugin is choking on (note the other 3 junit files look ok, and this does not happen on every run, sometimes all junit files are ok):
<?xml version='1.0' encoding='UTF-8'?>
<testsuites tests='0' failures='0'/>
🚫 fastlane environment 🚫
Stack
Key Value OS 11.3 Ruby 2.6.5 Bundler? true Git git version 2.28.0 Installation Source ~/.rvm/gems/ruby-2.6.5/bin/fastlane Host macOS 11.3 (20E232) Ruby Lib Dir ~/.rvm/rubies/ruby-2.6.5/lib OpenSSL Version OpenSSL 1.1.1d 10 Sep 2019 Is contained false Is homebrew false Is installed via Fabric.app false Xcode Path /Applications/Xcode.app/Contents/Developer/ Xcode Version 12.5 System Locale
Error No Locale with UTF8 found 🚫 fastlane gems
Gem Version Update-Status fastlane 2.183.0 ✅ Up-To-Date Loaded fastlane plugins:
Plugin Version Update-Status fastlane-plugin-test_center 3.15.3 ✅ Up-To-Date Loaded gems
Gem Version did_you_mean 1.4.0 rake 13.0.3 CFPropertyList 3.0.3 public_suffix 4.0.6 addressable 2.7.0 artifactory 3.0.15 atomos 0.1.3 aws-eventstream 1.1.1 aws-partitions 1.456.0 aws-sigv4 1.2.3 jmespath 1.4.0 aws-sdk-core 3.114.0 aws-sdk-kms 1.43.0 aws-sdk-s3 1.94.1 babosa 1.0.4 bundler 2.1.4 claide 1.0.3 colored 1.2 colored2 3.1.2 colorize 0.8.1 highline 2.0.3 commander 4.6.0 declarative 0.0.20 digest-crc 0.6.3 unf_ext 0.0.7.7 unf 0.1.4 domain_name 0.5.20190701 dotenv 2.7.6 emoji_regex 3.2.2 excon 0.81.0 faraday-excon 1.1.0 faraday-net_http 1.0.1 faraday-net_http_persistent 1.1.0 multipart-post 2.0.0 ruby2_keywords 0.0.4 faraday 1.4.1 http-cookie 1.0.3 faraday-cookie_jar 0.0.7 faraday_middleware 1.0.0 fastimage 2.2.3 gh_inspector 1.1.3 jwt 2.2.3 memoist 0.16.2 multi_json 1.15.0 os 1.1.1 signet 0.15.0 googleauth 0.16.2 httpclient 2.8.3 mini_mime 1.1.0 trailblazer-option 0.1.1 uber 0.1.0 representable 3.1.1 retriable 3.1.2 rexml 3.2.5 webrick 1.7.0 google-apis-core 0.3.0 google-apis-androidpublisher_v3 0.2.0 google-apis-playcustomapp_v1 0.2.0 google-apis-iamcredentials_v1 0.3.0 google-apis-storage_v1 0.3.0 google-cloud-env 1.5.0 google-cloud-errors 1.1.0 google-cloud-core 1.6.0 google-cloud-storage 1.31.0 json 2.5.1 mini_magick 4.11.0 naturally 2.2.1 plist 3.6.0 rubyzip 2.3.0 security 0.1.3 simctl 1.6.8 terminal-notifier 2.0.0 unicode-display_width 1.7.0 terminal-table 1.8.0 tty-screen 0.8.1 tty-cursor 0.7.1 tty-spinner 0.9.3 word_wrap 1.0.0 nanaimo 0.3.0 xcodeproj 1.19.0 rouge 2.0.7 xcpretty 0.3.0 xcpretty-travis-formatter 1.0.1 trainer 0.9.1 xctest_list 1.2.1 fastlane-plugin-test_center 3.15.3 generated on: 2021-05-15
Hi @hactar; I won't be able to guess why this bad junit is being made. Have you reviewed it? Does it look valid? I see by the output that you did provide that you used the --verbose flag. I add a lot of output around junits and HTML reporting, do you see any clues there?
Are there any crashes in the test runner that you can see? Usually, full logs have more information before the problem that help me figure out what is going on, or at least let me add more logging around areas that I suspect are the culprit.
So, two things:
- Can you provide the full logs? If you're worried about advertising sensitive data to the world, I the new issue does provide a way for you to encrypt the logs.
- If you cannot provide the logs, give me a walkthrough of anything that you see in the logs before the problem: are there crashes in a test run? Is the junit valid? Are there any odd log outputs that indicate that bad data would be written to the logs?
Hey @lyndsey-ferguson, thanks, a few updates: I posted the junit already (BikemapTests-batch-3. The others, BikemapTests-batch-1, BikemapTests-batch-2 and BikemapTests-batch-4 look normal)
<?xml version='1.0' encoding='UTF-8'?>
<testsuites tests='0' failures='0'/>
Thats it, it does not contain anything other than this. I've gone deeper into the logs, and yes it appears that a test runner crashed during the run:
I took a look at Test-Transient Testing-2021.05.15_11-13-48-+0200.xcresult. but it does not appear to contain anything useful, see screenshot. The build log file that it is referencing does not exist.
Here's the full log, encrypted:
build-multi-scan-enc.zip
Would a good workaround be adding something to collate_junit_reports.rb above line 13 which filters out xml files where the root object did not contain a .name value?
@hactar that would be a workaround, but I wonder why the file doesn't exist?
I have to move this to the backlog again, personal life has gotten too busy to try and pick this up.
So just to be clear, the file does exist, its just malformed: its not a full report, but only contains two lines because the test runner crashed:
<?xml version='1.0' encoding='UTF-8'?>
<testsuites tests='0' failures='0'/>
I tried the work around, but its not complete, because the tests that were supposed to run in that testrunner are never run or rerun, so multi_scan completes with the workaround, but because only 38 of the 47 tests or so where run, it marks the result as a failure.
We have found a different workaround, we have removed parallel_testrun_count from our multi_scan settings. By turning off the parallel test run we no longer have crashing test runners, and therefore the tests get run and rerun reliably - at the cost of parallelism.
Sure, it is malformed, but we don't know why. That's what concerns me.
multi_scan uses the results of those junit files to determine which tests passed so it doesn't have to run them again. Without a valid junit, the program doesn't know.
Thanks for the note about the parallelism. There were problems when running tests in parallel and extra params were sent via a Scanfile that broke things: I believe I fixed them, but make sure that you're not using a Scanfile, or it is relatively empty to test if that is the problem.
@hactar I'm trying to decrypt the log files that you sent me. When I run the command to decrypt the key that you encrypted with my public key, I am using this command:
openssl rsautl -decrypt -ssl -inkey ~/.ssh/id_rsa -in secret.txt.key.enc -out secret.txt.key
When I try to read the secret.txt.key, my computer tells me that it is a binary key (which it shouldn't).
Did you encrypt that key with my public key like this?
# download https://github.com/lyndsey-ferguson/fastlane-plugin-test_center/files/5577804/lyndsey-ferguson-id_rsa.pub.pkcs8.zip -> lyndsey-ferguson-id_rsa.pub
openssl rsautl -encrypt -pubin -inkey <my public key> -in secret.txt.key. -out secret.txt.key.enc
And if so, did you create the secret.txt.key and encrypt the log file with these commands?
The exact command I ran was this:
openssl rsautl -encrypt -pubin -inkey lyndsey-ferguson-id_rsa.pub.pkcs8 -in secret.txt.key -out secret.txt.key.enc
I created the key using openssl aes-256-cbc -in build-multi-scan.txt -out build-multi-scan.txt.enc -pass file:secret.txt.key
Thanks for the tip with the Scanfile, we do still have a scanfile around, will see if removing it changes anything (the disabling of parallel testing is holding up though, the issue has not occured since)
Isn't it just the one generated by running build-without-testing? I'm having the same issue and wasn't able to resolve it by setting parallel test count to 1.
@hactar, I have been working on making multi_scan use xcresult files instead of junit files, if you're interested in testing (alpha quality), follow the issue for the feature request
@lyndsey-ferguson Cool thanks! we're busy with our iOS 15 release but will test it once we find a moment.
same issue here, trying the workaround
multi_scan(workspace: ENV["WORKSPACE"],
devices: [DEFAULT_SIMULATOR],
scheme: ENV["UI_TEST_SCHEME"],
skip_slack: true,
result_bundle: true,
include_simulator_logs: false, # does not work in lab (causes Error 74)
output_types: "junit",
try_count: 5,
# parallel_testrun_count: 3, /* DISABLING AS A WORK AROUND TO THIS ISSUE https://github.com/lyndsey-ferguson/fastlane-plugin-test_center/issues/339 */
collate_reports: true)
I want to apologize that I have not responded to this issue. A family health issue is requiring me to narrow my focus on the essentials and I don't have the time I need to focus on the plugin more than a minute. If you can get some other contributors to this project to help with this, I would consider merging a resulting PR.