multi_scan is timing out with no error logs
Closed this issue · 5 comments
New Issue Checklist
- Updated
fastlane-plugin-test_centerto the latest version - I read the README.md
- I reviewed the example(s) for the action(s) I am using
- I have removed any sensitive data such as passwords, authentication tokens, or anything else I do not want to world to see
Issue Description
The question is how can I debug this effectively to determine if it's an issue with this library, a problem with CircleCI, or if the problem lies within the UITests? We don't get any data beyond what I've provided below.
Running multi_scan on CircleCI & multi_scan is stalling. It gets stuck here and never progresses:
INFO [2020-03-25 21:51:34.86]: ▸ Loading... INFO [2020-03-25 21:51:36.99]: ▸ 2020-03-25 21:51:36.996 xcodebuild[790:7041] IDETestOperationsObserverDebug: Writing diagnostic log for test session to: INFO [2020-03-25 21:51:36.99]: ▸ /Users/distiller/project/build/DerivedData/test_output/g47/CompanyUITests-batch-1/Company-Dev.test_result/Staging/1_Test/Diagnostics/CompanyUITests-7C6C0BFE-B1CE-4FF2-A238-467E17A59945/CompanyUITests-5EBDFE08-8DFC-4CED-9AE3-7B5DDB5D300A/Session-CompanyUITests-2020-03-25_215136-do8fuT.log INFO [2020-03-25 21:51:36.99]: ▸ 2020-03-25 21:51:36.997 xcodebuild[790:6836] [MT] IDETestOperationsObserverDebug: (D03636BC-799B-4AC8-AAD2-949A32D676F5) Beginning test session CompanyUITests-D03636BC-799B-4AC8-AAD2-949A32D676F5 at 2020-03-25 21:51:36.997 with Xcode 11B500 on target <DVTiPhoneSimulator: 0x7ff7102df2f0> { INFO [2020-03-25 21:51:36.99]: ▸ SimDevice: iPhone 8 (F3026518-DE25-4715-B0E4-47C2C1A2A640, iOS 13.2, Shutdown) INFO [2020-03-25 21:51:36.99]: ▸ } (13.2.2 (17B102)) INFO [2020-03-25 21:51:51.94]: ▸ 2020-03-25 21:51:51.947 xcodebuild[790:6836] [MT] IDETestOperationsObserverDebug: (D03636BC-799B-4AC8-AAD2-949A32D676F5) Finished requesting crash reports. Continuing with testing.
Complete output when running fastlane, including the stack trace and command used
Here's everything leading up to that point:
DEBUG [2020-03-25 21:51:13.00]: Checking if there are any plugins that should be loaded...
DEBUG [2020-03-25 21:51:13.00]: Loading 'fastlane-plugin-test_center' plugin
INFO [2020-03-25 21:51:13.01]: gem 'fastlane-plugin-test_center' is already installed
DEBUG [2020-03-25 21:51:17.16]: Loading 'fastlane-plugin-badge' plugin
INFO [2020-03-25 21:51:17.16]: gem 'fastlane-plugin-badge' is already installed
+-----------------------------+---------+-----------------------------+
| Used plugins |
+-----------------------------+---------+-----------------------------+
| Plugin | Version | Action |
+-----------------------------+---------+-----------------------------+
| fastlane-plugin-test_center | 3.8.15 | suppressed_tests |
| | | suppress_tests |
| | | suppress_tests_from_junit |
| | | tests_from_xctestrun |
| | | quit_core_simulator_service |
| | | collate_test_result_bundles |
| | | tests_from_junit |
| | | multi_scan |
| | | collate_html_reports |
| | | collate_junit_reports |
| | | collate_json_reports |
| fastlane-plugin-badge | 1.4.0 | add_badge |
+-----------------------------+---------+-----------------------------+Successfully loaded Appfile at path '/Users/distiller/project/fastlane/Appfile'
Successfully loaded Appfile at path '/Users/distiller/project/fastlane/Appfile'
INFO [2020-03-25 21:51:17.35]: Sending anonymous analytics information
INFO [2020-03-25 21:51:17.35]: Learn more at https://docs.fastlane.tools/#metrics
INFO [2020-03-25 21:51:17.35]: No personal or sensitive data is sent.
INFO [2020-03-25 21:51:17.35]: You can disable this by addingopt_out_usageat the top of your Fastfile
DEBUG [2020-03-25 21:51:17.37]: Using index: 47
DEBUG [2020-03-25 21:51:17.37]: Using test_without_building: true
INFO [2020-03-25 21:51:17.38]: ------------------------------
INFO [2020-03-25 21:51:17.38]: --- Step: default_platform ---
INFO [2020-03-25 21:51:17.39]: ------------------------------
INFO [2020-03-25 21:51:17.39]: Driving the lane 'ios container' 🚀
INFO [2020-03-25 21:51:17.40]: -----------------------------------------------------
INFO [2020-03-25 21:51:17.40]: Step: Switch to ios run_uitests_with_retry lane
INFO [2020-03-25 21:51:17.40]: -----------------------------------------------------
INFO [2020-03-25 21:51:17.40]: Cruising over to lane 'ios run_uitests_with_retry' 🚖
INFO [2020-03-25 21:51:17.40]: ------------------------
INFO [2020-03-25 21:51:17.40]: --- Step: multi_scan ---
INFO [2020-03-25 21:51:17.40]: ------------------------
Successfully loaded Appfile at path '/Users/distiller/project/fastlane/Appfile'Successfully loaded Appfile at path '/Users/distiller/project/fastlane/Appfile'
Successfully loaded Appfile at path '/Users/distiller/project/fastlane/Appfile'
+-----------------------+----------------------+
| Summary for multi_scan (test_center v3.8.15) |
+-----------------------+----------------------+
| try_count | 2 |
+-----------------------+----------------------+Successfully loaded Appfile at path '/Users/distiller/project/fastlane/Appfile'
Successfully loaded Appfile at path '/Users/distiller/project/fastlane/Appfile'
ERROR [2020-03-25 21:51:17.44]: Using deprecated option: '--custom_report_file_name' (Use
--output_filesinstead)
DEBUG [2020-03-25 21:51:17.44]: Preparing Scan config options for multi_scan testing
Successfully loaded Appfile at path '/Users/distiller/project/fastlane/Appfile'Successfully loaded Appfile at path '/Users/distiller/project/fastlane/Appfile'
ERROR [2020-03-25 21:51:17.44]: Using deprecated option: '--custom_report_file_name' (Use
--output_filesinstead)
INFO [2020-03-25 21:51:17.50]: $ xcodebuild -showBuildSettings -workspace Company.xcworkspace -scheme Company-Dev
WARN [2020-03-25 21:51:20.53]: Command timed out after 3 seconds on try 1 of 4, trying again with a 6 second timeout...
WARN [2020-03-25 21:51:26.54]: Command timed out after 6 seconds on try 2 of 4, trying again with a 12 second timeout...
DEBUG [2020-03-25 21:51:34.17]: Fetching available simulator devices
INFO [2020-03-25 21:51:34.47]: Starting test run 1
DEBUG [2020-03-25 21:51:34.47]: Deleting xcresults:
DEBUG [2020-03-25 21:51:34.47]: Restarting Simulator F3026518
INFO [2020-03-25 21:51:34.77]: Starting scan #1 with 7 tests for batch #1.
Successfully loaded Appfile at path '/Users/distiller/project/fastlane/Appfile'Successfully loaded Appfile at path '/Users/distiller/project/fastlane/Appfile'
DEBUG [2020-03-25 21:51:34.78]: retrying_scan #update_scan_options
DEBUG [2020-03-25 21:51:34.78]: Setting workspace to Company.xcworkspace
DEBUG [2020-03-25 21:51:34.78]: Setting scheme to Company-Dev
DEBUG [2020-03-25 21:51:34.78]: Setting test_without_building to true
DEBUG [2020-03-25 21:51:34.78]: Setting only_testing to ["CompanyUITests/testDefaultBankAffiliates", "CompanyUITests/testLocationBasedBankAffiliates", "CompanyUITests/testMaxIncreaseGranterIsInvalid", "CompanyUITests/testMaxIncreaseNoBoostsLeftResponse", "CompanyUITests/testMaxIncreaseRequesterIsInvalid", "CompanyUITests/testMaxIncreaseRequesterNotFound", "CompanyUITests/ReferralsTests/testShareFlowFromHome"]
DEBUG [2020-03-25 21:51:34.78]: Setting include_simulator_logs to false
DEBUG [2020-03-25 21:51:34.78]: Setting result_bundle to true
DEBUG [2020-03-25 21:51:34.78]: Setting output_style to rspec
DEBUG [2020-03-25 21:51:34.78]: Setting output_directory to /Users/distiller/project/build/DerivedData/test_output/g47/CompanyUITests-batch-1
DEBUG [2020-03-25 21:51:34.78]: Setting xctestrun to ./build/DerivedData/Build/Products/Company-Dev_iphonesimulator13.2-x86_64.xctestrun
DEBUG [2020-03-25 21:51:34.78]: Setting derived_data_path to ./build/DerivedData
DEBUG [2020-03-25 21:51:34.78]: Setting output_types to junit
DEBUG [2020-03-25 21:51:34.78]: Setting skip_detect_devices to false
DEBUG [2020-03-25 21:51:34.78]: Setting reset_simulator to false
DEBUG [2020-03-25 21:51:34.78]: Setting reinstall_app to false
DEBUG [2020-03-25 21:51:34.78]: Setting clean to false
DEBUG [2020-03-25 21:51:34.78]: Setting open_report to false
DEBUG [2020-03-25 21:51:34.78]: Setting output_files to report.junit
DEBUG [2020-03-25 21:51:34.78]: Setting buildlog_path to ~/Library/Logs/scan
DEBUG [2020-03-25 21:51:34.78]: Setting should_zip_build_products to false
DEBUG [2020-03-25 21:51:34.78]: Setting use_clang_report_name to false
DEBUG [2020-03-25 21:51:34.78]: Setting disable_concurrent_testing to true
DEBUG [2020-03-25 21:51:34.78]: Setting skip_build to false
DEBUG [2020-03-25 21:51:34.78]: Setting build_for_testing to false
DEBUG [2020-03-25 21:51:34.78]: Setting xcargs to -parallel-testing-enabled NO
DEBUG [2020-03-25 21:51:34.78]: Setting slack_use_webhook_configured_username_and_icon to false
DEBUG [2020-03-25 21:51:34.78]: Setting slack_username to fastlane
DEBUG [2020-03-25 21:51:34.78]: Setting slack_icon_url to https://s3-eu-west-1.amazonaws.com/fastlane.tools/fastlane.png
DEBUG [2020-03-25 21:51:34.78]: Setting skip_slack to false
DEBUG [2020-03-25 21:51:34.78]: Setting slack_only_on_failure to false
DEBUG [2020-03-25 21:51:34.78]: Setting destination to ["platform=iOS Simulator,id=F3026518-DE25-4715-B0E4-47C2C1A2A640"]
DEBUG [2020-03-25 21:51:34.78]: Setting xcodebuild_command to env NSUnbufferedIO=YES xcodebuild
DEBUG [2020-03-25 21:51:34.78]: Setting fail_build to true+------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Summary for scan 2.137.0 |
+------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| workspace | Company.xcworkspace |
| scheme | Company-Dev |
| test_without_building | true |
| only_testing | ["CompanyUITests/testDefaultBankAffiliates", "CompanyUITests/testLocationBasedBankAffiliates", "CompanyUITests/testMaxIncreaseGranterIsInvalid", "CompanyUITests/testMaxIncreaseNoBoostsLeftResponse", "CompanyUITests/testMaxIncreaseRequesterIsInvalid", "CompanyUITests/testMaxIncreaseRequesterNotFound", "CompanyUITests/ReferralsTests/testShareFlowFromHome"] |
| include_simulator_logs | false |
| result_bundle | true |
| output_style | rspec |
| output_directory | /Users/distiller/project/build/DerivedData/test_output/g47/CompanyUITests-batch-1 |
| xctestrun | ./build/DerivedData/Build/Products/Company-Dev_iphonesimulator13.2-x86_64.xctestrun |
| derived_data_path | ./build/DerivedData |
| output_types | junit |
| skip_detect_devices | false |
| force_quit_simulator | false |
| reset_simulator | false |
| reinstall_app | false |
| clean | false |
| open_report | false |
| output_files | report.junit |
| buildlog_path | ~/Library/Logs/scan |
| should_zip_build_products | false |
| use_clang_report_name | false |
| disable_concurrent_testing | true |
| skip_build | false |
| build_for_testing | false |
| xcargs | -parallel-testing-enabled NO |
| slack_use_webhook_configured_username_and_icon | false |
| slack_username | fastlane |
| slack_icon_url | https://s3-eu-west-1.amazonaws.com/fastlane.tools/fastlane.png |
| skip_slack | false |
| slack_only_on_failure | false |
| xcodebuild_command | env NSUnbufferedIO=YES xcodebuild |
| fail_build | true |
| xcode_path | /Applications/Xcode-11.2.1.app |
+------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+INFO [2020-03-25 21:51:34.86]: $ set -o pipefail && env NSUnbufferedIO=YES xcodebuild -destination 'platform=iOS Simulator,id=F3026518-DE25-4715-B0E4-47C2C1A2A640' -derivedDataPath './build/DerivedData' -resultBundlePath '/Users/distiller/project/build/DerivedData/test_output/g47/CompanyUITests-batch-1/Company-Dev.test_result' -disable-concurrent-testing -xctestrun './build/DerivedData/Build/Products/Company-Dev_iphonesimulator13.2-x86_64.xctestrun' -parallel-testing-enabled NO -only-testing:CompanyUITests/testDefaultBankAffiliates -only-testing:CompanyUITests/testLocationBasedBankAffiliates -only-testing:CompanyUITests/testMaxIncreaseGranterIsInvalid -only-testing:CompanyUITests/testMaxIncreaseNoBoostsLeftResponse -only-testing:CompanyUITests/testMaxIncreaseRequesterIsInvalid -only-testing:CompanyUITests/testMaxIncreaseRequesterNotFound -only-testing:CompanyUITests/ReferralsTests/testShareFlowFromHome test-without-building | tee '/Users/distiller/Library/Logs/scan/Company-Company-Dev.log' | xcpretty --test --report junit --output '/Users/distiller/project/build/DerivedData/test_output/g47/CompanyUITests-batch-1/report.junit' --report junit --output '/var/folders/z7/8rr0l_45719dk91y30bbhvw40000gn/T/junit_report20200325-705-m8ddd1'
Environment
🚫 fastlane environment 🚫
fastlane files:
`./fastlane/Fastfile`
# Uncomment the line if you want fastlane to automatically update itself # update_fastlane #destination: DESTINATION, SCHEME="Company-Dev" DESTINATION="platform=iOS Simulator,name=iPhone 8,OS=13.2.2" ENV["FASTLANE_HIDE_CHANGELOG"]="1" ENV["FASTLANE_SKIP_UPDATE_CHECK"]="1" default_platform(:ios) platform :ios do before_all do #update_fastlane #setup_circle_ci end #Example call: fastlane container index:15 desc "Runs group of UITests" lane :container do |options| UITestCases = JSON.parse(File.read('UITestCases-ForCI.json')) run_uitests_with_retry( only_testing: UITestCases[options[:index]], output_directory: "./build/DerivedData/test_output/g#{options[:index]}", test_without_building: options[:test_without_building]) end end MY_XCODE_PROJECT_FILEPATH = File.absolute_path('../Company.xcworkspace') lane :run_uitests_with_retry do |values| if (values[:test_without_building] == true) #find xctestrun file xctestrun_file = Dir["/build/DerivedData/*.xctestrun"].first multi_scan( workspace: "Company.xcworkspace", try_count: 2, scheme: SCHEME, test_without_building: values[:test_without_building], only_testing: values[:only_testing], include_simulator_logs: false, result_bundle: true, output_style: "rspec", output_directory: values[:output_directory], xctestrun: xctestrun_file, derived_data_path: "./build/DerivedData", device:'iPhone 8 (13.2.2)', output_types: "junit" ) else #multi_scan fails if xctestrun is passed with test_without_building: false with this error #Unresolved conflict between options: 'xctestrun' and 'build_for_testing' multi_scan( workspace: "Company.xcworkspace", try_count: 2, scheme: 'Company-Dev', only_testing: values[:only_testing], include_simulator_logs: false, result_bundle:true, output_style: "rspec", output_directory: values[:output_directory], derived_data_path: "./build/DerivedData", device:'iPhone 8 (13.2.2)', output_types: "junit" ) end end`./fastlane/Appfile`
# app_identifier("[[APP_IDENTIFIER]]") # The bundle identifier of your app # apple_id("[[APPLE_ID]]") # Your Apple email address # For more information about the Appfile, see: # https://docs.fastlane.tools/advanced/#appfilefastlane gems
Gem Version Update-Status fastlane 2.142.0 🚫 Update available Loaded fastlane plugins:
Plugin Version Update-Status fastlane-plugin-test_center 3.8.15 🚫 Update available fastlane-plugin-badge undefined 💥 Check failed Loaded gems
Gem Version did_you_mean 1.3.0 slack-notifier 2.3.2 rouge 2.0.7 xcpretty 0.3.0 terminal-notifier 2.0.0 terminal-table 1.8.0 plist 3.5.0 public_suffix 2.0.5 addressable 2.7.0 multipart-post 2.0.0 word_wrap 1.0.0 babosa 1.0.3 colored 1.2 highline 1.7.10 commander-fastlane 4.4.6 excon 0.72.0 unf_ext 0.0.7.6 unf 0.1.4 domain_name 0.5.20190701 http-cookie 1.0.3 faraday-cookie_jar 0.0.6 faraday_middleware 0.13.1 fastimage 2.1.7 gh_inspector 1.1.3 multi_xml 0.6.0 rubyzip 1.3.0 security 0.1.3 xcpretty-travis-formatter 1.0.0 dotenv 2.7.5 bundler 2.0.2 jwt 2.1.0 uber 0.1.0 declarative 0.0.10 declarative-option 0.1.0 representable 3.0.4 retriable 3.1.2 mini_mime 1.0.2 multi_json 1.14.1 signet 0.12.0 os 1.0.1 googleauth 0.10.0 httpclient 2.8.3 google-api-client 0.36.4 emoji_regex 1.0.1 nanaimo 0.2.6 colored2 3.1.2 claide 1.0.3 CFPropertyList 3.0.2 atomos 0.1.3 xcodeproj 1.15.0 unicode-display_width 1.6.1 tty-screen 0.7.1 tty-cursor 0.7.1 tty-spinner 0.9.3 faraday 0.17.3 json 2.3.0 mini_magick 4.10.1 naturally 2.2.0 simctl 1.6.8 memoist 0.16.2 digest-crc 0.4.1 google-cloud-errors 1.0.0 google-cloud-env 1.3.0 google-cloud-core 1.5.0 google-cloud-storage 1.25.1 forwardable 1.2.0 logger 1.3.0 stringio 0.0.2 ipaddr 1.2.2 openssl 2.1.2 ostruct 0.1.0 strscan 1.0.0 date 2.0.0 fileutils 1.1.0 etc 1.0.1 io-console 0.4.7 zlib 1.0.0 libxml-ruby 3.1.0 rexml 3.1.9 psych 3.1.0 mutex_m 0.1.0 xctest_list 1.1.8 colorize 0.8.1 fastlane-plugin-test_center 3.8.15 generated on: 2020-03-25
- If you use
scaninstead, does it work? - Do you have access to the logs, like
/Users/distiller/project/build/DerivedData/test_output/g47/CompanyUITests-batch-1/Company-Dev.test_result/Staging/1_Test/Diagnostics/CompanyUITests-7C6C0BFE-B1CE-4FF2-A238-467E17A59945/CompanyUITests-5EBDFE08-8DFC-4CED-9AE3-7B5DDB5D300A/Session-CompanyUITests-2020-03-25_215136-do8fuT.log? If so, are there any clues there?
-
Haven't tried yet.
-
Because it times out, it isn't able to save that log as an artifact so we do not get access to it
Okay, I would try running scan first. If that also hangs, you may want to try to put together your only simple call to xcodebuild and see if it can run.
INFO [2020-03-25 21:51:34.86]: $ set -o pipefail && env NSUnbufferedIO=YES xcodebuild -destination 'platform=iOS Simulator,id=F3026518-DE25-4715-B0E4-47C2C1A2A640' -derivedDataPath './build/DerivedData' -resultBundlePath '/Users/distiller/project/build/DerivedData/test_output/g47/CompanyUITests-batch-1/Company-Dev.test_result' -disable-concurrent-testing -xctestrun './build/DerivedData/Build/Products/Company-Dev_iphonesimulator13.2-x86_64.xctestrun' -parallel-testing-enabled NO -only-testing:CompanyUITests/testDefaultBankAffiliates -only-testing:CompanyUITests/testLocationBasedBankAffiliates -only-testing:CompanyUITests/testMaxIncreaseGranterIsInvalid -only-testing:CompanyUITests/testMaxIncreaseNoBoostsLeftResponse -only-testing:CompanyUITests/testMaxIncreaseRequesterIsInvalid -only-testing:CompanyUITests/testMaxIncreaseRequesterNotFound -only-testing:CompanyUITests/ReferralsTests/testShareFlowFromHome test-without-building | tee '/Users/distiller/Library/Logs/scan/Company-Company-Dev.log' | xcpretty --test --report junit --output '/Users/distiller/project/build/DerivedData/test_output/g47/CompanyUITests-batch-1/report.junit' --report junit --output '/var/folders/z7/8rr0l_45719dk91y30bbhvw40000gn/T/junit_report20200325-705-m8ddd1'
I could swear that you sent emails that without --verbose this works fine. If so, can you try with --scan as well? If it also hangs, then that is an Issue that you should open with fastlane folks.
Closed as I cannot reproduce and there is no forthcoming logs to help diagnose this. I can reopen this Issue if more information is provided.