open-telemetry/opentelemetry-ruby

Flaky test on truffle ruby

mwear opened this issue · 2 comments

mwear commented

The opentelemetry-exporter-otlp / ubuntu-latest tests are failing regularly on Truffle Ruby on main. See this example or the cut and pasted output below:

Prepare all required actions
Getting action download info
Run ./.github/actions/test_gem
Run # 🛠️ Setup 🛠️
Run ruby/setup-ruby@v1.144.1
Modifying PATH
Downloading Ruby
Extracting  Ruby
Print Ruby version
Installing Bundler
Run # 💎 Install dependencies and generate appraisals 💎
Run # 🍿 Test Gem 🍿
🔎 Appraising google_protobuf_3.14
🔎 Appraising google_protobuf_3.15
  Gems included by the bundle:
    * addressable (2.8.4)
    * appraisal (2.2.0)
    * ast (2.4.2)
    * bundler (2.4.17)
    * byebug (11.1.3)
    * coderay (1.1.3)
    * crack (0.4.5)
    * docile (1.4.0)
    * faraday (0.17.6)
    * google-protobuf (3.23.4)
    * googleapis-common-protos-types (1.7.0)
    * hashdiff (1.0.1)
    * json (2.6.3)
    * language_server-protocol (3.17.0.3)
    * method_source (1.0.0)
    * minitest (5.19.0)
    * multipart-post (2.3.0)
    * opentelemetry-api (1.2.1)
    * opentelemetry-common (0.20.0)
    * opentelemetry-exporter-otlp (0.26.1)
    * opentelemetry-registry (0.3.0)
    * opentelemetry-sdk (1.3.0)
    * opentelemetry-semantic_conventions (1.10.0)
    * opentelemetry-test-helpers (0.4.0)
    * parallel (1.23.0)
    * parser (3.2.2.3)
    * pry (0.14.2)
    * pry-byebug (3.10.1)
    * public_suffix (5.0.3)
    * racc (1.7.1)
    * rainbow (3.1.1)
    * rake (12.3.3)
    * regexp_parser (2.8.1)
    * rexml (3.2.6)
    * rubocop (1.55.1)
    * rubocop-ast (1.29.0)
    * ruby-progressbar (1.13.0)
    * simplecov (0.22.0)
    * simplecov-html (0.12.3)
    * simplecov_json_formatter (0.1.4)
    * thor (1.2.2)
    * unicode-display_width (2.4.2)
    * webmock (3.7.6)
    * yard (0.9.34)
    * yard-doctest (0.1.17)
  Run options: --seed 23868
  
  # Running:
  
  ..........F....S.............................
  
  Finished in 2.821293s, 15.9501 runs/s, 44.6604 assertions/s.
  
    1) Failure:
  OpenTelemetry::Exporter::OTLP::Exporter::#export#test_0017_translates all the things [/home/runner/work/opentelemetry-ruby/opentelemetry-ruby/exporter/otlp/test/opentelemetry/exporter/otlp/exporter_test.rb:748]:
  The request POST http://localhost:4318/v1/traces with given block was expected to execute 1 time but it executed 0 times
  
  The following requests were made:
  
  POST http://localhost:4318/v1/traces with body '�������d������ͥ�%T�����ZRT�W�������*��ś_�����R��CU����^���*���RT��R!���,��83?O����P�X�@h53���[IQbrj��[����������_甃_�~�4�+������q��������Z,E��%��������y��;.�?���Ȩb�`�r���QD��HP���Ǻ��O-�X��Ŗ��	�����F=��0��a3��ɮ�R���"����W\��Zd�j���پ�͜3���iWBx�59#3'ŀ�r�����x�s1&	1	0z�q1�	q*Κ	���A��BL�L^\\��B�\�e�9^�\��B\Z�\L��@��%ù ;��d��kj���R\\,�%%E@�ZY���Z	�����LBa\|\<�%��E��(���_�����sTJ	��ZlũE��7`�\���q�����n�pv���' with headers {'Accept'=>'*/*', 'Accept-Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3', 'Content-Encoding'=>'gzip', 'Content-Type'=>'application/x-protobuf', 'User-Agent'=>['OTel-OTLP-Exporter-Ruby/0.26.1 Ruby/3.0.3 (x86_64-linux; truffleruby/22.3.1)', 'Ruby']} was made 1 time
  
  ============================================================
  
  45 runs, 126 assertions, 1 failures, 0 errors, 1 skips
  
  You have skipped tests. Run with --verbose for details.
  rake aborted!
  Command failed with status (1)
  <internal:core> core/kernel.rb:376:in `load'
  <internal:core> core/kernel.rb:376:in `load'
  <internal:core> core/kernel.rb:376:in `load'
  <internal:core> core/kernel.rb:376:in `load'
  <internal:core> core/kernel.rb:376:in `load'
  <internal:core> core/kernel.rb:376:in `load'
  /home/runner/.rubies/truffleruby-22.3.1/bin/bundle:23:in `<main>'
  Tasks: TOP => test
  (See full trace by running task with --trace)
  Error: Process completed with exit code 1.
pichlermarc commented

@mwear could it be that an identically named test is also failing for exporter/otlp-http/test/opentelemetry/exporter/otlp/http/trace_exporter_test.rb? 🤔

Looks to be flaky on not just truffleruby though (main+ruby3.0, #1506 +truffleruby)

github-actions commented

👋 This issue has been marked as stale because it has been open with no activity. You can: comment on the issue or remove the stale label to hold stale off for a while, add the keep label to hold stale off permanently, or do nothing. If you do nothing this issue will be closed eventually by the stale bot.