codewars/runner

PowerShell tests fail with no feedback

hobovsky opened this issue · 3 comments

hobovsky commented

Some kinds of errors cause PowerShell tests fail with completely no feedback other than "Time: 2277ms
Response received but no data was written to STDOUT or STDERR.".

Example: this kumite.

I think this happens mostly on syntax errors, but I am not sure if it's the only kind of errors causing this, is it limited only to some snippets, etc.

kaz-yoshihara commented

We're converting the NUnitXML, so that's probably why we get nothing if the test doesn't start. We can change how we run the tests if there's a way to capture the errors.

Here's how we run the tests.

The files:

  • Solution.ps1
  • Solution.Tests.ps1
  • Preloaded.ps1 (optional)

The command:

pwsh -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Bypass Run-Tests.ps1

Run-Tests.ps1:

Import-Module -Name Pester
Import-Module -Name Pester-Codewars

$config = New-PesterConfiguration
$config.Should.ErrorAction = 'Continue'
$config.Output.Verbosity = 'None'
$config.Run.PassThru = $true

Invoke-Pester -Configuration $config | ConvertTo-NUnitReport | ConvertTo-Codewars

Pester-Codewars is https://github.com/codewars/Pester-Codewars and stored in ~/.local/share/powershell/Modules/Pester-Codewars/Pester-Codewars.psm1.

hobovsky commented

After the Invoke-Pester finishes, something takes care of processing the NUnit XML, right? Is it possible to check out the stdout/stderr if the report is not found, and present it as output of a test run?

I can check if/how such error is reported, I just do not know if it's possible to use stdout if there's no XML report.

EDIT: Okay I think I got it.

kazk commented

The last line in Run-Tests.ps1:

Invoke-Pester -Configuration $config | ConvertTo-NUnitReport | ConvertTo-Codewars

I forgot the details, but ConvertTo-NUnitReport should be generating the NUnit XML. Then that's piped to ConvertTo-Codewars .
I don't think we do anything to the stderr.