Cached data being return?

Question

Cached data being return?

Closed this issue 6 years ago · 17 comments

I called a route incorrectly passing it hashtable instead of a json object and it return the output from the last command. I don't think this should be expected behaviour.

# The following shows the two route definitions
PS C:\> (Get-WebRoute)['Get-User']['GET']
    $params = @{}
    $request.Query | ForEach-Object { $params[$_] = $request.Query[$_] }
    $result = Get-User @params | ConvertTo-Json
    $response.Send($result)

PS C:\> (Get-WebRoute)['Remove-User']['DELETE']
    $params = @{}
    $request.body.psobject.properties | ForEach-Object { $params[$_.Name] = $_.Value }
    $result = Remove-User @params
    $response.Send($result)

# incorrectly creating a hashtable instead of a json object
PS C:\> $remove = @{UserName='joe.bloggs@domain.com'}

# Create a user
PS C:\> Invoke-RestMethod -Method POST -Uri http://localhost:8080/New-User -Body $body -ContentType application/json
# Note: nothing is returned from this call, as expected

# Try to remove the user with the hashtable being sent
PS C:\> Invoke-RestMethod -Method DELETE -Uri http://localhost:8080/Remove-User -Body $remove -ContentType application/json
# No output, this I think is simply returning the same output from the New-User call, which is nothing, to prove see below

# Make call which returns output
PS C:\> Invoke-RestMethod -Method GET -Uri 'http://localhost:8080/Get-User?Username=joe.bloggs@domain.com'
DisplayName         : Joe Bloggs
Enabled             : True
LockedOut           : False
PasswordLastSet     : 09/11/2017 12:59:25
LastLogonDate       :

# call again with incorrect hashtable being sent
PS C:\> Invoke-RestMethod -Method DELETE -Uri http://localhost:8080/Remove-User -Body $remove -ContentType application/json

DisplayName         : Joe Bloggs
Enabled             : True
LockedOut           : False
PasswordLastSet     : 09/11/2017 12:59:25
LastLogonDate       :
# As you can see  the output from Get-User is returned, to prove see the output when Remove-User is provided with a json object
PS C:\> $remove = @{UserName='joe.bloggs@domain.com'} | ConvertTo-Json
PS C:\> Invoke-RestMethod -Method DELETE -Uri http://localhost:8080/Remove-User -Body $remove -ContentType application/json

PS C:\> Get-ADUser joe.bloggs
Get-ADUser : Cannot find an object with identity: <snip>
At line:1 char:1
+ Get-ADUser joe.bloggs
+ ~~~~~~~~~~~~~~~~~~~~~

PS C:\> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.14393.1770
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.14393.1770
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Answer 1 · 2017-11-09T17:02:33.000Z

Interesting! Can you add a -Verbose flag to your Invoke-RestMethods and also for context, what is Remove-User suppose to return?

Answer 2 · 2017-11-09T18:27:38.000Z

Also just to confirm, you added the -UseJsonBodyParserMiddleware flag to Start-Polaris to parse the json body.

I'm having a hard time repro so if you could give me a complete repro that would help (from creation of routes to starting the server to the Invoke-RestMethods)

Answer 3 · 2017-11-09T23:31:14.000Z

So remove-user returns nothing, I've got a repro for you.
I'll post the code on its own and then the execution which includes the code when executed.

Answer 4 · 2017-11-09T23:31:41.000Z

Set-Location C:\
Clear-Host
Import-Module C:\Polaris\Polaris.psm1

function Build-WebRoute {
    [cmdletbinding()]
    Param(
        [Parameter(
            Mandatory=$true,
            ValueFromPipeline=$True
        )]
        [string[]]$Name
    )

    Begin{}

    Process{
        $cmdlet = Get-Command -Name $Name

        $queryBlock = '
            $params = @{{}}
            $request.Query | ForEach-Object {{ $params[$_] = $request.Query[$_] }}
            $result = {0} @params
            $response.Send($result)
        ' -f $Name

        $bodyBlock = '
            $params = @{{}}
            $request.body.psobject.properties | ForEach-Object {{ $params[$_.Name] = $_.Value }}
            $result = {0} @params
            $response.Send($result)
        ' -f $Name

        Switch ($cmdlet.Verb) {
            'Get' { $verb = 'GET' }
            'New' { $verb = 'POST' }
            'Set' { $verb = 'PATCH' }
            'Remove' { $verb = 'DELETE' }
            default { $verb = 'POST' }
        }

        If ($verb -eq 'GET') {
            $sb = [ScriptBlock]::Create($queryBlock)
        }
        Else {
            $sb = [ScriptBlock]::Create($bodyBlock)
        }

        New-WebRoute -Path "/$Name" -Method $verb -ScriptBlock $sb
    }

    End{}
}

Build-WebRoute -Name New-Item
Build-WebRoute -Name Get-Item
Build-WebRoute -Name Remove-Item
Start-Polaris -UseJsonBodyParserMiddleware

$json = @{"Path"="C:\TestDirectory";"ItemType"="Directory"} | ConvertTo-Json -Compress
Invoke-RestMethod -Uri http://localhost:8080/New-Item -Method POST -Body $json -ContentType application/json -Verbose
Get-Item C:\TestDirectory

# Not json
$json = @{"Path"="C:\TestDirectory"}

# Test 1
Invoke-RestMethod -Method GET -Uri 'http://localhost:8080/Get-Item?Path=C:\TestDirectory' -Verbose
Invoke-RestMethod -Uri http://localhost:8080/Remove-Item -Method DELETE -Body $json -ContentType application/json -Verbose

# Test 2
Invoke-RestMethod -Method GET -Uri 'http://localhost:8080/Get-Item?Path=C:\Windows' -Verbose
Invoke-RestMethod -Uri http://localhost:8080/Remove-Item -Method DELETE -Body $json -ContentType application/json -Verbose

$json = @{"Path"="C:\TestDirectory"} | ConvertTo-Json -Compress
Invoke-RestMethod -Uri http://localhost:8080/Remove-Item -Method DELETE -Body $json -ContentType application/json -Verbose
Get-Item C:\TestDirectory

$PSVersionTable

Answer 5 · 2017-11-09T23:33:16.000Z

PS C:\> Import-Module C:\Polaris\Polaris.psm1
PS C:\>
PS C:\> function Build-WebRoute {
>>     [cmdletbinding()]
>>     Param(
>>         [Parameter(
>>             Mandatory=$true,
>>             ValueFromPipeline=$True
>>         )]
>>         [string[]]$Name
>>     )
>>
>>     Begin{}
>>
>>     Process{
>>         $cmdlet = Get-Command -Name $Name
>>
>>         $queryBlock = '
>>             $params = @{{}}
>>             $request.Query | ForEach-Object {{ $params[$_] = $request.Query[$_] }}
>>             $result = {0} @params
>>             $response.Send($result)
>>         ' -f $Name
>>
>>         $bodyBlock = '
>>             $params = @{{}}
>>             $request.body.psobject.properties | ForEach-Object {{ $params[$_.Name] = $_.Value }}
>>             $result = {0} @params
>>             $response.Send($result)
>>         ' -f $Name
>>
>>         Switch ($cmdlet.Verb) {
>>             'Get' { $verb = 'GET' }
>>             'New' { $verb = 'POST' }
>>             'Set' { $verb = 'PATCH' }
>>             'Remove' { $verb = 'DELETE' }
>>             default { $verb = 'POST' }
>>         }
>>
>>         If ($verb -eq 'GET') {
>>             $sb = [ScriptBlock]::Create($queryBlock)
>>         }
>>         Else {
>>             $sb = [ScriptBlock]::Create($bodyBlock)
>>         }
>>
>>         New-WebRoute -Path "/$Name" -Method $verb -ScriptBlock $sb
>>     }
>>
>>     End{}
>> }
PS C:\>
PS C:\> Build-WebRoute -Name New-Item
PS C:\> Build-WebRoute -Name Get-Item
PS C:\> Build-WebRoute -Name Remove-Item
PS C:\> Start-Polaris -UseJsonBodyParserMiddleware

Port ScriptBlockRoutes
---- -----------------
8080 {[New-Item, System.Collections.Generic.Dictionary`2[System.String,System.String]], [Get-Item, System.Collections.Generic.Dictionary`2[System.String,System.String]]...


PS C:\>
PS C:\> $json = @{"Path"="C:\TestDirectory";"ItemType"="Directory"} | ConvertTo-Json -Compress
PS C:\> Invoke-RestMethod -Uri http://localhost:8080/New-Item -Method POST -Body $json -ContentType application/json -Verbose
VERBOSE: POST http://localhost:8080/New-Item with 51-byte payload
VERBOSE: received 16-byte response of content type text/plain
VERBOSE: Content encoding: iso-8859-1
C:\TestDirectory
PS C:\> Get-Item C:\TestDirectory


    Directory: C:\


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----       09/11/2017     23:27                TestDirectory


PS C:\>
PS C:\> # Not json
PS C:\> $json = @{"Path"="C:\TestDirectory"}
PS C:\>
PS C:\> # Test 1
PS C:\> Invoke-RestMethod -Method GET -Uri 'http://localhost:8080/Get-Item?Path=C:\TestDirectory' -Verbose
VERBOSE: GET http://localhost:8080/Get-Item?Path=C:\TestDirectory with 0-byte payload
VERBOSE: received 16-byte response of content type text/plain
VERBOSE: Content encoding: iso-8859-1
C:\TestDirectory
PS C:\> Invoke-RestMethod -Uri http://localhost:8080/Remove-Item -Method DELETE -Body $json -ContentType application/json -Verbose
VERBOSE: DELETE http://localhost:8080/Remove-Item with 25-byte payload
VERBOSE: received 16-byte response of content type text/plain
VERBOSE: Content encoding: iso-8859-1
C:\TestDirectory
PS C:\>
PS C:\> # Test 2
PS C:\> Invoke-RestMethod -Method GET -Uri 'http://localhost:8080/Get-Item?Path=C:\Windows' -Verbose
VERBOSE: GET http://localhost:8080/Get-Item?Path=C:\Windows with 0-byte payload
VERBOSE: received 10-byte response of content type text/plain
VERBOSE: Content encoding: iso-8859-1
C:\Windows
PS C:\> Invoke-RestMethod -Uri http://localhost:8080/Remove-Item -Method DELETE -Body $json -ContentType application/json -Verbose
VERBOSE: DELETE http://localhost:8080/Remove-Item with 25-byte payload
VERBOSE: received 10-byte response of content type text/plain
VERBOSE: Content encoding: iso-8859-1
C:\Windows
PS C:\>
PS C:\> $json = @{"Path"="C:\TestDirectory"} | ConvertTo-Json -Compress
PS C:\> Invoke-RestMethod -Uri http://localhost:8080/Remove-Item -Method DELETE -Body $json -ContentType application/json -Verbose
VERBOSE: DELETE http://localhost:8080/Remove-Item with 28-byte payload
VERBOSE: received 0-byte response of content type text/plain
VERBOSE: Content encoding: iso-8859-1

PS C:\> Get-Item C:\TestDirectory
Get-Item : Cannot find path 'C:\TestDirectory' because it does not exist.
At line:1 char:1
+ Get-Item C:\TestDirectory
+ ~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (C:\TestDirectory:String) [Get-Item], ItemNotFoundException
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetItemCommand

PS C:\>
PS C:\> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      6.0.0-beta.9
PSEdition                      Core
GitCommitId                    v6.0.0-beta.9
OS                             Microsoft Windows 10.0.16299
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Answer 6 · 2017-11-09T23:34:21.000Z

PS I will get round to writing the pester tests and submitting the PR for Build-WebRoute, I found a bug in it which I've fixed, so probably for the best that I finish the pester tests first.

Answer 7 · 2017-11-10T03:12:01.000Z

So that does repro for me. Interesting. I think we might not be cleaning up the response object. I might be wrong though. I'll run this under the debugger in VSCode and see what we get

Answer 8 · 2017-11-10T07:46:56.000Z

Ok so,

When you do Invoke-RestMethod without converting the body to json, it sends over that hashtable as a string:

Take a look at the string in the middle of this image:

That is how Invoke-RestMethod works, so we must play with that. Now, I'm pretty sure, there was an error in my Json parsing middleware so I wrapped that in a try/catch to catch when strings don't parse correctly. That said, 2 errors get thrown in your script block:

$bodyBlock = '
            $params = @{{}}
            $request.body.psobject.properties | ForEach-Object {{ $params[$_.Name] = $_.Value }}
            $result = {0} @params
            $response.Send($result)
        ' -f $Name

$request.body.psobject.properties | ForEach-Object { $params[$_.Name] = $_.Value }
with a Index operation failed; the array index evaluated to null. I'm guessing because $request.body is null

and $result = Remove-Item @params
because it's expecting the -Path

I thought that Polaris was throwing errors (returning 500 InternalServerError) when this happened but I guess that's not true. I'll open an issue to fix error reporting.

As for returning the previous value... It could possibly be because if you're running with only 1 runspace, $result is set from the last call. This might be fixable by using the UseLocalScope boolean on AddScript which will run the scripts in the local scope (rather than global scope - if that's the default)
https://docs.microsoft.com/en-us/dotnet/api/system.management.automation.powershell.addscript?view=powershellsdk-1.1.0#System_Management_Automation_PowerShell_AddScript_System_String_System_Boolean_

Answer 9 · 2017-11-10T11:36:45.000Z

I tried setting all three instances of AddScript in Polaris.cs to uselocalscope, trying lines 178 and 188 to true on there own and together, then finally all three AddScript's 178, 185 and 188 to true. None of those stopped $request from returning the old information.

Re-ran invoke-build on each change and restarted the PowerShell session, probably would have been quick to remove-module polaris...

Answer 10 · 2017-11-10T15:50:00.000Z

How strange. I'm 100% certain that the correct script block is being run.

I'll have to maybe do a Write-Verbose within your script block to see what $result is set to. I'm still convinced that it's keeping the same value for $result

Also this brings up a good point that we should have a way to get the Verbose logs of the script execution. I'm thinking with a Query param Verbose=true

Answer 11 · 2017-11-10T17:34:57.000Z

I agree that $result is keeping the old value, although I'm don't understand the underlying execution, does each script block execution not get its own session / runspace?

The option for verbose logs sounds cool.

Answer 12 · 2017-11-11T04:57:36.000Z

So it uses a RunspacePool. When you do Start-Polaris, you can optionally set the minimum and maximum number of runspaces. My understanding is that these are separate instances of PowerShell. By default, Polaris runs with 1 runspace.

If you set $result in the global scope in a route, that will set it in the global scope for that whole runspace.

This is my understanding at least

Answer 13 · 2017-11-11T04:58:29.000Z

1 runspace per route call would take too long because a new instance of PowerShell would need to be provisioned

Answer 14 · 2017-11-11T09:32:33.000Z

I've got #70 out now to actually return errors

Answer 15 · 2017-11-13T22:23:22.000Z

Ok this is what we need to address this:

https://docs.microsoft.com/en-us/dotnet/api/system.management.automation.runspaces.runspace.resetrunspacestate?view=powershellsdk-1.1.0#System_Management_Automation_Runspaces_Runspace_ResetRunspaceState

Answer 16 · 2017-11-15T16:00:54.000Z

Adding a call to that method after https://github.com/PowerShell/Polaris/blob/master/PolarisCore/Polaris.cs#L234?

Answer 17 · 2018-04-27T00:35:13.000Z

We're no longer using runspaces and went with an event based web server so this should be fixed now! Thanks to @Tiberriver256