Out of memory error using static route

[BladeFireLight]

It would seem get-content does not like large files

[0]:
System.OutOfMemoryException: Array dimensions exceeded supported range.
   at System.Collections.ArrayList.set_Capacity(Int32 value)
   at System.Collections.ArrayList.EnsureCapacity(Int32 min)
   at System.Collections.ArrayList.Add(Object value)
   at Microsoft.PowerShell.Commands.FileSystemContentReaderWriter.ReadByteEncoded(Boolean waitChanges, ArrayList blocks, Boolean readBack)
   at Microsoft.PowerShell.Commands.FileSystemContentReaderWriter.Read(Int64 readCount)
   at Microsoft.PowerShell.Commands.GetContentCommand.ProcessRecord()At line:4 char:14
+ ...    $bytes = Get-Content "C:\WIM\Srv1709.wim" -Encoding Byte -ReadCoun ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[TylerLeonhardt]

Thanks for this!

How big is the file?

[BladeFireLight]

4-6Gb not really something that can be stored in memory.
that error was when compiled for 4.7.1
compiled for core2 it just hangs. no error.

[BladeFireLight]

tried to reduce memory usage it with this
New-GetRoute -Path '/test.wim' -ScriptBlock { $response.SetContentType("application/octet-str
eam"); $response.Send((get-content C:\WIM\Srv1709.wim -AsByteStream)); }
no help.

[TylerLeonhardt]

Hmm ok. PowerShell just probably can't handle loading the huge file into memory.

Maybe Serve Static should not create scripts. That was kinda a hack to get some functionality. I have an idea on how to address this in the C# of Polaris - rather than in the PowerShell

[BladeFireLight]

I have been stareing at the code for the last two days, and it does look like it would need a separate code path. everything looks to expect a script.

[TylerLeonhardt]

Sorry - Can you say that a bit differently? I don't think I quite follow your last comment

[BladeFireLight]

I would like to help fix this. I have been looking at a number of options that work with large files. but I'm not sure where to start. the main part of the code expects a PowerShell script.

[TylerLeonhardt]

So what I was thinking is modifying this dictionary:
https://github.com/PowerShell/Polaris/blob/master/PolarisCore/Polaris.cs#L20

And instead of it being:

Dictionary<string, Dictionary<string, string>>

It could be:

Dictionary<string, Dictionary<string, RouteAction>>

Where RouteAction is a new object we make that contains:

{
Type - string or enum that could be "FilePath" or "ScriptBlock"
Data - the data for that type 
}

Then in the listener loop:
https://github.com/PowerShell/Polaris/blob/master/PolarisCore/Polaris.cs#L138

Which is responsible for handling requests we can check which type the route action is and take the right action.

if (RouteAction == "ScriptBlock") {
    // trigger script block via PowerShell 
} else if (RouteAction == "File Path") {
    // do your idea
} 

Does that make sense / sounds any good?

[TylerLeonhardt]

I think you should try the static route again. @Tiberriver256 made a change that handles files that are served statically a bit better.

[TylerLeonhardt]

I'm gonna close this since we can’t repo and have changed things. If you’re still seeing the problem, please open a new issue and we'll help further 😁