microsoft/DbgShell

AddressTransformation integer handling does the opposite of what it is supposed to

Zhentar opened this issue · 2 comments

I had figured that the inconsistent handling between addresses with only decimal digits was just an unfortunate cost of PowerShell, until I saw the AddressTransformation... so I poked at it a bit more:

> Read-DbgMemory -Address 0x17409680
17409680  "󸇳."

> 0x17409680 | Read-DbgMemory
Read-DbgMemory : Could not access memory: 90108800
At line:1 char:14
+ 0x17409680 | Read-DbgMemory
+              ~~~~~~~~~~~~~~
    + CategoryInfo          : ReadError: (15301904384:UInt64) [Read-DbgMemory], DbgMemoryAccessException
    + FullyQualifiedErrorId : MemoryAccessFailure,MS.Dbg.Commands.ReadDbgMemoryCommand

> Read-DbgMemory -Address 17409680
Read-DbgMemory : Could not access memory: 0109a690
At line:1 char:1
+ Read-DbgMemory -Address 17409680
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ReadError: (17409680:UInt64) [Read-DbgMemory], DbgMemoryAccessException
    + FullyQualifiedErrorId : MemoryAccessFailure,MS.Dbg.Commands.ReadDbgMemoryCommand

Thanks for the report!

The relevant code is:

if( (pso.BaseObject is int) ||
    (pso.BaseObject is long) ||
    (pso.BaseObject is double) ||
    (pso.BaseObject is float) )
{
    // This will get what the user actually typed (if it was typed).
    var asTyped = LanguagePrimitives.ConvertTo< string >( pso );

And the problem is that the comment is wrong in the case of pipelining: if the value was piped in, LanguagePrimitives.ConvertTo succeeds in returning a string, but that string does not match what was originally typed (it's been converted into a decimal string representation instead of the "0x..." that was typed). :'(

I have some ideas of some more tricks we could play.

For the Read-DbgMemory -Address 17409680 case, the problem was that var pso = inputData as PSObject; returned null - Powershell is just passing in a boxed int, not a PSObject.