rapid7/meterpreter

Backgrounding channel multiple times may kill the meterpreter session

wchen-r7 opened this issue · 14 comments

wchen-r7 commented

To be able to reproduce this bug, please do:

  • Set up a Windows box. I had a Windows 8 box while testing this.
  • A 32-bit Windows meterpreter. I had a windows/meterperter/reverse_tcp
  • Not sure if this matters, but I'm on ruby 2.1.5p273

And then you do this:

  • Get a session
  • Type "shell" to get an interactive command prompt
  • Press [Ctrl]+[Z] to background the channel, and Y
  • You'll see this error: "Failed to spawn shell with thread impersonation. Retrying without it."
  • Press [Ctrl]+[Z] again, and Y
  • You'll see this error: "Error running command shell: ThreadError can't be called from trap context"
  • At this point, you are probably at the meterpreter prompt. Press [Ctrl]+[Z] again.
  • You will see this error: "Session manipulation failed: can't be called from trap context" with the following backtrace:
/Users/sinn3r/rapid7/msf/lib/rex/post/meterpreter/packet_dispatcher.rb:154:in `synchronize'
/Users/sinn3r/rapid7/msf/lib/rex/post/meterpreter/packet_dispatcher.rb:154:in `send_packet'
/Users/sinn3r/rapid7/msf/lib/rex/post/meterpreter/packet_dispatcher.rb:211:in `send_packet_wait_response'
/Users/sinn3r/rapid7/msf/lib/rex/post/meterpreter/packet_dispatcher.rb:187:in `send_request'
/Users/sinn3r/rapid7/msf/lib/rex/post/meterpreter/channel.rb:309:in `interactive'
/Users/sinn3r/rapid7/msf/lib/rex/post/meterpreter/ui/console/interactive_channel.rb:48:in `_suspend'
/Users/sinn3r/rapid7/msf/lib/rex/ui/interactive.rb:259:in `block in handle_suspend'
/Users/sinn3r/.rvm/gems/ruby-2.1.5@metasploit-framework/gems/rb-readline-0.5.1/lib/rbreadline.rb:4482:in `call'
/Users/sinn3r/.rvm/gems/ruby-2.1.5@metasploit-framework/gems/rb-readline-0.5.1/lib/rbreadline.rb:4482:in `read'
/Users/sinn3r/.rvm/gems/ruby-2.1.5@metasploit-framework/gems/rb-readline-0.5.1/lib/rbreadline.rb:4482:in `rl_getc'
/Users/sinn3r/.rvm/gems/ruby-2.1.5@metasploit-framework/gems/rb-readline-0.5.1/lib/rbreadline.rb:4530:in `rl_read_key'
/Users/sinn3r/.rvm/gems/ruby-2.1.5@metasploit-framework/gems/rb-readline-0.5.1/lib/rbreadline.rb:4705:in `readline_internal_charloop'
/Users/sinn3r/.rvm/gems/ruby-2.1.5@metasploit-framework/gems/rb-readline-0.5.1/lib/rbreadline.rb:4801:in `readline_internal'
/Users/sinn3r/.rvm/gems/ruby-2.1.5@metasploit-framework/gems/rb-readline-0.5.1/lib/rbreadline.rb:4823:in `readline'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/input/readline.rb:132:in `readline_with_output'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/input/readline.rb:86:in `pgets'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/shell.rb:184:in `run'
/Users/sinn3r/rapid7/msf/lib/rex/post/meterpreter/ui/console.rb:66:in `interact'
/Users/sinn3r/rapid7/msf/lib/msf/base/sessions/meterpreter.rb:412:in `_interact'
/Users/sinn3r/rapid7/msf/lib/rex/ui/interactive.rb:49:in `interact'
/Users/sinn3r/rapid7/msf/lib/msf/ui/console/command_dispatcher/core.rb:1800:in `cmd_sessions'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/dispatcher_shell.rb:427:in `run_command'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/dispatcher_shell.rb:389:in `block in run_single'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/dispatcher_shell.rb:383:in `each'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/dispatcher_shell.rb:383:in `run_single'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/shell.rb:200:in `run'
/Users/sinn3r/rapid7/msf/lib/metasploit/framework/command/console.rb:38:in `start'
/Users/sinn3r/rapid7/msf/lib/metasploit/framework/command/base.rb:82:in `start'
./msfconsole:48:in `<main>'
  • And at this point your meterpreter session is already bad. It will still be listed as a session, but you won't be able to interact with it:
msf exploit(handler) > sessions -i 4
[*] Starting interaction with 4...
OJ commented

This is not cool.

  • 😄1
bcook-r7 commented

An interesting other behavior is that a 64-bit meterpreter will, for odd sessions, throw an error and spawn a new session, and for even sessions, actually background them.

https://gist.github.com/bcook-r7/88fdc649cb72d3bd34f7

OJ commented

I think I might know what this is related to.

OJ commented

Ruby 1.9, Windows 7 SP1 x64 with 32bit Meterpreter

msf exploit(handler) > run

[*] Started reverse handler on 10.1.10.40:8000 
[*] Starting the payload handler...
[*] Sending stage (770048 bytes) to 10.1.10.42
[*] Meterpreter session 1 opened (10.1.10.40:8000 -> 10.1.10.42:49340) at 2015-01-08 16:05:25 +1000

meterpreter > shell
Process 1016 created.
Channel 1 created.
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

Z:\scratch\meterpreter\10.1.10.40>^Z
Background channel 1? [y/N]  y
meterpreter > channel -l

    Id  Class  Type
    --  -----  ----
    1   3      stdapi_process

meterpreter > channel -i 1
Interacting with channel 1...

whoami
whoami
win-s45guq5kgvk\oj

Z:\scratch\meterpreter\10.1.10.40>^Z
Background channel 1? [y/N]  y
meterpreter > DAMMIT!
[-] Unknown command: DAMMIT!.
OJ commented

Ruby 1.9, Windows 2012 with 32bit Meterpreter

msf exploit(handler) > run

[*] Started reverse handler on 10.1.10.40:8000 
[*] Starting the payload handler...
[*] Sending stage (770048 bytes) to 10.1.10.11
[*] Meterpreter session 2 opened (10.1.10.40:8000 -> 10.1.10.11:61817) at 2015-01-08 16:07:14 +1000

meterpreter > sysinfo
Computer        : PDC2012
OS              : Windows 2012 (Build 9200).
Architecture    : x64 (Current Process is WOW64)
System Language : en_US
Meterpreter     : x86/win32
meterpreter > shell
Process 4072 created.
Channel 1 created.
Microsoft Windows [Version 6.2.9200]
(c) 2012 Microsoft Corporation. All rights reserved.

C:\Users\OJ Reeves\Desktop>^Z
Background channel 1? [y/N]  y
meterpreter > channel -i 1
Interacting with channel 1...

whoami
whoami
pwnage\oj reeves

C:\Users\OJ Reeves\Desktop>^Z
Background channel 1? [y/N]  y

meterpreter >
OJ commented

Not seeing a breakage, I'll change over to Ruby 2.1 and see what happens.

OJ commented

Bingo! As soon as I switch to Ruby 2.1.5 I get the problem:

msf exploit(handler) > exploit

[*] Started reverse handler on 10.1.10.40:8000 
[*] Starting the payload handler...
[*] Sending stage (770048 bytes) to 10.1.10.11
[*] Meterpreter session 1 opened (10.1.10.40:8000 -> 10.1.10.11:61930) at 2015-01-08 16:27:25 +1000

meterpreter > shell
Process 2148 created.
Channel 1 created.
Microsoft Windows [Version 6.2.9200]
(c) 2012 Microsoft Corporation. All rights reserved.

C:\Users\OJ Reeves\Desktop>^Z
Background channel 1? [y/N]  y
[-] Failed to spawn shell with thread impersonation. Retrying without it.
Process 3908 created.
Channel 2 created.
Microsoft Windows [Version 6.2.9200]
(c) 2012 Microsoft Corporation. All rights reserved.

C:\Users\OJ Reeves\Desktop>
OJ commented

I could see the Error running command shell: ThreadError can't be called from trap context message appear when I remove the begin/rescue/end block from around cmd_execute, but I have no idea why this might be the case. I'm going to have to lean on something who is much more savvy with Ruby here.

I don't think this is something that Meterpreter is doing wrong given that Ruby 1.9.x is fine. What do you think @wchen-r7 ?

OJ commented

What do the likes of @hmoore-r7 @jlee-r7 @limhoff-r7 and @todb-r7 think?

wchen-r7 commented

@OJ I don't know much about the problem, but yeah this seems to be a Ruby 2.x specific problem. It's really weird that when I press [CTRL]+[Z], it actually wants to spawn another shell, which is clearly the opposite of what I want. I added a puts caller.inspect right after the "Failed to spawn shell with thread impersonation" error, and I got this:

/Users/sinn3r/rapid7/msf/lib/rex/post/meterpreter/ui/console/command_dispatcher/stdapi/sys.rb:245:in `cmd_shell'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/dispatcher_shell.rb:427:in `run_command'
/Users/sinn3r/rapid7/msf/lib/rex/post/meterpreter/ui/console.rb:104:in `run_command'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/dispatcher_shell.rb:389:in `block in run_single'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/dispatcher_shell.rb:383:in `each'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/dispatcher_shell.rb:383:in `run_single'
/Users/sinn3r/rapid7/msf/lib/rex/post/meterpreter/ui/console.rb:68:in `block in interact'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/shell.rb:190:in `call'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/shell.rb:190:in `run'
/Users/sinn3r/rapid7/msf/lib/rex/post/meterpreter/ui/console.rb:66:in `interact'
/Users/sinn3r/rapid7/msf/lib/msf/base/sessions/meterpreter.rb:412:in `_interact'
/Users/sinn3r/rapid7/msf/lib/rex/ui/interactive.rb:49:in `interact'
/Users/sinn3r/rapid7/msf/lib/msf/ui/console/command_dispatcher/core.rb:1800:in `cmd_sessions'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/dispatcher_shell.rb:427:in `run_command'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/dispatcher_shell.rb:389:in `block in run_single'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/dispatcher_shell.rb:383:in `each'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/dispatcher_shell.rb:383:in `run_single'
/Users/sinn3r/rapid7/msf/lib/msf/ui/console/command_dispatcher/exploit.rb:148:in `cmd_exploit'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/dispatcher_shell.rb:427:in `run_command'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/dispatcher_shell.rb:389:in `block in run_single'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/dispatcher_shell.rb:383:in `each'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/dispatcher_shell.rb:383:in `run_single'
/Users/sinn3r/rapid7/msf/lib/rex/ui/text/shell.rb:200:in `run'
/Users/sinn3r/rapid7/msf/lib/metasploit/framework/command/console.rb:38:in `start'
/Users/sinn3r/rapid7/msf/lib/metasploit/framework/command/base.rb:82:in `start'
./msfconsole:48:in `<main>'

That doesn't look like the right code path for doing [CTRL]+[Z], does it?

I'm not familiar with the meterpreter code, but I speculate that something in Ruby 2.x is causing a condition to change (maybe a simple if...else condition), so some method in Metasploit ends up taking the opposite code path. I once caught a respond_to? bug that was causing this kind of behavior, so that's the first thing that comes to my mind. It's just something that's possible, but I could be completely wrong.

OJ commented

The code tries to spawn a shell when an exception is caught. So clearly we need to do something smarter there. Because any exception results in a new shell rather than the one case we're actually interested in. Some work can definitely be done there.

That doesn't explain the ThreadError though :(

todb-r7 commented

this seems to be a Ruby 2.x

This implies there should be an accompanying bug on rapid7/metasploit-framework

hdm commented

This popped up Ruby 1.9.x a bit as well, it all boils down to the way signal handling has been semi-broken in threaded Ruby. The real fix would be to replace ^C/^Z with some kind of command escape sequence (!!kill, !!background, etc).

sempervictus commented

I can confirm that the issue is present with 1.9, but this is an MRI-specific problem. When running on RBX 2.5.2 i've not noticed this issue, though i did need to wire in signal trapping to the console as there were some issues with readline IIRC.

I would vote that we keep the quick keyboard shortcuts - quick session traversal can be very useful when you end up on the reactive side of an engagement. Might make sense to use some less frequently used key combinations (signals), but its a useful feature from the CLI standpoint to have the shortcut.

Meterpreter shell, by the way, seems to handle these inputs differently from the rest of framework. Maybe we should look to unify console CLI input handling with a consumer thread responsible for all inputs, signals, traps, etc?