[bounty] $1000 make RDP work
Opened this issue · 13 comments
current status:
i did a large part of the work here:
#686
but something is not working:
screenshots are showing my own screen instead of the connected users' screen
i properly am able to list sessions, get tokens, etc. but then it fails - tried both in python and in rust, same issue
definition of done:
make one of those bins work
eg:
- screenpipe runs on a Windows 22 server
- n clients connect to it through Microsoft Remote Desktop, each seeing their own user/session/screen
- screenpipe capture each client screen/session (and not the local server screen)
you can send a new PR if you want - copy pasting my code and making it work
i can do the plumbing to the rest of the code (which is the easy part)
PS: it's a screenpipe enterprise solution, not related to our main codebase
/bounty 1000
💎 $1,000 bounty • Screenpi.pe
Steps to solve:
- Start working: Comment
/attempt #850with your implementation plan - Submit work: Create a pull request including
/claim #850in the PR body to claim the bounty - Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts
Thank you for contributing to mediar-ai/screenpipe!
Add a bounty • Share on socials
| Attempt | Started (GMT+0) | Solution |
|---|---|---|
| 🟢 @nikhilxe | Dec 6, 2024, 1:17:26 PM | WIP |
current status:
i did a large part of the work here: #686
but something is not working:
screenshots are showing my own screen instead of the connected users' screen
i properly am able to list sessions, get tokens, etc. but then it fails - tried both in python and in rust, same issue
definition of done:
make one of those bins work
eg:
* screenpipe runs on a Windows 22 server * n clients connect to it through Microsoft Remote Desktop, each seeing their own user/session/screen * screenpipe capture each client screen/session (and not the local server screen)you can send a new PR if you want - copy pasting my code and making it work
i can do the plumbing to the rest of the code (which is the easy part)
PS: it's a screenpipe enterprise solution, not related to our main codebase
/bounty 1000
@louis030195 So if I understand correctly, one of the 3 users connects to the RDP server but does whatever work on their local machine, and screenpipe in the server will capture what each user is doing in their local machine and register it separately.
Doesn't this mean the server will need to be able to see the client's screen? Then wouldn't we need to do it the other way around? I.e. the server connects through RDP to the user machines, and captures the RDP contents.
@Neptune650 there is supposedly an API to just get the stream of user screen
because most of the time cloud desktop, like cloud gaming, etc. is just streaming the server OS to the local computer
@Neptune650 there is supposedly an API to just get the stream of user screen
because most of the time cloud desktop, like cloud gaming, etc. is just streaming the server OS to the local computer
@louis030195 btw, the "rdp2" program and screenpipe both run on the server? Or rdp2 runs on the client? We can talk over telegram if you'd like.
/attempt #850
| Algora profile | Completed bounties | Tech | Active attempts | Options |
|---|---|---|---|---|
| @nikhilxe | 1 bounty from 1 project | C++, JavaScript, CSS & more |
Cancel attempt |
@louis030195 I don't think it would be straight-forward due to Windows restrictions.
My only guess to achieve this is, we need to add run-as-service feature to the app, then run/install rdp2.exe as a service eg. rdp2.exe --service. Listen to user logon events, and on every new user logon, execute itself i.e rdp2.exe again, but in the respective user session context.
So there will be one process running as service and then one process each for every user session.
rdp2.exe --service (Service)
|_ rdp2.exe (User 1)
|_ rdp2.exe (User 2)
|_ rdp2.exe (User 3)
This approach should definitely work
PS: Also need to ensure, rdp2.exe is in a Path that is accessible to other users as well, like %PROGRAMFILES%, and have execute permission for user group.
@Neptune650 there is supposedly an API to just get the stream of user screen
because most of the time cloud desktop, like cloud gaming, etc. is just streaming the server OS to the local computer@louis030195 btw, the "rdp2" program and screenpipe both run on the server? Or rdp2 runs on the client? We can talk over telegram if you'd like.
yes i just run rdp2 or rdp on the server
https://screen-record.com/ is able to record users' sessions without having to do anything on the client side
other thing to try https://gist.github.com/louis030195/e17d4b3c637928b89358c6816e461150
other thing to try https://gist.github.com/louis030195/e17d4b3c637928b89358c6816e461150
@louis030195 This solution most probably won't work from my experience¹, since RDP itself uses Windows Desktop Duplication API behind the scenes, and it takes exclusive control, preventing other applications from accessing the duplication API while the RDP session is active.
¹ Unless they have changed this behaviour in the latest Windows updates.
other thing to try https://gist.github.com/louis030195/e17d4b3c637928b89358c6816e461150
@louis030195 This solution most probably won't work from my experience¹, since RDP itself uses Windows Desktop Duplication API behind the scenes, and it takes exclusive control, preventing other applications from accessing the duplication API while the RDP session is active.
¹ Unless they have changed this behaviour in the latest Windows updates.
@louis030195 Antony is right; to actually record the user’s sessions, you would need a service in users’ client. Honestly, we can try to use fibers for that purpose to not be detected by kernel, thus not showing up in the process window. Fibers are a legit security issue in the Windows design part, but it can be used like this. But it needs to be attached to a process to work. So, I am not sure if it's possible with sandboxing of modern applications, and this goes straight into malware territory. But I think they just record the user by not showing themselves in the process manager using a similar method, then send the data in-real time.
To record multiple remote desktop sessions on a server like Windows Server 2022, you can capture each session's screen individually. This is done using the server's built-in tools to record each user's desktop separately. Thus, the cloud gaming.
@Neptune650 there is supposedly an API to just get the stream of user screen
because most of the time cloud desktop, like cloud gaming, etc. is just streaming the server OS to the local computer@louis030195 btw, the "rdp2" program and screenpipe both run on the server? Or rdp2 runs on the client? We can talk over telegram if you'd like.
yes i just run rdp2 or rdp on the server
https://screen-record.com/ is able to record users' sessions without having to do anything on the client side
other thing to try https://gist.github.com/louis030195/e17d4b3c637928b89358c6816e461150
(*dxgi_factory).EnumAdapters1(0, &mut adapter);
if adapter.is_null() {
return Err("Failed to find DXGI Adapter".into());
}
You are trying to access the wrong driver here. You need to use Microsoft's "Remote Display" driver instead. It wouldn't make sense to try to reach the physical driver of another device from your device.
return Err("Failed to duplicate output".into());
Duplication is not enabled by default for remote displays. Because simply, there is no "real" desktop to capture, and since it will 100% of the time fail, it's going to return your own physical desktop because of AcquireNextFrame
I wasn't sure if I should /-attempt on this, as someone else already started. However, I'll provide feedback anyways.
@zelosleone not sure to understand,
To record multiple remote desktop sessions on a server like Windows Server 2022, you can capture each session's screen individually. This is done using the server's built-in tools to record each user's desktop separately. Thus, the cloud gaming.
what do you mean by that? which API is used in this case?
@zelosleone not sure to understand,
To record multiple remote desktop sessions on a server like Windows Server 2022, you can capture each session's screen individually. This is done using the server's built-in tools to record each user's desktop separately. Thus, the cloud gaming.
what do you mean by that? which API is used in this case?
https://learn.microsoft.com/en-us/windows/win32/termserv/terminal-services-api
WTSVirtualChannelQuery and WTSQuerySessionInformationW via Remote Desktop Services API
@zelosleone that's exactly what i did first but it was only capturing local machine instead of users: