Implement GDI+Drawing Orders

[Res260]

Recent versions of RDP use GDI+ for graphic rendering (although PyRDP MITM disables most of it, it might not be possible in future RDP versions). This library is written by Microsoft, but a Unix implementation exists: https://www.mono-project.com/docs/gui/libgdiplus/

The plan would be to do a wrapper for this library in Python to call the needed methods in the PyRDP Player.

[obilodeau]

Note to future self: Wine would also be a library to consider. It has GDI drawing for sure.

[alxbl]

I've started looking into this. This will be a multi-step process but I can see it helping a lot with MITM performance (ref: #162, #161).

Implementation Roadmap

• Add a GDI passthrough option that doesn't downgrade the video to bitmap
• Add MS-RDPEGDI PDU parsing support (protocol level only, no rendering)
• Figure out what we're going to use to render GDI and handle the caches
• Implement GDI->Qt Adapter

This could take a while to do, especially when trying to avoid extra dependencies, at the very least, we should try to make the GDI dependencies optional once we get to that point. For now I'm focusing on adding support for GDI passthrough, as this is a quick win and still lets us intercept keyboard, clipboard, and crawl shared drives.

[Res260]

@alxbl take note that a client that supports GDI drawing orders should always choose it over bitmaps, so enabling gdi support without parsing the graphics could mean that almost no connection will have a replay with graphics.

That said, if done correctly, gdi drawing orders pdus could be saved in the replay so graphics can be reconstructed after the graphic rendering is implemented.

[alxbl]

Today I made a proof-of-concept to test the impact of letting video passthrough in GDI. It's very straight forward and appears to be well-supported by pyrdp-player since the bitmap PDUs are never sent and the MITM is not aware of the GDI PDUs, so no parsing is even attempted.

This appears to be well-supported on both mstsc and FreeRDP, so it's promising.

[alxbl]

@Res260 Yes, indeed. At the moment this would be a trade-off for people that are interested in possibly keylogging, and file exfiltration, but would not support graphical replay. I am planning to make sure that all PDUs are saved so that once we support GDI, saved replays should work. This is going to be best effort though since no rendering testing will be possible until much further down the line.

EDIT: I haven't tested it, but theoretically, payloads should still be supported, making the tool work well for lateral movement.

[alxbl]

So after a bit of empirical testing, it looks like GDI is going through the Fast-Path first and foremost.

I'll spend more time reading the spec to confirm, and start working on some initial parsing code.

[Res260]

Let me know if you need guidance for how to implement a parser and different PDUs. It should be straightforward if you look at other parsers/pdus

[alxbl]

Interestingly, it looks like TS_DRAW_GDIPLUS_CAPABILITYSET is never sent by either server or client on Windows 10. I can't find anything about the message being deprecated or unused, either. I wanted to do a quick sanity check before diving into writing parser code, but it looks like for now I'll just focus on implementing the TS_FP_UPDATE_ORDERS path and individual drawing orders. I do see update orders being sent when the order capability is present, so something special must be happening. On the plus side, the capability parsing code is already there.

I spent a long time reading the spec yesterday night and this will be a lot of code, even just for parsing, since a lot of the messages have compression built-in.

I'm going to look at the FreeRDP license and whether it would be feasible to borrow some of their parsing code to make this faster.

[Res260]

Keep in mind that PyRDP downgrades the RDP version used, so it might be why you don't see it. Make sure that the capabilityset pdu is sent in the current PyRDP's RDP version!

…

On Tue, 25 Feb 2020 at 10:39, Alexandre Beaulieu ***@***.***> wrote: Interestingly, it looks like TS_DRAW_GDIPLUS_CAPABILITYSET is never sent by either server or client on Windows 10. I can't find anything about the message being deprecated or unused, either. I wanted to do a quick sanity check before diving into writing parser code, but it looks like for now I'll just focus on implementing the TS_FP_UPDATE_ORDERS path and individual drawing orders. I do see update orders being sent when the order capability is present, so something special must be happening. On the plus side, the capability parsing code is already there. I spent a long time reading the spec yesterday night and this will be a lot of code, even just for parsing, since a lot of the messages have compression built-in. I'm going to look at the FreeRDP license and whether it would be feasible to borrow some of their parsing code to make this faster. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#50?email_source=notifications&email_token=ADPMNLYRM2KSWWYWWXHYNTLREU3TJA5CNFSM4GJQQJC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEM4N7PY#issuecomment-590929855>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADPMNL3T37CJEPTJLSGT4M3REU3TJANCNFSM4GJQQJCQ> .

[alxbl]

@Res260 Thanks, I'm aware that several messages were modified to downgrade the server's capabilities. I current have GDI traffic going through despite the (puzzling) lack of that capability. Everything else looks fine though, so I can continue progress on the GDI parsing code.

I'm torn between taking and adapting the FreeRDP code or porting it to pure Python(*). Both would probably take a similar amount of effort, and regardless if we want to render to Qt surfaces, we'll need to re-implement the actual GDI blitting functions for Qt's surfaces.

There's probably at least a month's worth of effort in either case.

(*) Pure python will likely require some C-modules similar to rle for performance reasons, but I'll see about that where I get there.

[obilodeau]

Can't we do pure passthrough for now? Like send the bytes for that channel directly to the server and send whatever the server replies directly to the client. No parsing. If it's not possible, I would like a detailed explanation of why.

[alxbl]

That's already done in the gdi-support branch. I can make merge those changes immediately and continue on a different branch for GDI parsing if you'd like.

[obilodeau]

Rename that branch gdi-passthrough, put the activation of that feature behind a CLI flag (both pyrdp-mitm.py and the twistd plugin) and submit that PR for review. Then resume the current work in the gdi-support branch. Thanks!

I would like to throw GDI passthrough against regular bitmap downgrade on the Internet and compare how scanners behave regarding each.

[alxbl]

The twisted plugin? The change doesn't involve a twisted plugin so I'm not sure what you mean.

I'll rename the branch, update the CHANGELOG and open the PR.

[obilodeau]

If you change the CLI, the twistd plugin also parses options and needs to be updated. There's a comment at the top of the argparse stuff that talks about keeping that in sync.

[alxbl]

Hmm, I just saw that comment. I'll update the PR.

[alxbl]

Quick update on this... the protocol parsing is going well. I expect to be done by the end of next week. Once it's done, I'll do some tests on a live session to see the performance impact of parsing. In reality, though, it will most likely not be enabled on the MITM server, as we do not support video injection.

Once I'm satisfied that the parser is working properly, the next steps will be to double check that all of the order messages are being properly recorded and sent to the player and then finally start working on GDI cache management and executing of the orders.

[alxbl]

I'm at a bit of a cross-road now.

I'm pretty much done with the GDI parsing (minus handling of fragmented packets) but there's one thing left to address: Secondary drawing orders for Glyph caching need to know the capabilities reported by the client during the initial connection.

Currently, this state is discarded in SlowPathMITM.py after onConfirmActive. I would like @xshill and @Res260's input as to your preferred approach to send that state to downstream parsers. I'm currently looking at adding a clientCapabilities and serverCapabilities to the MITM state class that would track the respectively reported capabilities. Does this sound like the right approach? If state starts to feel bloated, we can eventually refactor it into sub objects (i.e. MITM context, MITM configuration, and session context)

Thoughts?

[Res260]

We DMed and the conclusion was that GDI parsing should not be done (or minimally be done) on the MITM, the MITM should only save GDI packets so they can be parsed and rendered during replay.

[alxbl]

To clarify: It will not be done by the MITM (and was never intended to), but the current implementation is done in the MITM to be able to test relatively quickly. The player has a different pipeline to receive the capabilities and the required state will be added there.

[alxbl]

I've migrated the code to the player now. and Drawing Order PDUs are already being saved, as I suspected. Meaning that existing GDI traces SHOULD be playable.

Another interesting I noticed while starting initial work on the frontend is that my test environment only seems to use a very limited subset of drawing orders:

• FrameMarker
• MemBlt
• CacheBitmapV2

I'm fairly certain that this is a side effect of that GDI capability set not being present. I'll test more tomorrow with a Windows to Windows RDP session and playing wit the mstsc client settings as well as the capabilities that we are tampering. In any case, this will let me start working on a limited subset of the rendering that should be easily testable before expanding to the rest of the EGDI pipeline.

Things are progressing well.

[alxbl]

All of the PDU parsing is done (but mostly untested), and I've started work on the initial Qt frontend for GDI operations.

I'm going to focus on the 3 operations from my previous comment, as an initial step to make sure that we can get some proper rendering going. After that I'll try to get some sample sessions which make use of more advanced features so that I can implement and test in parallel.

[alxbl]

Writing an Adapter between GDI and Qt's QPainter looks like it's going to be fairly straight forward since most of the raster ops and required drawing functions are already supported. The only thing I'm still worried about is glyph rendering, since Qt doesn't seem to accept raw glyphs.

As a rudimentary proof of concept, I was able to get the player to render a session which only uses FrameMarker, MemBlt and CacheBitmapV2. This is a very promising result, as I wasn't expecting to get to this point so quickly.

With that in place, I'm going to start working on the GDI->Qt adapter.

[Res260]

damn, big hype

[alxbl]

I figured out where the other drawing orders went... if we downgrade to <=24bpp, only MemBlt, ScrBlt and CacheBitmapV2 are used. In 32bpp, the full range seems to be used (well, as far as I can tell, since my parser just exploded instantly :) )

This will be a good chance to work on fixing any parser bugs in parallel with developing the rendering of those primitives. As an added bonus we will probably have proper 32 bit support once this is done.

My only worry is that there might be some currently unimplemented codecs that show up along the way. If that happens, I might make an initial release of drawing orders with only <= 24bpp support, and continue working on full 32 bit support later on.

I will also need to do something about the persistent cache in the MITM to have a way from retrieving cache entries that the client already has.

[alxbl]

My last update was slightly mistaken. Using 32bpp revealed some bugs in my parser, but once fixed, the drawing orders still consisted only of CacheBitmapV3, MemBlt and ScrBlt. I have no idea how to get the other drawing orders to be used.

In any case, I'll write some untested code to support the remaining drawing orders, and if we ever find a dataset to test against, we can iron out the bugs that will most likely be found.

As for the persistent cache, I've made an initial attempt at forcing the server to send us the cache entries. It works by spoofing Persistent Key List PDU to have 0 cache key entries in them. This is rudimentary and will cause a re-caching to occur on the client side, so it can be detected by a smart scanner. Eventually we can improve the code to remember the client's cache state and drop those caching orders that the client doesn't need once the PyRDP cache is up to date.

TL;DR: The currently implemented MS-RDPEGDI drawing order subset for 16/24bpp appears to be working. I'll continue working on the Qt frontend and hopefully we can find a good data set for other drawing orders.

[alxbl]

Good news. I have most of the code written now. The only drawing orders that are not supported are the NineGrid, Glyph and GDI+ opaque record orders. I am not planning to support them for the time being, as I was not able to find an RDP client/server that uses them. In fact, most of the vector graphics drawing orders do not appear to be used either, but I've implemented them for completeness and to get the GDI->Qt interop figured out as much as possible.

If we do find a good data set to test against, I am sure there will be a lot of bugs to fix in the untested parsers and drawing orders. For the time being I propose keeping this whole PR gated behind the --gdi switch. People that use it would do so at the risk of their video not being immediately viewable.

My plan is to take a break from this to clear up my mind a little and do a thorough cleanup/refactor pass to make sure the code is structured in a way that I'm happy with.
I'll also be closing this PR and re-opening with a rebased/clean history that keeps only meaningful milestone commits for merging purposes.

[alxbl]

@Res260 was able to get a replay between a Windows 10 client and Windows 7 server which appears to use different drawing orders. This will be a good chance to do some testing and stabilization work. I'll be going through it on Monday.

[alxbl]

As expected, the Windows 7 RDP Server is making use of a much wider range of drawing operations. I'll start by ironing out all of the bugs for the trace provided by @Res260 and then setup a VM to test on Windows 7 and Windows 8, to be sure that we have everything working.

This is great news :)

My only worry right now is that glyph orders are being used, and I'll have to figure out a good way to make this work with Qt.

[alxbl]

NineGrid support requires RDP6.0 Bulk Compression support. I'll see if it's possible to just downgrade the server's DemandActive to prevent NineGrid orders. This would be a big chunk of work to implement.

Other than glyph rendering, it seems that Qt's composition modes are not behaving the way I would expect them to, so this will require a significant amount of debug work.

Windows 10 Servers should be supported now, though.

[alxbl]

Disabling NineGrid worked. Since RDP streams the glyphs as bitmaps, the Qt text management API will not be usable. It doesn't look like too much work to add glyph rendering support, though, so I'll proceed that way.

There are only a few things left to take care of... namely:

• Glyph Rendering Support
• Improve/Fix Qt Blending & Composition Modes (GDI->Qt adapter)
• Test with Windows 8 and Windows 2012
• Documentation
• Code Clean up

[alxbl]

Good news! Windows 7 with most drawing orders work well enough that I feel the feature is ready to enter cleanup, testing, and documentation phase.

There are still a few non-blocking rendering artifacts that I would like to troubleshoot, but those can be done separately. I'm going to spend today reviewing the code and making any necessary changes. Then I'm going to re-base the PR on master and clean up the commit history.

I'll still need to test on a Windows 8 and 2012 server to make sure nothing critical has been missed.

The feature will remain gated behind --gdi for the time being.