RTST CTD Radeon Pro D3D (non)compatibility - and potential "game render" issue in OBS

Edit: crap, forgot my system specs:

Windows 8.1 x64 (DxDiag file link: http://goo.gl/I1CKIE)

RTST # 0.10.1.4

OBS # 0.638b (I'm a version behind because I use the AMD VCE plugin that isn't always immediately up to date)

So, after getting the Beta 10 installed, I didn't notice any difference. I would still CTD, whether or not I was running OBS at the time (but still running RTST). I'm an AMD GPU user, and I have Radeon Pro installed so I can use the "alternative frame render" on my xfire setup for PS2 (to remove microstutter). Even though I have unchecked anything to do with monitoring in Radeon Pro, I still had to make an API exception for PS2. I can have afterburner (without RivaTuner installed) and Bandicam running at the same time no problem now. So my issue seems to be strictly Radeon Pro. So here's what I did:

http://i.imgur.com/edPJULS.png

That seems to be fixed now. I've run PS2 multiple times, with OBS/RTST in different starting sequences, starting them before the game or after the game has launched all works just fine.

Now, over to what is likely an issue with OBS Game Capture, or RTST's hook working with it. The game runs fine, and RTST runs fine, but OBS does not capture RTST's in-game HUD. This only happens when using Game Capture though. Here's two screenshots, one using Game Capture and one (clearly showing) windowed mode and Window Capture.

Game Capture
http://i.imgur.com/2GKuUXOl.jpg

Window Capture
http://i.imgur.com/5i0bbbbl.jpg

As should be visible, the only thing not drawn on OBS (to the right) is the in-game HUD. Whether or not this is by design, I don't know. I would assume it's not though, since it's worked before. I also checked to see if OBS picked up RTST as its own D3D layer, it did not. I'm not sure what would be required to figure out what's wrong at this point, if anything is wrong. But I'd be more than happy to follow some steps that could point in the right direction.

Thank you for the detailed description and taking the time to figure out a solution to your problem-- I don't have an AMD card to test RTST with myself. I'll also probably end up linking your post in our troubleshooting section.

Getting OBS to actually pick up RTST's overlay using game capture is difficult and requires a specific order to work and if it does work will often break right after an alt-tab. OBS's hook is likely triggering first and capturing the frame data before RTST draws its information.

One thing I have been slowly working on is improvements with the hooking process which can reduce crashes and help compatibility with other programs. Finishing this would likely fix your original crash issue all together without having to add an exception for ps2. It could potentially help with obs as well, and any other software which has had problems while running with RTST. The real problem ends up being this in itself is one of the most time consuming and complex systems to develop, dealing with largely low-level operations. I responded here with some more information about an improved hook.

My Apologies for resurrecting an old thread, but what did the OP do to fix his problem with RadeonPRO and Recursion? The Pictures the OP posted are no longer available. I use RPRO's FXAA and I want it to run with Recursion... If it's possible :P

Thank You for your Time :)