Saturday, December 26, 2020

Screen capture sample, clarified

Few years ago, Microsoft introduced so called UI Composition API, and even provided a dedicated GitHub repository for that
And as usual, "just a code" is not always enough. Especially if we're talking about an API that is not that documented. Now, it's time to close some gaps about screen sharing API.

First thing first, here is the sample:
You can build it, run it, use it. Let's try to understand it more!
After you evaluated provided SimpleCapture class, you would see the list of steps required:
  • Call CreateCaptureItemForWindow() or CreateCaptureItemForMonitor()
  • Create Direct3D device (yes, we use Direct3D for that obviously 2D graphics, weird thing)
  • Create a frame pool that will receive captured frames (Direct3D11CaptureFramePool class) from the captured item
  • Create a session for the capturing via the frame pool (GraphicsCaptureSession class)
  • Call StartCapture() method for the session
  • Now, TryGetNextFrame() method is at your service to read a stream of captured frames at any moment(s) you need it
So far so good? Not so fast. After you try to integrate that approach at very your code, you may ask yourself: how should I create the frame pool? Surprisingly, there are two options: For your convenience, I placed URLs to the manual. "Creates a frame pool...", "Creates a frame pool where the dependency on the DispatcherQueue is removed...". Very obvious, heh.

Another question: should you query your pool with TryGetNextFrame() called explicitly by hand, or subscribe to the pool's FrameArrived event and call TryGetNextFrame() there instead? Or maybe both options? Not so apparently as well.
Despite all that open-source movements Microsoft did last years, Composition UI is not open-sourced, at least yet. Let's investigate the API by series of educated guesses and science-based pokes!

And after some playing around, you could figure out that...

  • ... if you use Direct3D11CaptureFramePool::Create() method to instantiate your pool, then you can call TryGetNextFrame() by the same thread to receive captured frames. Either, you can subscribe to FrameArrived event and call TryGetNextFrame() inside the event handler (however, your thread should implement message looping technique to be able to receive such events). Regardless the approach you choose, you stay in the scope of the same thread.
  • ... if you use Direct3D11CaptureFramePool::CreateFreeThreaded() method to instantiate your pool, any attempt to call TryGetNextFrame() by the same thread will return null as a result. This case, you must subscribe to the pool's FrameArrived event to receive captured frames. And beware, the event will be triggered in scope of a worker thread (the API will create it for you automatically). So, do not forget all that multi-threading synchronization precautions while sharing the data across different threads.

As a summary, I would say Microsoft created very good API, so come and use it.

No comments:

Post a Comment