I am using an ImageOverlay to present UAS video on a 3D SceneView.
Whilst this works, I am finding it very resource intensive to convert every frame of video from a System.Drawing.Bitmap to an ImageFrame that I can assign to the ImageOverlay ImageFrame property.
In particular, when working with in memory images, the only constructor for the ImageFrame that I can use seems to be the one that takes a RuntimeImage, and the only constructor I can use for that takes a byte array.
So as every frame of video is received I have to do the following:
using (var ms = new MemoryStream())
{
VideoFrameBitmap.Save(ms, ImageFormat.Bmp);
ImageAsBytes = ms.ToArray();
}imageOverlay.ImageFrame = new ImageFrame(new RuntimeImage(ImageAsBytes), new Polygon(points));
The call to VideoFrameBitmap.Save method is particularly processor intensive, and means I have to skip frames when processing 4K video. Note I have tried ImageConverter.ConvertTo but it is actually around 6 times slower than using the memory stream!
So my question is, is there another way to update the ImageFrame on an ImageOverlay, from a Bitmap source without having to go through a conversion to a byte Array?
There is RuntimeImage.FromStreamAsync(stream) that you can use, but under the covers it unfortunately have to do the same thing, so you're really not saving much. I would however recommend trying JPEG or PNG so you reduce the byte-buffer size, and how much needs to be sent into the native library - while that will create a small overhead for encoding and decoding, it might overall create a small boost in less memory allocation and interop overhead.