3

I have a task to create an augmented reality application which has a GUI similar to this one (the overlay route and the FollowMe concept). The interesting part of the task is that the application should be done on a desktop OS (Debian/Ubuntu) with the help of the Qt framework. Python is the preferred language for the application.

So, the big difference to the aforementioned mobile app is that my application will use a USB camera plugged into a PC.

I have tried a few different approaches and none of them seems to satisfy the requirements:

  • this example does not work;
  • this solution uses a 'two layers' approach, but when I try to render the upper widget it fully covers the bottom widget;
  • this solution works, but it has 2 disadvantages: OpenCV's main usage is not video stream rendering (wrong tool, I guess GStreamer could fit better) and also it renders only simple 4-point shapes and it does not allow any complex polygons with holes;

I have also skimmed through the Nokia's Qt camerality source code, but it seems that this problem is solved on Symbian in a completely different way which is inapplicable on PC (embedded camera API vs a generic USB camera on PC).

I tend to think that the problem can be divided into two questions:

  • What is the best way to render a camera video stream in a Qt PC application?
  • What is the best way to overlay an animated 3D image (probably OpenGL generated) over a camera video stream in a Qt PC application?
Community
  • 1
  • 1
Alexey Rusakov
  • 29
  • 2

1 Answers1

0

If you are targeting a desktop I would use openGL for the whole process.
Use a quad in the background with a texture streaming the video (ie draw each video frame to the openGL texture) then render all your 3D model in openGL ontop of this.

Generaly all the video player frameworks (phonon, directshow etc) all work well as long as you ONLY want to do the simplest "show video in a box" task - as soon as you try and mix them with anything else it's harder than just drawing the images.

(Of course I may be wrong and I would love it if phonon allowed showing a video in an openGL quad with a couple of lines of code!)

Martin Beckett
  • 94,801
  • 28
  • 188
  • 263