Monday, 28 October 2013

Video latency investigation

Following my first FPV flight, I wanted to get a better idea of the latency of the video feed. How does the Oculus Rift compare to a regular RC FPV system? What's the latency of different cameras and displays?

In this experiment, I only consider the image capture and display stages, not the wireless transmission stage involved in an FPV system. Why not? Because I fried my 5.4Ghz transmitter so I'm waiting for a new one!!!

OK, so the experiment is pretty straight forward:
  • A display (TV or monitor) shows "in real time" what a camera sees
  • I drop a ping-pong ball in front of the camera
  • At the same time, I film the actual physical event AND what the display shows with another camera
  • Then by watching the recorded video, I can spot the exact time when the ball bounces against the floor in real-life. Then I can do the same thing a few milliseconds later when the display shows the event. The difference in these two times gives me the latency of the entire image-capture-and-display system

The camera I use for filming the whole scene is a Go Pro Hero 3 set to its minimum resolution: WVGA 848x480 and maximum frame rate: 240 frames per second. This rate is huge, each frame of the video represents only about 4 milliseconds! This sounds great in theory in terms of precision but in practice things are not that clear when it comes to measuring the exact time of the "second" bounce on the TV/monitor. Indeed, due to the nature of image sensors and LCD screens, images of a rapid event look blurry. This can be seen on the following sample footage (resampled at 25 fps so it appears slow-mowy):


Here are the parts I've combined in the tests:


Sony CCD.
A traditional analog camera
used in FPV systems
GoPro Hero 3.
Also used in FPV systems
(analog live output)
EasyCap USB adapter.
Analog video input for PC
Sony PS3 Eye.
Often used in augmented
reality or vision applications
Raspberry PI camera
Funai LCD TV.
Old-school 4/3 TV
LG PC Monitor

Oculus Rift

And here are the combinations I've tried and the corresponding results. Once again, these are pretty "crude" latencies, but they allow comparison of various pieces of equipment.

Setup Average
latency in ms
Sony CCD + LCD TV 37
GoPro Hero 3 + LCD TV 54
Sony CCD + EasyCap + PC + Monitor 120
GoPro Hero 3 + EasyCap + PC + Monitor 143
Sony CCD + EasyCap + PC + Oculus Rift 123
GoPro Hero 3 + EasyCap + PC + Oculus Rift 131
Sony PS3 Eye (640x480 75Hz) + PC + Monitor 160
Sony PS3 Eye (320x240 187Hz) + PC + Monitor 58
Pi Camera (default HD) + Raspberry Pi + LCD TV98
Pi Camera (160x120) + Raspberry Pi + LCD TV90

In conclusion:

  • You can't beat a  traditional "analog" RC FPV system as it's latency is around 40 ms. Note again that we're not counting the wireless transmission here, just the image capture and display.
  • Adding a PC between the camera and the display (USB capture + rendering) adds 80 to 100 ms... In total we get about 150 ms. That's 3 to 4 times the latency of an RC system!
  • The webcams I've tried (i.e cameras that provide the computer with digital images directly) don't do very well either. Decreasing the resolution can help though. That's the case for the PS3 Eye at 320x240.
Once I receive my 5.8Ghz video transmitter, I'll be able to do a couple more tests with it (inserting it between the camera and the display). We'll see how much it adds in terms of latency (I suspect not much).



4 comments:

  1. I bet the LCD is introducing latency, would be interesting to test with a CRT.

    ReplyDelete
    Replies
    1. If I get my hands on one, I'll try. It seems that old-fashion CRTs are great in terms of latency. That reminds me the post from Carmack about all this:

      http://www.geek.com/chips/john-carmack-explains-why-its-faster-to-send-a-packet-to-europe-than-a-pixel-to-your-screen-1487079/

      http://superuser.com/questions/419070/transatlantic-ping-faster-than-sending-a-pixel-to-the-screen/419167#419167

      Delete
  2. This comment has been removed by a blog administrator.

    ReplyDelete