Vive facial tracker, high latency and loss of packages

Submitted by Mackandelius on #1

Using Oculus Quest 2 with a Vive facial tracker.
Both on my phone and Quest, VirtualHere seems to be skipping frames (the facial tracker is essentially a camera), the tracker when normally plugged in using usb shows very tiny movements, but using VirtualHere it seems to be skipping all small changes. There is also around 1 second of latency which isn't there when using USB.

Has VirtualHere recently updated their camera compression to remove these smaller changes or is there something wrong with my setup?

Submitted by Michael on Thu, 2021/04/29 - 11:26

#2

Virtualhere doesnt skip frames or compress the camera image. It never has and its not technically possible. USB must be passed through as is. (VirtualHere does compress the usb packets but its not related to frame compression of the video)

Its entirely your network latency and/or wifi signal, and possibly the cpu inside the quest 2 struggling to keep up.

In the virtualhere client right click USB Hubs->About-Statistics and it will show the latency. It should be <15ms

Submitted by Mackandelius on Sun, 2021/05/02 - 13:33

#3

Virtualhere reports a minimum of 40ms of latency with frequent spikes to 250ms, that strikes me as too high.

Doing some testing I do now that I shouldn't be the Quest that is the problem, both my last (a 2016 flagship) and current phone(cheap thing even less powerful than my last phone) gave me the same effect as with my Quest. The Quest 2 has on paper a CPU more powerful than most flagships of at least last generation.
Trying to narrow it down I've also tried using it on a different network setup, no change.

I know that this unorthodox setup is possible, because there is a video of someone going through how to do it and I've also seen other people using this exact same setup, following the same video and it works for them.

Last thing I tried was using a different computer, no change either. Which hopefully rules out hardware incompatibilities. Am at the moment completely reseting this other computer in the hope that there will be a change and if that is then I'll know I have something installed on my computer that screws with VirtualHere.

Is there any list on programs that cause problems for VirtualHere, because if it works on the reset computer then I will have a lot of programs to go through and that would be needlessly time consuming.

Submitted by Mackandelius on Sun, 2021/05/02 - 13:46

#4

Here is a link to the graph that I get when using it, the 410ms spike should be ignored, I've forgotten exactly why, but I'm guessing it happened when I unplugged and re-plugged in the facial tracker (camera).
https://i.imgur.com/ybRvAwd.jpg

Submitted by Michael on Sun, 2021/05/02 - 17:32

#5

What exact router are you using for wifi? It must be that.

Submitted by Mackandelius on Mon, 2021/05/03 - 13:58

#6

Now that I've tested it once again on a completely reset computer (no change), it might as well be my router. It is the TP-Link AX1500 or AX10 (interface says former, store and box says latter). Importantly it must be noted that it is hardware version 1.0, I know for a fact that the later hardware versions have problems with streaming (stuttering and lost packages). I've had this router for around a year now and not encountered any problems with anything else.

I was trying to cross the router off from being the culprit by using my phone as a hotspot, trying to see if it got worse. I do now realize that that it might have performed worse, but not enough for it to be noticeable. Not sure how I am going to test it without using the router.

I will try to see what happens if you use a windows USB server instead of the android ones I've used, that way I can see if it is wireless/android that is the problem.

Submitted by Michael on Mon, 2021/05/03 - 19:37

#7

"I know that this unorthodox setup is possible, because there is a video of someone going through how to do it and I've also seen other people using this exact same setup, following the same video and it works for them."

Do you have a link to this video? Its on youtube? I'd be interested to see.

Submitted by Mackandelius on Tue, 2021/05/04 - 00:58

#8

https://www.youtube.com/watch?v=pFESUGjbxeo

Tutorial starts at 2:24.

He has done a previous video where he tried to use a small router which could use VirtualHere and while he didn't show footage of it in action, going of his descriptions it might have had the same problems that I have. https://youtu.be/BFBgYgJBiAQ

Submitted by Mackandelius on Tue, 2021/05/04 - 11:21

#9

Literally using this post as a log at this point.

After successfully using the facial tracker with my second wired computer, I tried using it connected to my Quest again but this time without various features enabled that could drain performance (Guardian, passthrough), I also used a feature available in the VR game streaming app I use which increases the "clock rate", trying this at an early night after I had just started my computer, it seemed to work perfectly. After having to go away and do something else I re-tried it at around midnight, now it didn't work for some reason.

Doing some quick tests while writing this I tried only using 1 monitor and focusing on the various programs that are used.

Last lead I have is that it just doesn't work at night, my router could be decreasing its output or simply work less because it is late, maybe some kind of hard coded power saving feature, will have to try it earlier once again when I have the time. While my memory isn't the greatest I don't think I have tinkered with this earlier than 2 hours after dinner before.