Slow serial data transfer over cellular network

Submitted by snc on #1

Hi, I'm working on a project requires remote (over cellular network) access to the devices with Serial or proprietary USB communication protocols. Everything looks good and all devices I have tested recognized and available on remote computer, and everything working fine when server and client are on the same physical network. But when I try to communicate with device over cellular connection, data transfer between application and device becomes very slow and after some time application drops connection with message "Device is not responding". Network packet test showing the device is responding but very slow, like a packet of data every 0.5 sec and data packet sizes ranges from 3 to 120 bytes. The protocol behind is very simple serial request-reply protocol with request length about 40 bytes and reply length from 40 to 200 bytes. I understand the cellular network is slow, but the same software communicates with device perfectly well using the same connection when using simple Python script forwarding serial data over IP network. Is there any solution for this problem?
Best regards,
Serguei

Submitted by Michael on Wed, 2019/05/15 - 09:49

#2

In the virtualhere client right click on the server and select Properties->Latency. What does that look like?

Submitted by Michael on Wed, 2019/05/15 - 13:08

#4

It looks pretty bad, what is make/model of cpu on your server? I suspect you need a faster cpu

Submitted by snc on Wed, 2019/05/15 - 13:23

#5

Well, it's Raspbery Pi 3 B+. It's not only that. The whole system working over VPN, connecting multiple sites to single network. I believe it is not only the network delay cause trouble, but how the data stream is created also. Any chance cpu optimized server version will make a difference? There is something strange, because very slow python implementation working reasonable well.

Submitted by Michael on Wed, 2019/05/15 - 13:32

#6

Can you run:

ping -c 100 server_ip

from the client, then send the output at the same time send the serial over virtualhere

I want to see what your network times are disregarding virtualhere.

Submitted by Michael on Wed, 2019/05/15 - 14:17

#8

Ok you definitely need a better network. The network is causing all the latency spikes.

Even with direct serial you will still have problems because you are getting 300+ms latency between transmissions sometimes which will cause a timeout issue regardless.

You could try to increase the FIFO buffer size. I dont know your serial driver but on the client you could try increasing the FIFO buffer to see if it helps. For example see here for FTDI https://www.ftdichip.com/Support/Documents/AppNotes/AN232B-04_DataLaten… . That document talks about the theory, but i think there is a GUI to adjust this as well if you open Windows Device Manager->Look for the COM port and click Properties and go through each tab and see if there is a FIFO buffer size or something similar.

Submitted by snc on Thu, 2019/05/16 - 04:36

#9

OK. I definitely need your help. :-) I definitely want to use your software instead of creating multiple home made monsters separate for each device type.
There is something strange how your server send (or client receive) data over slow network. I create a test setup - Raspberry Pi3+ with Virtualhere server sharing USB to Serial bridge, another Raspberry Pi3+ with Virtualhere client using this shared USB to Serial bridge, both Raspberries are on the same physical network. Second Raspberry sharing virtual USB to Serial bridge (ttyACM0) using "TCP/IP - serial bridge" from pyserial. I can connect from my laptop to the second Raspbery using cellular connection on laptop and cellular connection on raspberries network, and software able to communicate with device without any problem. Direct connection from laptop to Virtualhere server doesn't work. Help!

Submitted by snc on Thu, 2019/05/16 - 06:21

#10

Actually everything is even worse. I can successfully connect and read device if I have Raspberry virtualhere server on one network with cellular connection, Raspberry virtualhere client on another network with another cellular connection, this one sending serial stream to my laptop over same cellular connection also. Problem with windows client?

Submitted by Michael on Thu, 2019/05/16 - 08:59

#11

There is no problem with virtualhere its operating fine. Its your network. If you are doing serious programming of remote devices you need to have a decent network. Is there a reason your network is not very good, My 4G tests in australia have a pretty consistent 40ms ping time...Are you in a remote area or the aerial is not very good?

Submitted by snc on Thu, 2019/05/16 - 09:26

#12

OK. It is network. Why the same network working fine with linux client?

Submitted by Michael on Thu, 2019/05/16 - 09:51

#13

OK so you are switching to the linux client now? I thought you were using the windows client so i based my answers on that

Anyway the only difference between the VirtualHere windows version and the Linux version is the client kernel driver.

Im wondering if it might be the fifo buffer in the linux driver as opposed to the FIFO buffer in the windows serial driver. Taking a quick look on google im not sure how to see what the buffer size is in linux. Can you have a look on google to see if you can figure out latency/buffer sizes for your serial chip in linux and see if it is equivalent to the setting in windows.

If they are different try to set them the same as the linux. Otherwise i can login to your system and take a look (e.g via teamviewer) maybe there is an optimization i can do inside the windows kernel driver for virtualhere.

Submitted by snc on Thu, 2019/05/16 - 11:41

#14

I tried to play with FIFO buffer settings without much success. I'll give you access to my system via teamviewer. How I can contact you in private?

Submitted by Michael on Thu, 2019/05/16 - 11:57

#15

First can you tell me what and where you changed the settings in linux for that

Submitted by snc on Thu, 2019/05/16 - 12:09

#16

I did not changed settings for linux. I try to change settings for windows, in registry (RxFIFO) and in port settings. It looks like with 1, I can see at least the application starts to read data from device, showing beginning of the progress bar.

Submitted by Michael on Thu, 2019/05/16 - 12:34

#17

OK yeah thats what i thought, im not going to be debugging your network issues for you when you arent a customer and you havent tried yourself. You need to do this yourself first

Submitted by snc on Thu, 2019/05/16 - 13:10

#18

OK. I see we are talking different languages. First, I'm not interested in you debugging my network. Second, I'm not paying customer because, unfortunately for me, your product not performing well for me at the moment. Third, I was under impression you are interested to improve your product and I believe here is the situation where your product suppose to work, bit it is not. Please do not tell me about slow network again, transferring hundred bytes @ 115200 bps do not required fast connections. I still have a hope to find a solution and become paying customer. Thank you for your effort anyway.

Submitted by Michael on Thu, 2019/05/16 - 14:34

#19

No it's not virtualhere so there is no issues with it