Occasional Slow USB Response

[Dinosaur]

Hi All

I am running Mint 17.3 Cinnamon 64 Bit, Kernel 3.19.0-32-generic.
It runs on a Fitlet-i PC

My system continuously polls an I/O board connected to the usb port for I/O
When I do a request to get Inputs it typically takes 10 to 11 mSec to complete the response.
However, almost exactly every 10 secs it takes 60 to 90 mSec.

I suspect the Linux system is interrupting me for some reason, and so I disconnected Eth network, and made sure
nothing is running that should not be. Yet the problem persists.

When I ran this system on XPe, I did not have this problem.

Nothing else will or should be running when my application is running.
Also the Startup menu is empty.

Has anyone else experienced this kind of problem or have any suggestions ?

Regards

[caseih]

How are you interfacing with the usb device? Are you using libusb or is it a usb/serial device?

[Dinosaur]

Hi All

I am using the Linux serial device ttyUSBn.

Reading a lot about Video drivers causing this kind of problem. !!!

Regards

EDIT: Have also swapped from USB3 port to USB2 but no change.

[D.J.Peters]

Is it as normal "user" or as "root" or both ?

Joshy

[Dinosaur]

Hi All

It is as normal user.
I should also say that the PC uses a HDMI output for display, and I have tried both HDMI to DVI and HDMI to VGA.

Regards

[caseih]

USB serial devices often do have subtle behavior differences between platforms and drivers. Really the RS232 is being emulated down the whole stack. There's not a real uart that your CPU can talk to per se. I know machines with real serial ports are rare these days but can you try it on a real serial port? How about different makes of USB Serial cable? Have you disabled hardware and software flow control when opening the port? Is your protocol simple enough that you can fire up minicom (which makes it easy to change parameters like flow control) and test it manually?

[Dinosaur]

Hi All

Have you disabled hardware and software flow control when opening the port?

I know how to do that in Assembly with RS232, but there seems to be no switches for it in the Open Com statement ?

I think I have minimised the problem, or at least the occurrence of it.
Originally my "pseudo code" was:

1. Tx request
2. Call Rx routine and see if any chr in buffer
If so , read them
If last chr then success.
3. If Not last Chr, Call Rx again after doing other things(no more then a mSec later)

This was repeated until last chr was received.
In my test program this resulted in 100% cpu use.

Then I changed the code to do:
1. Tx Request
2. Call Rx routine
Do Loop until last chr received.
Still 100% cpu use.

This resulted in a slow response every 30 sec or so.

Then in a strategic place I placed
Sleep 15

After that the slow response was every 80 sec's or so.
cpu use now down to 50%

My conclusion is that something is still interrupting every 10 sec's, BUT the coincidence of it
happening when I am waiting to Rx chr's is significantly reduced.

Regards

[caseih]

Hm, that's a good question. I'm not sure how to specify flow control in FB. Maybe one of the devs can comment.

Care to post the code? You should be able to get CPU down to near zero.

[Dinosaur]

Hi All

Thanks for the offer caseih, but I am not sure I want to get anywhere closer to Zero cpu load.

The tricky bit with all my systems is that Timing is critical. Not so much the accuracy, but the repeatability.
On my current machines, I am injecting a Smoothie mix (frozen like Ice Cream) into a bag maker.
The Target weight is controlled by the Nozzle open time.
If that changes from cycle to cycle (doing 40 bags per minute) my weight changes.

Even putting in the 15mSec Sleep, I had to think; "where in the cycle" can I afford to do that.

I still need to solve the 10 sec problem, and not getting any replies on the Linux Mint site.
But reading other posts, it appears that "USB Slowing down" is a very common Linux problem.

Regards

[BasicCoder2]

The Target weight is controlled by the Nozzle open time.

That strikes me as a very unreliable method to determine a weight? Unless I misunderstand what you mean.
Surely there are other things that may go wrong such as the rate at which the Smoothie mix is ejected?
Wouldn't a simple weight based feedback be more reliable?
.

[Dinosaur]

Hi All

Caseih, you would think so, but on a Target weight of 215g I can keep within 2 g.
With a Nozzle open time of 300 mSec that is a stability of 1mSec.

And in an Ice cream plant, you want to minimise the complexity of equipment, there are
cleaners who's mission in life it is, to find a crack where they can point a powerfull hose.
So the KISS principle is very important.

But of course there are check weigher's down stream to keep an eye on things.

PS:At the same time I am putting Ice, Frozen Fruit and Juice in the same pack, so closed loop Weight feed back becomes unreliable
if you don't know which product is varying.

Regards

[BasicCoder2]

... the KISS principle is very important.

If software is unreliable because it is running from within an operating system then hardware might be a robust solution.
A simple timer circuit.
.

[Dinosaur]

Hi All

That would be going full circle back 25 years, when I used ISA I/O boards that had the 8254 Timer chip on it.
The program sent the timing value and the rest was done without any cpu attention.

Actually BasicCoder2, it is not such a bad idea.
Must google for a usb board that I can connect and behave like an 8254

mmm??

Regards

[caseih]

Doesn't sound like your requirements are very hard real time. Soft real time, though. Bear in mind that neither Linux nor USB is inherently designed for real-time operation. Neither makes an guarantees about timing, beyond what is required to make USB itself function.

Is the hardware you are communicating with something you've programmed? Guess I'm asking why the timing has to be done in Linux. You ask about a USB board that can behave like an 8254. Well the answer these days is a microprocessor. In fact you could set up an arduino to do all the monitoring and time measurements and just send Linux the answer over serial port.

As to CPU use, I/O bound operations should never saturate the CPU. In a tight loop the way to ease CPU use (which may help with the slow down) is to use this form of sleep inside the loop:

Code: Select all

do
        sleep 1,1
loop until inkey = "q"

This will give up the time slice without missing anything. You'll see CPU drop to near 0, which is where it should be when you're simply waiting for bytes. In hard real time (or on an Arduino) you would never do this of course. But there wouldn't be CPU contention on an Arduino.

[Dinosaur]

Hi All

Caseih I actually contacted the manufacturer of the I/O board and they came back and told me they have a timed output instruction.
They didn't include it with the manual as it is experimental.
Also it is affected by heavy usb traffic, so can't use that either.

But I didn't sit still either.
In the logic of the program, the open time of the Nozzle is the only really critical thing.
So, 70mSec before it is due to shut, I abort all usb traffic, then when the time is up, the first usb transaction
is turning the Nozzle off. Printing some values on the screen after running for 3 hours, showed that only once
the 300mSec went to 301mSec.

I still got 30 or more incidents where the response time went over 50mSec, but now they are happening
at a time where it does not matter.

I also put some sleep statements in, and got the cpu % down to 30% when the machine is idling, and 50% when in full blown production.
However, I was not aware of other options on Sleep, so thank you for making me realise that sometimes I should look uo
statements that I think I know. (but really don't)