Nano network lockup

[CPEng]

Hello,

I've been having alot of difficulty with a Nano 10 PLC getting into a state where it appears the Ethernet locks up every 1-2 days. The PLC gets to a state where it's webpage will not load fully, it loses packets on an echo request and most importantly remote file services stops working and I get no logs. Since the PLC is remote the easiest way to get it to work again is to reprogram it and do a Power ON reset, a simple reset from the from remote monitoring will not fix remote file services. I thought the issue was related to power so I added a 470 micro farad capacitor on the input, and eventually added a separate power supply, one that we got through TRi.

The setup is a Nano 10 with Fram rtc 256. Its firmware is R77. This Plc reads pulses at a rate of 100-200 hz using an interrupt and a 32 bit Var as a counter. Jumper 1 is set to keep all data over power ups, which means I assume that it adds overhead to my interrupt service routine incrementing the variable.

As I said earlier it happens every 1-2 days, but works perfectly otherwise. My remote file services waits at least 10 seconds before resending data, and the ladder logic looks to be working fine during the Ethernet lock ups. I can remote monitor over the Ethernet port but only just barely as it becomes very slow.

I'm at my wits end, please help.

CPEng

[support]

What are the functions of your remote file services? Is the remote file server acting as a depository for files that the Nano-10 upload to? Or does the remote file service download the extended file from the FRAM-RTC-256 periodically?

Is your remote file service software also reading or writing data to the Nano-10? What kind of protocol is it using?

Is the remote file service communicating with the Nano-10 via Internet or LAN?

What happen to the older file once the remote file service service has obtained its latest file? Who delete the file? Is the Nano-10 that delete the file or the the remote file service is the one delete the file using FTP protocol?

You can also send more information to support@tri-plc.com so that we can attempt to duplicate your setup to see if we can replicate the problem.

[CPEng]

Thanks for the fast response. I am out of the office till Monday so I can't send any code or files until then. I can give you some more background.

The remote file services are only too a TL server. The two commands used are "Append" and "Email".  It tries 3 times 10 second intervals to send the data with and OK reply then moves on. IF unsuccessful it waits a minute before sending the next data.

The network topology is a private LAN / WLAN.
TL server 192.168.0.10/24 --> router --> outdoor wds wireless AP --> outdoor wds wireless client --> network switch --> 172.16.16.60/24 Nano 10 PLC.  When the Ethernet port gets slow and drops packets I can ping the Wireless client and another device on the local switch to the Nano with no problem as a test for wireless. I have reset every device except te PLC and it doesn't fix the problem
, only a PLC Power On reset fixes it.

It is a hard problem for me to debug because it takes many hours on continuos running before it shows up. I have got it to run for 4 days without screwing up with Jumper 1 not set, so that will be my next step to disable that. I would really like to utilize that feature, it just doesn't make sense why it will work fine for 24 hours or so then mess up.

I have another nano 10 with Fram that I can swap out for testing to see if it's hardware related too. I won't know for sure if it fixes it for a few days after I install it.


Thanks
CPEng

[support]

It sounds like it may be a rare abnormal combination of events that may have corrupted some of memory used by the network for storing networking parameters and led to network difficulties.  A reboot will re-populate these data area with the correct data and resume the operation.

We may or may not be able replicate the issue if it has something to do with the unique combination of Ethernet switches and router setup. You could test it in a LAN environment and see if the same result would occur. When you contact us by email we will also try to send you a new TLServer program to test if it would work better with the Nano-10 <WRITE> command.

Since the Nano-10 is the one which is initiating the communication, it is possible for it to keep track of the number of network connection failure and if it become increasingly difficult it could store the critical data inside FRAM and execute a REBOOT command to perform an orderly restart of the PLC. Upon starting up it can then load the critical data back from the FRAM into its data memory or variables and continue operation. This way it ensure that the PLC can continue to operate uninterrupted.

[support]

This issue is resolved after the user upgraded the TLServer to the Version 3.16E which fixed some network related bugs.

[CPEng]

What he said ^^

Thanks