Modul doesn't recover after AT^SJOTAP | Telit Cinterion IoT Developer Community
September 11, 2015 - 9:29am, 6257 views
Hi.
We have quite some BGS5T modules in the field and some of them need to have there MIDlets updated. To do that, we can issue AT^SJOTAP command from Java code, when specific SMS is sent, normal text SMS not PDU.
Most of the ***** this works fine, but some***** it happens that module won't recover after the update. Green light will be on, but it won't accept any new SMS commands, so MIDlet is probably not running. So you need restart it with turning off and on its power supply. I couldn’t check what happens on the module at that time, because I wasn't physically there, so I don't know if it accepts AT commands or if MIDlets is running.
What could be problem for that? Even if there is some ERROR inside MIDlet Watchdog should restart it after 60 seconds. Or is watchdog maybe the problem?
Regards Jure
Hello,
It would be hard to diagnose this without physical access to the module.
If the SIM card ***** PIN and the MIDlet enters it on start it might suggest that MIDlet is not running.
Watchdog behaviour depends on your application. If it is software watchdog which is activated by the MIDlet it will not start if MIDlet will not start after reboot.
But the OTAP procedure is designed to be as safe as possible - it stops all running apps, tries to download the updated application and only replaces the old one with the new one if the new one is successfully downloaded, contains the valid application. Then the module is rebooted and the new application should be started automatically. If the downloading fails the module is also rebooted after the timer expires and the old application is started.
If you are using the hardware watchdog built in the terminal - this could probably cause the module reset during OTAP procedure if the watchdog timer is too short. But in this case the old application should start after the reboot.
Some investigation will be needed.
Is it possible that there is a problem within the application which causes the new app crash or not functioning correctly? Is the application working after manual reset? If yes that might suggest some problem in the application which could start but not enter the PIN.
How about the power - maybe there was a power outage in a critical moment - generally this should not cause the problems and the old app should start.
Best regards,
Bartłomiej
Hi Bartłomiej,
SIM PIN is not a problem, because all SIM cards have PIN disabled.
I use software watchdog. In one case it's time is set to 90 seconds and in other one is set to 90 seconds. Hardware watchdog settings are like this:
WD=DFT_VAL,1,1
WD=ALWAYS_ON,60000,6
WD=ON,1,1
After restart the modules started working, but I don't know if it was the old or the new version of MIDlet.
I made some testes today in the office and on about 5 time that I tryed the SJOTAP module froze. Every time I issued OTAP on ASC1 so that I could monitore what is happening and every time it ended with result 900 Success, even when module woulden't responde.
[OTAP] Installation completed with result: 900 Success
[OTAP] No NotifyURL set. Not posting result.
[OTAP] DONE with status: No Error
[OTAP] operation OK
When module wouldn’t respond I couldn’t open USB port. When I plugged in another module USB port was working fine. And I also couldn’t use ASC1 port which I used before for OTAP. I couldn’t check what happened with MIDlet because I couldn’t set system. Output to ASC1. But I had autostart delay set to 45 seconds, so before that AT commands should be working. And even if the MIDlet had started and encountered some problem, it should restart, since first thish that happens in MIDlet is watchdog initialization.
When I unpluged modules powersource it started working again.
Can you suggest any more tests or can you see any potential problems?
EDIT:
It had happend again. This time I had ASC1 set as system.out so I could see that MIDlet stoped completly when it tryed to send some data over GPRS. This means that softwear watchdog was initialized and should restart the program after 60 secconds.
I have main while loop where there is sleep for 500ms, every 1 minutes there is temperature read and every 5 minutes data is sent over GPRS, but this is also called when MIDlet first enters that while loop. Do you know what could be the reason that watchdog didn't restart MIDlet?
This is watchdog initialization:
IMlet()
{
try {
System.out.println("Creating MainThread...");
watchdog = new Watchdog2();
watchdog.start(60);
There was no exceptions.
Best regards,
Jure
Hello Jure,
So if PIN is disabled the module should register to the network after restart even if there is some problem with the application (unless it is not the serious problem with the module itself). Have you tried to start OTAP again by OTAP SMS. Maybe this would help to get everything working again remotely.
It would be good to add some version information to the application and print it to the system out so you would know which one is running.
Please note that when you set the System.out to some interface it is reserved for System.out exclusively even when your MIDlet is not running. When you set OTAP tracer to some interface it is blocked until the module restarts. If the USB connection on PC is active it should be closed before module restarts, if not it may not be able to communicate with the module on USB after restart.
If OTAP tracer always shows success I'd live the system out on ASC1 and trace it all the time to check what is going on when the problem occurs. Or check the OTAP tracer on USB only.
The software watchdog must be activated by AT^SCFG command. If it is it should reset the module if you stop kicking it. Maybe your loop is still working.
When your MIDlet stopped completely, have you tried to connect on some interface to issue some AT commands and check if the list of running applications? Or was it impossible to issue any commands while you were able to do so before the application has stuck.
Regards,
Bartłomiej
Hello Bartłomiej,
SMS which I send to trigger SJOTAP is just normal SMS which java program reads as a command to start update. So when problem happens module won't detect that new SMS was received, because program is frozen. Today I had ASC1 for System. Out and ASC0 for OTAP trace and none of them would output anything. Program again froze at the point where it should make HTTP request. But software watchdog again didn't restart the program or module.
I know that ASC is reserved for OTAP tracer, but it should be released when SJOTAP is finished, or not? For that I used ASC1 and ASC2. When I had problem with USB I closed connection in the terminal (PC) and unplugged cable from USB port, this normally works. But in this case it doesn't help.
AT^SCFG is enabled, this is return on at^scfg: ^SCFG: "Userware/Watchdog","1". If loop would be running I should get system. Out msg every half second. But it would be possible that also ASC1 is working anymore, but that MIDlet is still running.
Like I wrote above, I was using ASC1 for debugging and ASC0 for OTAP tracer. ASC1 wouldn’t output anything, so I sent SMS for update like your suggested but nothing happened. After that I tried to issue AT command over USB but I again couldn’t open port, so I unplugged / pluged USB and closed/opened terminal connection on PC terminal, but I still couldn’t open port.
Before that it also happened that module restarted on received SMS, before it read SMS and no error was printed. But after few restarts MIDlet started working normally.
Best Regards,
Jure
Hello Jure,
You would have to send the OTAP SMS according to Java User's Guide that would be received by the module and not your application. Maybe that would help.
So if the app didn't initiate the OTAP (no output on the OTAP tracer interface) maybe there's some problem in the application. OTAP tracer should output any configuration or change you make with AT command or OTAP SMS.
The port reserved for OTAP tracer is released after the module restarts (regardless if the application was successfully downloaded or not).
This problem is strange. I get the feeling that it is not necessarily connected with OTAP but maybe I'm wrong.
Another thing is the watchdog problem. If you have initiated it correctly and are not kicking it the module should reboot. Is it possible that it is kicked somewhere else while your main loop has stopped working for some reason?
How about the logs from the application - does it stop responding in the same place every time or not?
According to your descriptions there may be a problem in your application or in the module. Are you able to reproduce it on any module or just on some of them?
Can you also check ATI1, AT^SCFG?, AT^SPOW? responses?
Regards,
Bartłomiej
Hello Bartłomiej,This are AR reponses that you requested:
When I was watching logs from the application, application froze 2 *****. And both ***** it was when it was trying to send some data over GPRS. So it could be connected with this.
I searched trough whole project where watchdog.kick() is issued and only place is inside the main loop. And after application froze, I didn't get any logs. Not from main loop not from any other part of application. I will try to do some more test and if I find out anything new, I will let you know.
Thank you.
Regards,
Jure
Hello Jure,
It seems that you have the latest official firmware for this module.
Have you tested the watchdog before? Was it working? Maybe just try to write the simple MIDlet to test watchdog.
Regards,
Bartłomiej
Hello Bartłomiej,
watchdog works, if I stop application with AT^SJAM = 2.... then moudle will restart because watchdog won't kick anymore. But when MIDlets stops at opening socket, watchdog won't restart module. It happend on two diffrent MIDlets and on quite some modules. And all modules had at^scfg="Userware/Watchdog","1" issued at initalization. It would happen before that MIDlet would stop at opening socket, but at those cases it would throw out some error after about 5mins or it would be restarted by watchdog if watchdog time was smaller then that.
Regards,
Jure
Hi Jure,
That would mean that when the application sticks on socket opening it causes the whole module to hang or at least the watchdog to stop working. This should not happen of course. But on the other hand this could be also caused by some bugs in the application. So I can see two things that you could do. Investigate your application and make sure that everything is correct there, sockets are closed properly, other resources are freed, there is no dead lock etc.
Try to use the terminal's hardware watchdog instead of software watchdog. As the hardware watchdog is independent of the module's firmware it should always work regardless of the module's current state.
Best regards,
Bartłomiej
Hi Bartłomiej,
could also be something that has to do with celluar network? Because things worked fine, then sudenly problem appared on multiple modules with diffrent MIDlets. And yestrday when I was trying to achive same module state and get some more data, I issued update for more then 20 ***** and it worked like it should. It bothers me, because some***** it works and someties it doesn't, if I could atleast know what triggers the problem, that would be quite an advancemnt.
With hardware watchdog if I understand I can monitor state of UART Rx and set GPIO 8 to output. I have an application where I am constatly reciving data on UART. Would HW watchdog still restart it even if MIDlets stops, but there is still data coming to UART?
If I use GPIO 8 for watchdog, do I need to connect somthing to it, or do I need just to set it as output and then don't even use it in java application?
Best regards,
Jure
Pages