Cellular telemetry using SIMCom modules

[potater1]

Telemetry via text messages using SIMCom cellular modules (only SIM800L tested so far).
#3948

Features:

• request telemetry message by making phone call or sending SMS
• message transmission at regular intervals (configurable, turn on/off by sending SMS)
• turn forced RTH on/off by sending SMS
• configurable accelerometer triggered messages to report landing/crash or freefall

[teckel12]

@potater1 You'll need to exclude this from F3 processors due to limited RAM.

[potater1]

So does it go like this:
#ifndef STM32F3
#define USE_TELEMETRY_SIM
#endif
in target/common.h
?

btw What does "Travis CI build failed" mean?

[teckel12]

@potater1 Travis CI does unit tests and then builds all targets. It does this for every pull request to make sure the pull request doesn't introduce gross bugs.

You can see the results of one of these tests by scrolling to the bottom of this page:
https://travis-ci.org/iNavFlight/inav/jobs/494832428

Building SPRACINGF3NEO
Linking SPRACINGF3NEO
Memory region         Used Size  Region Size  %age Used
           FLASH:      248708 B       250 KB     97.15%
    FLASH_CONFIG:          0 GB         6 KB      0.00%
             RAM:       41152 B        40 KB    100.47%
             CCM:        7868 B         8 KB     96.04%
       MEMORY_B1:          0 GB         0 GB      -nan%
/home/travis/build/iNavFlight/inav/gcc-arm-none-eabi-7-2018-q2-update/bin/../lib/gcc/arm-none-eabi/7.3.1/../../../../arm-none-eabi/bin/ld: obj/main/inav_SPRACINGF3NEO.elf section `.bss' will not fit in region `RAM'
/home/travis/build/iNavFlight/inav/gcc-arm-none-eabi-7-2018-q2-update/bin/../lib/gcc/arm-none-eabi/7.3.1/../../../../arm-none-eabi/bin/ld: region `RAM' overflowed by 192 bytes
collect2: error: ld returned 1 exit status
make[1]: *** [obj/main/inav_SPRACINGF3NEO.elf] Error 1
make: *** [SPRACINGF3NEO] Error 2
The command "./.travis.sh" exited with 2.

So the SPRACINGF3NEO target failed to build. The reason was RAM overflowed by 192 bytes (40,960 bytes available, 41,152 bytes used).

As far as excluding the code for F3 processors, yes, you could do it as you stated, or extend what is already existing in target/common.h in the development branch:

#if defined(STM32F3)
#define USE_UNDERCLOCK
//save flash for F3 targets
#define CLI_MINIMAL_VERBOSITY
#define SKIP_CLI_COMMAND_HELP
#define SKIP_CLI_RESOURCES
#else
#define USE_TELEMETRY_SIM
#endif

Then, update your changes to only include if USE_TELEMETRY_SIM is defined, via:

#if defined(USE_TELEMETRY_SIM)

  // Your code here

#endif

[teckel12]

This makes me want to refactor the code for the SKIP_TASK_STATISTICS, SKIP_CLI_COMMAND_HELP & SKIP_CLI_RESOURCES defines. They should probably be USE instead of SKIP to avoid the unnecessarily complicated double-negative conditionals.

[hali9]

@potater1
You have added some cli parameters, so please add documentation in Cli.md about this parameters.
What parameter what they mean, what is the unit and the default value.
You can also add some documentation in Telemetry.md in docs folder by adding GSM or SIM label.
We will also need add label for telemetry in inav-configurator, should be GSM or SIM ?

[hali9]

Maybe:
if (!failsafeMessageSent && FLIGHT_MODE(FAILSAFE_MODE) && ARMING_FLAG(WAS_EVER_ARMED)) {
Multicopter can disarm after landing.
There are also known cases of disarming plane in flight.
So failsafe without arming is passible and should be perceived as an un-normal situation.

[potater1]

Ok this is changed now.

[hali9]

And:
if (!FLIGHT_MODE(FAILSAFE_MODE))
If I understand correctly, when is failsafe sms is send only one.
Maybe when failsafe is enabled send SMS periodicaly using sim_transmission_interval.

[potater1]

Yeah makes sense. I was thinking you might not necessarily want that, and could switch it on manually. But then in most cases it's probably good.

[potater1]

@teckel12
Ok got it, I clicked the Travis link and thought it was broken, forgot I have Javascript blocking on.

I put the conditional #define USE_TELEMETRY_SIM right after the other USE_TELEMETRY_ ones, under the FLASH_SIZE > 128 condition.

[potater1]

I renamed the program to SIM, as apparently GSM is just the name of the 2G standard, and doesn't apply to newer networks. It could be called "SIM" or "SIMCom" in the configurator... well who knows maybe it works with other than SIMCom modems too, but we can probably assume people will use a SIMCom one as they seem to be widely available?

[hali9]

When I using this google link the point is not appears. Problem is in ",500m" when I remove this it work well.
But I have better idea:
maps.google.com/?q=@%ld.%07ld,%ld.%07ld
This link is better because it work in internet browser google maps and in application google maps even when there is not internet connection (offline).

[wx4cb]

@hali9 would it be easier to put the actual sprintf text in the top of the file as either a const or a #define? I do that all the time as it's easier to look at the top then have to search through a potentially large file to find it.

Also makes it easier to change if the API changes. Also, it looks like you're missing an API key which depending on how many calls you may find that you're hitting their(google) request limit.

EG:
#define GoogleMapAPIURL "%s%d.%02dV %d.%dA ALT:%ld SPD:%ld/%d.%d DIS:%d/%d SAT:%d SIG:%d %s google.com/maps/place/%ld.%07ld,%ld.%07ld,500m\x1a"

OR

const char GoogleMapAPIURL[] = "%s%d.%02dV %d.%dA ALT:%ld SPD:%ld/%d.%d DIS:%d/%d SAT:%d SIG:%d %s google.com/maps/place/%ld.%07ld,%ld.%07ld,500m\x1a";

len = tfp_sprintf((char*)atCommand, GoogleMapAPIURL, ............

[fiam]

Maybe you could send the map link using OLC? Google Maps does support them and we already have the code for encoding latitude and longitude in common/olc.h

[hali9]

@wx4cb #define GoogleMapAPIURL or const char GoogleMapAPIURL[] - for me is no problem.

Also, it looks like you're missing an API key which depending on how many calls you may find that you're hitting their(google) request limit.

I do not think so. Because the page itself is shown with the indicated area and only the pin is missing.

@fiam Thank you for the suggestion, but it seems to me that calculating this is too complicated and rather difficult to understand if you do not use the application. Does this format work also offline? I think that the format maps.google.com/?q=@%ld.%07ld,%ld.%07ld is the best solution.

@potater1 I tested it today and now it work but when failsafe is gone I still get an SMS.
You enabled it by:
if (telemetryConfig()->simTransmissionInterval < 0) { telemetryConfigMutable()->simTransmissionInterval *= -1;
but never disabled sending, so:

if (!FLIGHT_MODE(FAILSAFE_MODE) && telemetryConfig()->simTransmissionInterval > 0)
            telemetryConfigMutable()->simTransmissionInterval *= -1;

but when before failsafe sending sms was enabled, this disable it too.
I think changing value of telemetryConfigMutable()->simTransmissionInterval is not good idea.
Maybe create flags sendFs and sendTelem and check it with or operator.

[potater1]

So I guess the logic would be:

• go into failsafe mode -> turn on regular transmission
• failsafe off -> turn transmission off if it was off before failsafe
  ?

OT question, I'm having trouble with the CLI, when trying to set a list of parameters in the configurator. It fails semi-randomly to recognize a parameter. Happens with the recent development branch commit that sim-dev was branched from. Configurator 2.1.1.

[hali9]

@potater1 See potater1#1
Yes I have too in 2.1.1, but I use 2.1.3 now. Did You try inav-configurator 2.1.3 ?

[potater1]

@hali9
Tried now with configurator 2.1.3. Same CLI problem. This happens with commit
7c24e68 but not with 958b49f from a while back. Should this be reported as an issue?

Hmm.. in your commit, it looks like it's sending constantly if sending is configured and the "T" message doesn't turn it off. And I'd like to be able to turn off sending in failsafe mode also. Here's an update I made to my version with one state variable and a bit of logic to handle switching on/off in failsafe and out of failsafe. The states are: don't send, send in failsafe mode, and send always. What do you think?

[hali9]

I have also problem in 2.1.3 but it occurs less. Should this be reported as an issue? Yes.
About sent states it looks good now.
Are You going to change the link to maps.google.com/?q=@%ld.%07ld,%ld.%07ld and/or remore ",500m" ?
It seems to Me that extern's are not needed in sim.c.
Also in sim.h You do not have to put all the functions. I think only handleSimTelemetry, freeSimTelemetryPort , initSimTelemetry is enough, because only those are used in telemetry.c.

How to setup aac settings to detect crash as >4g and land as 2-4g?
What extactly mean NegX settings ?

You write something like this:

            // skip SMS content
            while (serialRxBytesWaiting(simPort)) {
                if (serialRead(simPort) == '\n') return;
            }

I think this is wrong, because if modem does not sent all body of sms this not read all and get the rest of body in next iteration.
Should always set readingSMS = true and set up flag like:
goodSms = checkGroundStationNumber(&simResponse[7])
and then:
if(goodSms) readSMS();

[potater1]

Are You going to change the link to maps.google.com/?q=@%ld.%07ld,%ld.%07ld and/or remore ",500m" ?
It seems to Me that extern's are not needed in sim.c.
Also in sim.h You do not have to put all the functions. I think only handleSimTelemetry, freeSimTelemetryPort , initSimTelemetry is enough, because only those are used in telemetry.c.

Yes, I'll get to these in the next commit.

How to setup aac settings to detect crash as >4g and land as 2-4g?

The CLI parameters are cm/s/s.
acc_event_threshold_high = 4000
acc_event_threshold_neg_x = 2000
Neg_x is for detecting landings on a fixed wing, it reacts only to backwards ie. negative x axis acceleration. This threshold can be set quite low because strong backwards acceleration is less likely to occur during flight.

Should always set readingSMS = true

Yeah, I was trying to avoid adding yet another state variable with this cheap trick. The idea was that the rest of the line would just show up later as an extra line of nonsense which is ignored, unless it happens to begin with a recognized modem response.

But I guess instead of readingSMS we can have a readState variable with values "read_line", "read_sms", and "skip_line". It can be done right without an additional variable.

I'll get back to this tomorrow, have to go to sleep early!

[hali9]

And acc_event_threshold_low detect free fall, if I understand correctly. So this should do not exceed 980 (1G) ? Or rather be much smaller than 980.
readState is good option.

[potater1]

@hali9
Here's an update. Had no time to test it yet.

And acc_event_threshold_low detect free fall, if I understand correctly. So this should do not exceed 980 (1G) ? Or rather be much smaller than 980.

Yes. I put 900 as the upper limit for now.

[mistyk]

For information, I've tested and its not working without pin !

[JoseAntonioMG]

hello guys, I'm using inav 2.2 rc2 with an omniNTX F7 fc and the SIM800L module, I have configured the ports output a UART with GSM telemetry and also the CLI environment variables, however after a few minutes the FC is blocked and I have to restart it It does not send data by GSM.

[JoseAntonioMG]

I forgot to tell you that I tried the SIM800L module with Arduino and it works perfectly.

[hali9]

How do You power the SIM module. This module needs a 2A peak power. I am connecting RX, TX, GND to FC and GND, VCC to linear BEC 0,5A and between the 470uF electrolytic capacitor.

[JoseAntonioMG]

I have connected the module to an external power supply to the FC, specifically, a LIPO of 3.7v and 3A. The connections I have made with the FC are:

Module SIM800L with FC (UART4)
RX <---> TX
TX <---> RX

Module SIM800L with LIPO Battery 3.7v
Vcc <---> +
GND <---> -

[fiam]

@JoseAntonioMG Did you connect the SIM800L ground to the FC ground?

[JoseAntonioMG]

@fiam No

[JoseAntonioMG]

@hali9 and @fiam The FC is no longer blocked, I have connected GND from the FC to the GND of the SIM module. However I do not receive messages, I have configured the variables in CLI:

sim_ground_station_number = 653226677
sim_transmit_interval = 10
sim_transmit_flags = t
acc_event_threshold_high = 0
acc_event_threshold_low = 0
acc_event_threshold_neg_x = 0
sim_low_altitude = 1

The phone number does not have an international prefix, I think it is not necessary to put it.
In addition, the SIM card has the PIN disabled.

[JoseAntonioMG]

If I set the SIM pin to 0000, it does not register in the operator's network (the led of the SIM800L flashes 1 / s)
If I deactivate the pin of the SIM, it registers in the operator's network (the led of the SIM800L flashes 3 / s)

[JoseAntonioMG]

When is the SIM800L module initialized with pin 0000? When connecting the FC? or when arming?

[hali9]

Module is initialze after 10 second delay but send at command need additional about 10s.
You must first arm Fc to get response sms.
You can also clear sim_ground_station_number and send sms to module and without unpower FC check sim_ground_station_number how module see your phone number.

[JoseAntonioMG]

@hali9 But do we have to wait until the module LED flashes once every 3 seconds?

[hali9]

@JoseAntonioMG No, not wait for register in network.

[potater1]

Yes you have to have SIM800 ground connected to FC ground, otherwise you will get errors in serial communication.

I tried to disable the PIN, it works for me.

I just realized that if you use the wrong PIN it will lock the SIM card because it keeps spamming the PIN code to the module constantly.

@JoseAntonioMG Do you have an antenna connected? If you have less than great signal quality, it may be that you are able to get on the network but no to send messages without an antenna. Just a piece of wire can make a difference.

[JoseAntonioMG]

@potater1 Yes, I have an antenna connected and it works fine. I tested the module with Arduino and AT commands and I was able to send SMS without problems and with the PIN deactivated. I've also tried it with pin 0000 and, of course, I have to put the command "AT + CPIN = 0000" to connect to the network and it works.

However, it does not work when I connect it to the FC. I am using the UART6 of the FC, (RX FC-> TX SIM and TX FC-> RX SIM). In the configuration of INAV ports I have selected GSM-SMS in telemetry of the UART6 and I have configured the CLI variables as I mentioned in previous posts.

One thing I want to emphasize, is that the quadcopter is not armed since I am testing it indoors and the GPS does not take satellites. I do not know if this is a problem.

[potater1]

@JoseAntonioMG , try sending a SMS to the module, or calling the number when the LED is flashing slowly. Also try leaving sim_ground_station_number empty when you do this. It should respond, the first message or SMS sets the ground station number in case it's undefined.

[JoseAntonioMG]

I have done the test, using a new SIM with another operator and without establishing a PIN. But it does not work, I made a call to the number of the SIM card and I have also sent several SMS with the text "xxx", I put the variable sim_ground_station_number empty en CLI:

set sim_ground_station_number =

and the variable remains empty after making the call and sending the SMS.

[potater1]

@JoseAntonioMG That's odd if you're getting the slow flashing LED, but it's not sending messages. A power problem could explain it, but it's ruled out since you're using a battery. Do you have the SIM module ground connected to FC ground directly?

Hmm, one possibility is that the FC is not receiving comms from the module, so check SIM TX -> FC RX wire. Also try setting:
nav_extra_arming_safety = OFF
sim_ground_station_number = 653226677
sim_transmit_flags = t
and then try arming so you can get messages without calling/sending a message.

[JoseAntonioMG]

@potater1 Yes, I have connected the ground wire from the FC to the SIM module, as I mentioned a few days ago. Tomorrow I'll try what you tell me, and I'll tell you ...

[JoseAntonioMG]

Hi @potater1, I've done the test and it does not work. I can ARM the FC, but the SIM module does not send the SMS to the smartphone with the number that is configured in sim_ground_station_number.

Is it possible that the UART is not communicating with the SIM module? How can I know if there is communication between UART and SIM?

[JoseAntonioMG]

I love Long Range quadcopters and that's why, in a short time, I lost 2 quadcopters for not having a localization system for the LTE 2g / 3g network. I have considered the possibility of buying a module independent of the FC, which includes GPS and sends SMS automatically without the need to connect to the FC, but they cost very expensive and they increase a lot of weight. That is why I want to implement this possibility that you are developing in INAV, I think it's great

[potater1]

@JoseAntonioMG It's possible that the SIM800 registers on the network and sends messages when the connection from SIM800 TX to FC RX is cut.

Here's a version that prints the communications with the module to debug:
https://github.com/potater1/inav/tree/sim-test7

Compile and flash that one and add set log_level=DEBUG to settings.
Go to Sensors in the configurator, then click Open Debug Trace.
In the debug output, lines beginning with <<< are output from the module.

[JoseAntonioMG]

@potater1 it tells me that the page is not found

[JoseAntonioMG]

@potater1 "404 This is not the web page you are looking for"

[JoseAntonioMG]

@potater1 I have managed to access, copying the url and pasting into the browser. How is the source code compiled?

[JoseAntonioMG]

@potater1 On windows I have Cygwin. I can use it to compile INAV

[potater1]

I'm on Windows also and I use WSL which was easy to set up.
If you're having trouble compiling, just tell me what your FC is and I'll send you a .hex file.

[JoseAntonioMG]

@potater1 works!!!! I have changed the SIM800L module for the other one that was connected to arduino and it works. It seems that one of the SIM800L modules did not work well

[SZ-MXK]

@potater1 Can you modify the code to support SIM800C? SIM800C is easier to weld

[potater1]

@SZ-MXK Did you try the SIM800C already and it failed to work? The AT command manual I looked at also applies to the SIM800C so I think the code should work as is(?)