Do you have Amazon AWS Tutorial..?

Question

Do you have Amazon AWS Tutorial..?

Opened this issue 6 years ago · 134 comments

Hi, I've interested this board. Do you have the example for communication between the board and AWS IoT cloud?

Answer 1 · 2018-12-13T15:54:57.000Z

Not currently, no. However, there are example AT command logs I have from SIMCom for Microsoft Azure which theoretically should work for other platforms like AWS IoT and anything that requires certificates. Basically you store the certificate in the SIM7000's EFS (electronic file system) and use the SSL commands to connect.

Answer 2 · 2018-12-18T13:32:37.000Z

Thanks for your fast response, you can share example AT command logs for Microsoft Azure. I can't find the example AT command in this web http://www.simcomm2m.com/En/module/detail.aspx?id=175.

Answer 3 · 2018-12-18T14:53:20.000Z

I can't share it publicly so you would have to order a Botletics shield before I could share it.

Answer 4 · 2019-01-09T22:11:34.000Z

Can you share these with me? We have purchased several botletics shields to experiment with, and are trying to get them talking to Azure IoT Hub.

Answer 5 · 2019-01-12T04:20:54.000Z

Please email me, botletics "at" gmail "dot" com.

Answer 6 · 2019-03-29T23:33:09.000Z

Not currently, no. However, there are example AT command logs I have from SIMCom for Microsoft Azure which theoretically should work for other platforms like AWS IoT and anything that requires certificates. Basically you store the certificate in the SIM7000's EFS (electronic file system) and use the SSL commands to connect.

Hi, there are some example where shows how store and read data from SIM7000's EFS?

Answer 7 · 2019-04-01T01:00:10.000Z

This should help but also check the related AT command manual.

Answer 8 · 2019-04-01T12:57:10.000Z

This should help but also check the related AT command manual.

oh thanks you. i'm realy confused, i don't know from where extract the .cer, i mean a SD? a web server? or the download from pc?. where i could put the .cer to apply these commands.

thanks so much.

Answer 9 · 2019-04-07T21:33:46.000Z

That depends on what platform you're using (Azure, AWS, etc.) and that file would be on your computer and sent to the SIM7000 via USB with AT commands.

Answer 10 · 2019-04-24T19:30:30.000Z

i have a doubt, the certificate must be in what format? i try this
-----BEGIN CERTIFICATE-----
MIIDQTCCAimgAwIBAgITBmyfz5m/jAo54vB4ikPmljZbyjANBgkqhkiG9w0BAQsF
ADA5MQswCQYDVQQGEwJVUzEPMA0GA1UEChMGQW1hem9uMRkwFwYDVQQDExBBbWF6
b24gUm9vdCBDQSAxMB4XDTE1MDUyNjAwMDAwMFoXDTM4MDExNzAwMDAwMFowOTEL
MAkGA1UEBhMCVVMxDzANBgNVBAoTBkFtYXpvbjEZMBcGA1UEAxMQQW1hem9uIFJv
b3QgQ0EgMTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALJ4gHHKeNXj
ca9HgFB0fW7Y14h29Jlo91ghYPl0hAEvrAIthtOgQ3pOsqTQNroBvo3bSMgHFzZM
9O6II8c+6zf1tRn4SWiw3te5djgdYZ6k/oI2peVKVuRF4fn9tBb6dNqcmzU5L/qw
IFAGbHrQgLKm+a/sRxmPUDgH3KKHOVj4utWp+UhnMJbulHheb4mjUcAwhmahRWa6
VOujw5H5SNz/0egwLX0tdHA114gk957EWW67c4cX8jJGKLhD+rcdqsq08p8kDi1L
93FcXmn/6pUCyziKrlA4b9v7LWIbxcceVOF34GfID5yHI9Y/QCB/IIDEgEw+OyQm
jgSubJrIqg0CAwEAAaNCMEAwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMC
AYYwHQYDVR0OBBYEFIQYzIU07LwMlJQuCFmcx7IQTgoIMA0GCSqGSIb3DQEBCwUA
A4IBAQCY8jdaQZChGsV2USggNiMOruYou6r4lK5IpDB/G/wkjUu0yKGX9rbxenDI
U5PMCCjjmCXPI6T53iHTfIUJrU6adTrCC2qJeHZERxhlbI1Bjjt/msv0tadQ1wUs
N+gDS63pYaACbvXy8MWy7Vu33PqUXHeeE6V/Uq2V8viTO96LXFvKWlJbYK8U90vv
o/ufQJVtMVT8QtPHRh8jrdkPSHCa2XV4cdFyQzR1bldZwgJcJmApzyMZFo6IQ6XU
5MsI+yMRQ+hDKXJioaldXgjUkK642M4UwtBV8ob2xJNDd2ZhwLnoQdeXeGADbkpy
rqXRfboQnoZsG4q5WTP468SQvvG5
-----END CERTIFICATE-----

and not connect

at log is that
_{�<---

OK

--->AT+CFSgfis=3,"root_ca.pem"

_{�<---

+CFSGFIS: 1189

OK}}

+CNACT: 1,"100.100.197.199"

OK
--->AT+SMCONF="URL",a5xpqsmvbu9sq-ats.iot.us-west-2.amazonaws.com,8883

_{�<---

OK

--->AT+SMCONF="CLIENTID",device2

_{�<---

OK

--->AT+SMCONF="KEEPTIME",60

�<---

OK

--->AT+SMCONF="CLEANSS",0

�<---

OK

--->AT+SMCONF="QOS",0

�<---

OK

--->AT+CSSLCFG?

�<---

OK

--->AT+CSSLCFG="sslversion",0,3

�<---

OK

--->AT+CSSLCFG=0,1,0

�<---

ERROR

--->AT+CSSLCFG=convert,2,root_ca.pem

�<---

OK

--->AT+CSSLCFG=convert,1,my_client.pem,my_key.pem

�<---

OK

--->AT+CSSLCFG?

�<---

OK

--->AT+CIPSTATUS

�<---

OK

STATE: IP GPRSACT

--->AT+CIFSR

�<---

100.100.197.199

--->AT+CIPSTATUS

�<---

OK

STATE: IP STATUS

--->AT+SMSSL=1,root_ca.pem,my_client.pem

�<---

OK

--->AT+SMSSL?

�<---

+SMSSL: 1,"root_ca.pem","my_client.pem"

OK

--->AT+CSSLCFG?

�<---

OK

--->AT+CGATT?

�<---

+CGATT: 1

OK

--->AT+SMCONN

�<---

ERROR}}

When i try not secure connection with cloudmqtt these commands works but not with AWS

Answer 11 · 2019-05-10T14:25:56.000Z

Not currently, no. However, there are example AT command logs I have from SIMCom for Microsoft Azure which theoretically should work for other platforms like AWS IoT and anything that requires certificates. Basically you store the certificate in the SIM7000's EFS (electronic file system) and use the SSL commands to connect.

This issue should be open. I saw the azure example in your AT Command Logs; thank you for that. However, it seems AWS only supports Https. The firmware on some of the shields support SSL only via TCP. Is there info on specific firmware releases and features to confirm? I am using B017000G.

Answer 12 · 2019-05-13T20:25:25.000Z

Sorry, I'm not sure if there's anything on specific firmware versions.

Answer 13 · 2019-06-22T16:47:40.000Z

I apologize in advance for my ignorance, as this is my first time programming a SIM7000. I am trying to perform the same task as above but using hologram.io. I created a new function in the Adafruit_FONA.cpp library and called it postDataHTTPS:

boolean Adafruit_FONA::postDataHTTPS(const char *request_type, const char *URL, const char *body, const char *token, uint32_t bodylen) {
  // NOTE: Need to open socket/enable GPRS before using this function
  // char auxStr[64];
  
    sendCheckReply(F("AT+GMR"), ok_reply, 10000);
	sendCheckReply(F("AT+CNACT=1,\"hologram\""), ok_reply, 10000);
	sendCheckReply(F("AT+CNACT?"), ok_reply, 10000);
	sendCheckReply(F("AT+CSSLCFG=\"convert\",2,\"hologram.cer\""), ok_reply, 10000);
	sendCheckReply(F("AT+SHSSL=1,\"hologram.cer\""), ok_reply, 10000);

	char urlBuff[strlen(URL) + 22];
	sprintf(urlBuff, "AT+SHCONF=\"URL\",\"%s\"", URL);
	if (! sendCheckReply(urlBuff, ok_reply, 10000))
		return false;
	
	sendCheckReply(F("AT+SHCONF=\"BODYLEN\",100"), ok_reply, 10000);
	sendCheckReply(F("AT+SHCONF=\"HEADERLEN\",100"), ok_reply, 10000);
	sendCheckReply(F("AT+SHCONN"), ok_reply, 10000);
	
	char dataBuff[strlen(body) + 22];
	sprintf(dataBuff, "AT+SHBOD=\"%s\",100", body);
	
	//if (! sendCheckReply(dataBuff, ok_reply, 10000))
	//	return false;
	sendCheckReply(dataBuff, ok_reply, 10000);
	//sendCheckReply(F("AT+SHBOD=\"TEST\",100"), ok_reply, 10000);
  
	sendCheckReply(F("AT+SHAHEAD=\"Content-Length\",\"120\""), ok_reply, 10000);
	sendCheckReply(F("AT+SHSTATE?"), ok_reply, 10000);
	sendCheckReply(F("AT+SHREQ=3"), ok_reply, 10000);
	sendCheckReply(F("AT+SHREAD=0,227"), ok_reply, 10000);
	sendCheckReply(F("AT+SHDISC"), ok_reply, 10000);
  
  return true;
}

I've been reading the SIM7000 documentation for the HTTPS commands, and I am struggling to get it working as I am sure I have mistakes somewhere. In my Arduino sketch I have the following:

        // Post data to website via 2G or LTE CAT-M/NB-IoT
        // Create char buffers for the floating point numbers for sprintf
        // Make sure these buffers are long enough for your request URL
        char URL[150];
        char body[100];
        char deviceID[] = "######";
        char tagID[] = "[\"_RESTAPI_\", \"WATER_LOW\"]";
        char message[] = "\"Water_Low\"";

        // POST request
        sprintf(URL, "https://dashboard.hologram.io/api/1/csr/rdm");
        sprintf(body, "{\"deviceid\": %s, \"tags\": %s, \"data\": %s}", deviceID, tagID, message);

        Serial.println(F("Attempting to perform HTTPS POST..."));
        Serial.print("URL: ");
        Serial.print(URL);
        Serial.println();
        Serial.print("Body: ");
        Serial.print(body);
        Serial.println();
        if (!fona.postDataHTTPS("POST", URL, body)){
          Serial.println(F("Failed to complete HTTPS POST!"));
        } else {
          Serial.println(F("Successfully performed HTTPS POST!"));
        }

When the sketch runs, I get the following on the serial monitor:

Attempting to perform HTTPS POST...
URL: https://dashboard.hologram.io/api/1/csr/rdm
Body: {"deviceid": ######, "tags": ["_RESTAPI_", "WATER_LOW"], "data": "Water_Low"}
	---> AT+GMR
	<--- Revision:1351B03SIM7000A
	---> AT+CNACT=1,"hologram"
	<--- ERROR
	---> AT+CNACT?
	<--- +CNACT: 1,"###.###.###.###"
	---> AT+CSSLCFG="convert",2,"hologram.cer"
	<--- ERROR
	---> AT+SHSSL=1,"hologram.cer"
	<--- OK
	---> AT+SHCONF="URL","https://dashboard.hologram.io/api/1/csr/rdm"
	<--- OK
	---> AT+SHCONF="BODYLEN",100
	<--- OK
	---> AT+SHCONF="HEADERLEN",100
	<--- OK
	---> AT+SHCONN
	<--- ERROR
	---> AT+SHBOD="{"deviceid": ######, "tags": ["_RESTAPI_", "WATER_LOW"], "data": "Water_Low"}",100
	<--- ERROR
	---> AT+SHAHEAD="Content-Length","120"
	<--- ERROR
	---> AT+SHSTATE?
	<--- +SHSTATE: 0
	---> AT+SHREQ=3
	<--- ERROR
	---> AT+SHREAD=0,227
	<--- ERROR
	---> AT+SHDISC
	<--- ERROR

I used the LTE_Demo example sketch as a building block, so I have all of the other associated code in place and working well to set up the SIM7000. It is also getting a proper IP address when I issue the AT+CNACT? command.

The first problem is the error on the AT+CSSLCFG command, I think that is preventing the AT+SHCONN and AT+SHBOD commands from working. I am also not sure how I should be handling the quotation marks and commas inside the body for the AT+SHBOD command, do I simply prefix them with a back slash?. Any ideas on what I could be doing wrong? I downloaded the top-level Starfield Class 2 Certification Authority key, which is below:

However I am not sure how to include that on the AT+CSSHCFG command.

Edit: I was able to successfully install the QPST software and upload the CA key to the 'customer' directory in the alternate file system of the SIM7000. I now get an "OK" response on the AT+CSSHCFG command. However, I am still getting "ERROR" on the AT+SHCONN command. I wonder if it is related to the AT+SNACT command erroring out too? The following tool was helpful in getting some additional insight on how the certificates work: https://github.com/tmcadam/sim7000-tools

As well as this previous issue: #71

Thanks!

Answer 14 · 2019-06-23T09:02:59.000Z

I'm having the same issues as @jefflikesbagels

In addition, I'm concerned that if I release these IoT devices to our customers then we will have to recall them all to update the root CA when those certificates expire.

I wonder if there's a way to force the device to trust the root CA regardless of who it is and then just continue with the HTTPS POST.

I think this would be ok, since the configuration on what we're sending to will be baked in to the firmware of our devices.

Thoughts?

Answer 15 · 2019-06-23T10:45:00.000Z

In case it helps anyone, I managed to upload a cert with the following procedure (not using the EFS Explorer)

Copy a CA Root Cert (for the site you're trying to connect to) to the SIM7000:

Go to ssltools.com
Enter the website you want the root cert for (e.g. https://putsreq.com)
Download the root certificate PEM, which should give you a .cer file
Strip out all carriage returns and then count the characters in the cert file (look at the bottom of VSCode for a character count)

Use these commands to load the cert onto the SIM7000:

AT+CFSINIT
AT+CFSWFILE=3,"AddTrust.crt",0,1496,10000 <-- 1496 is the char length that noted before
paste the cert data into the terminal within 10 seconds
AT+CFSTERM

This puts the cert in the 'customer' directory on the SIM7000. Tomorrow I'll see if this lets me make that https post, try your luck with this approach in the mean time.

Answer 16 · 2019-06-23T10:57:15.000Z

@TimRoadley Thanks for the info. I've been thinking about how I can integrate all of this into the Arduino so I can remotely push a new certificate to the EEPROM using some creativity with hologram.io's tools, then use the AT+CFSWFILE command to write the certificate, but unfortunately I'm using an UNO which does not have enough EEPROM space to store the entire certificate. For my use case, I'm just doing a simple DIY project for a friend, so if it lasts until 2034 when the CA expires that's good enough for me haha. I guess I could expand the EEPROM with an additional chip, but at that point for all the extra work involved I might as well just switch to SMS alerts and pay the $0.20 per message instead.

What's strange is the ssltools site is giving me a 502 bad gateway when I tried to download the root certificate, but all of the others work. I originally just used Chrome to export it anyways. I think I am past the certificate part, so now I have to figure out why the AT+SHCONN command is failing.

Answer 17 · 2019-06-25T04:33:56.000Z

Well this is frustrating. I did some more digging, and found another issue that is preventing me from making progress. According to the SIMCOM technical documentation, the max string size for the URL on the AT+SHCONF and AT+SHREQ commands is 64 bytes. For sending a data route through Hologram, the Arduino needs to do an HTTPS POST to the following URL:
https://dashboard.hologram.io/api/1/csr/rdm?apikey=##############################
I was not adding the API key before (duh - hologram was rejecting the API call), but now that I am adding it, the length of the URL is 81 bytes. I tried setting up an HTTP redirect on my personal web server to shorten the length, but the redirect prevents the hologram REST API from parsing the data properly.

One thing that helped me immensely was using the Restlet Client Chrome Extension. Between that and sifting through the Hologram REST API documentation again helped me figure out what format it's actually expecting.

Getting back to the issue at hand, it could be possible that the AT+SHCONN command fails because the Hologram API is rejecting the connection due to the incorrect URL (without the API key). Surely I am missing something here? The 64 byte URL limit is going to completely break the SIM7000's ability to do HTTPS POST commands to activate Hologram data routes. The next option may be using a TCP socket connection to Hologram Cloud: Socket API, Device Key. It looks like that would be the better solution anyways.

Sorry to derail a bit from the original intention of working with AWS, but I believe the procedure will be very similar to Hologram, so this development will still be beneficial. If I should create a separate issue tracker just let me know. It looks like AWS supports both HTTPS and MQTT calls, while Hologram supports HTTPS and TCP socket calls.

Answer 18 · 2019-06-25T05:33:54.000Z

@jefflikesbagels out of interest what firmware version are you running (and what chip)?

My testing has paused since I blew up my SIM7000E with a firmware update. Be careful with firmware over the air (FOTA)!

Answer 19 · 2019-06-25T20:15:19.000Z

@TimRoadley I have a SIM7000A running 1351B03SIM7000A firmware.

Answer 20 · 2019-06-25T22:38:53.000Z

@jefflikesbagels I wonder if https://github.com/botletics/SIM7000-LTE-Shield/blob/master/SIM7000%20Documentation/Firmware/1351B04SIM7000A.rar would help (I have no idea)

Answer 21 · 2019-06-26T23:45:56.000Z

@TimRoadley Thanks, I went ahead and updated to B04 just for good measure.

I finally got the Arduino sending data to Hologram via the Socket API!!! The issue I found is very silly too. For the FONA library commands, a lot of them are used in the following (or a similar) fashion:

        // Connect to TCP server
        if (!fona.TCPconnect(host, port)) {
          Serial.println(F("Failed to connect to server!"));
          delay(5000);
          break;
        } else {
          Serial.println(F("Successfully connected to server!"));
        }
        delay(5000);
        // Send TCP payload
        if (!fona.TCPsend(TCPpayload,sizeof(TCPpayload))) {
          Serial.println(F("Failed to send TCP payload!"));
          delay(5000);
          break;
        }

Where there is an if statement checking whether the function returned false or true. With this code it was not working properly at all. However, on a whim I decided to try and simplify the code as much as possible, and removed all of these checks down to the following:

        fona.TCPconnect(host, port);
        fona.TCPsend(TCPpayload,sizeof(TCPpayload));
        fona.TCPclose();

And all of a sudden it started working! One thing I noticed before was that I would get the "failed to connect" message on the serial monitor, but would continue receiving responses from the SIM7000, almost like the code is getting ahead of itself. I know I've deviated really far from the original goal of using HTTPS POST, but give this a shot and see if it helps. It's possible that removing all of the extra if statements and logic will allow the SIM7000 to send data properly. Here's the final snippet of code for my TCP socket connection:

        // Send TCP payload to server via LTE CAT-M/NB-IoT
        char host[] = "cloudsocket.hologram.io";
        uint32_t port = 9999;
        char devicekey[] = "xxxxxxxx";
        char data[] = "Water_Low";
        char topics[] = "WATER_LOW";
        char TCPpayload[strlen(devicekey)+strlen(data)+strlen(topics)+24];
        sprintf(TCPpayload, "{\"k\":\"%s\",\"d\":\"%s\",\"t\":\"%s\"}", devicekey, data, topics);
        Serial.println(TCPpayload);

        // Connect to GPRS
        fona.enableGPRS(true);

        // Connect to TCP server
        fona.TCPconnect(host, port);

        // Send TCP payload
        fona.TCPsend(TCPpayload,sizeof(TCPpayload));

        // Disconnect from TCP server
        fona.TCPclose();

        // Disconnect from GPRS
        fona.enableGPRS(false);

Answer 22 · 2019-08-01T22:21:32.000Z

Hey guys, there is now a Botletics community forum that makes it easier to post questions and things. Feel free to join!

Answer 23 · 2020-07-18T18:29:24.000Z

ssltools.com

In case it helps anyone, I managed to upload a cert with the following procedure (not using the EFS Explorer)

Copy a CA Root Cert (for the site you're trying to connect to) to the SIM7000:

Go to ssltools.com

Enter the website you want the root cert for (e.g. https://putsreq.com)

Download the root certificate PEM, which should give you a .cer file

Strip out all carriage returns and then count the characters in the cert file (look at the bottom of VSCode for a character count)

Use these commands to load the cert onto the SIM7000:

AT+CFSINIT
AT+CFSWFILE=3,"AddTrust.crt",0,1496,10000 <-- 1496 is the char length that noted before
paste the cert data into the terminal within 10 seconds
AT+CFSTERM

This puts the cert in the 'customer' directory on the SIM7000. Tomorrow I'll see if this lets me make that https post, try your luck with this approach in the meantime.

@TimRoadley Hey I used all the things you stated from downloading and remove carriage return to sending. But I GOT an error while writing the command

AT+CFSWFILE=3,"dweet.crt",0,1901,10000
DOWNLOAD

ERROR

Do you know any reason why it happened?
do I have to place that file in a certain folder? or do I have to remove Begin certificate and end certificate line?

Answer 24 · 2020-07-30T03:34:06.000Z

@sethivansh6 ERROR during AT+CFSWFILE points at the module not receiving the (correct) file contents within the self-imposed time (you specified 10000 = 10 seconds). Perhaps there is a mismatch on the number of bytes (you specified 1901). No other content validation is performed in this step, it's just a straight EFS put.

Btw, when working in *nix command line, one can just wc -c (or even ls -l) the local file to get the exact number of bytes when preparing the transfer.

Answer 25 · 2020-12-09T23:46:28.000Z

Hey guys, please see this AWS AT command log from SIMCom. Hope it helps!

Answer 26 · 2021-05-25T19:52:51.000Z

@botletics Thanks, that helps a bit. I'm looking to configure the module to just validate the server cert. The client will be authenticated via username/password so there is no client cert.

Did anyone achieve this?

Answer 27 · 2021-05-26T02:50:25.000Z

Please see the addRootCA() and TCPconnect() functions here. Please also see this set of AT commands that another user tried that worked for him.

Answer 28 · 2021-05-26T08:00:09.000Z

Thank you. In addition, I got a response from SIMCOM:

The chipset for SIM7000E need CA\client crt\client key , 3 files, can not support only CA. But you can input dummy client crt and client key file, just to "cheat" the stack that it already has 3 file totally. BTW before connection please update date and time by NTP function, AT+CCLK? should return correct data and time. [...]

Answer 29 · 2021-05-31T04:53:04.000Z

@botletics I tried following the AWS IoT command log you posted however I am getting ERROR at AT+SMCONN.

How can I debug this? The same certificates and key allowed me to use their Python SDK to publish successfully.

AT+CCLK?
+CCLK: "21/05/31,04:30:24+00"

OK
AT+CPIN?
+CPIN: READY

OK
AT+CSQ
+CSQ: 29,99

OK
AT+CGREG?
+CGREG: 0,1

OK
AT+COPS?
+COPS: 0,0,"ROGERS ROGERS",7

OK
AT+CGNAPN
+CGNAPN: 1,"ciot"

OK
AT+CNACT=1,"ciot"
OK

+APP PDP: ACTIVE
AT+CNACT?
+CNACT: 1,"10.237.129.141"

OK
at+csslcfg=convert,2,AmazonRootCA1.pem
OK
at+csslcfg=convert,1,my_thing.cert.pem,my_thing.private.key
OK
AT+CSSLCFG="sslversion",0,3
OK
AT+SMSSL=1,AmazonRootCA1.pem,my_thing.cert.pem
OK
AT+SMCONF=url,##############-ats.iot.us-east-1.amazonaws.com,8883
OK
AT+SMCONF="clientid","basicPubSub"
OK
AT+SMCONF="KEEPTIME",60
OK
AT+SMCONF?
+SMCONF
CLIENTID: "basicPubSub"
URL: "##############-ats.iot.us-east-1.amazonaws.com:8883"
KEEPTIME: 60
USERNAME: ""
PASSWORD: ""
CLEANSS: 0
QOS: 0
TOPIC: ""
MESSAGE: ""
RETAIN: 0

OK
AT+SMCONN
ERROR

I also tried setting a topic and message (same topic that the python client successfully publishes to) with no luck.

Answer 30 · 2021-06-01T04:36:34.000Z

I finally figured this out!

I now have AT+SMCONN succeeding with AWS (haven't tested anything beyond this yet)

There's 2 important non-obvious steps.

Whatever url AWS gives you for your IoT Core endpoint, you have to strip out the "-ats" from it. So "a1k9ecig9j720o-ats.iot.us-east-1.amazonaws.com" becomes "a1k9ecig9j720o.iot.us-east-1.amazonaws.com"
You need to use the legacy root certificate provided by AWS here under "VeriSign Endpoints (legacy)". NOTE some regions apparently don't support legacy certs, so it seems the SIM7000 won't work in those.

I also updated my SIM7000A to the B04 firmware from B03, not sure if that mattered.

Contrary to what @tomlankhorst posted, it doesn't seem to matter if your RTC is sync'd.

Answer 31 · 2021-06-01T05:08:36.000Z

Not quite there yet... subscriptions work but publishing doesn't.

2019-06-26 14:08:06:424[Send->]AT+SMPUB="ryan","8",1,1

2019-06-26 14:08:06:428[Recv<-]AT+SMPUB="ryan","8",1,1

> test626
2019-06-26 14:08:21:365[Send->]test626
2019-06-26 14:08:21:398[Recv<-]test626
2019-06-26 14:08:27:766[Recv<-]
+SMSTATE: 0

OK

The connection drops immediately after publish, and in the AWS (Cloudwatch) logs is this:

{
    "timestamp": "2021-06-01 04:56:38.276",
    "logLevel": "ERROR",
    "traceId": "6c4ca615-12d0-93d3-6434-f5f85365cc66",
    "accountId": "1234567890123",
    "status": "Failure",
    "eventType": "Publish-In",
    "protocol": "MQTT",
    "topicName": "ryan",
    "clientId": "basicPubSub",
    "principalId": "82b899e4bcb6bfc158f83be904d9a305e2b21ee255f0d7062cbe6ad3eda05f7a",
    "sourceIp": "xxx.xxx.xxx.xxx",
    "sourcePort": 29573
}

Answer 32 · 2021-06-01T16:18:18.000Z

Publishing works with this:

AT+SMPUB="test-topic",8,0,0
> testing1
OK
AT+SMPUB="test-topic",8,1,0
> testing2
OK

fails with:

AT+SMPUB="test-topic",8,2,0
> testing3
+SMSTATE: 0

OK

fails with:

AT+SMPUB="test-topic",8,0,1
> testing3
+SMSTATE: 0

OK

In summary, server retain can't be used and qos=2 can't be used. This is an AWS limitation.

Answer 33 · 2021-06-02T14:08:50.000Z

Can't seem to get persistent connections working with AWS

After AT+SMCONF="CLEANSS",0 I connect, then subscribe. Confirm I receive messages on the subscribed topic.
Then I kill the power on the shield, publish a new (qos=1) message to the subscribed topic from another client, power the shield back on, reconnect (SMCONN). Queued message not received. Even new messages on the topic are not received.
Resubscribe to topic (SMSUB), queued messages still not received, but new messages are received.

After power up and reconnect in step 2 I see this message in AWS Cloudwatch:

{
    "timestamp": "2021-06-02 13:54:53.996",
    "logLevel": "ERROR",
    "traceId": "8133c8d9-621f-d392-0b22-5f55b4c22f3e",
    "accountId": "144349053222",
    "status": "Failure",
    "eventType": "Disconnect",
    "protocol": "MQTT",
    "clientId": "ARMS-GF-D01",
    "principalId": "82b899e4bcb6bfc158f83ac904d9a305e2b21ee255f0d7062cbe6ad3eda05f7a",
    "sourceIp": "74.198.90.117",
    "sourcePort": 16187,
    "reason": "DUPLICATE_CLIENT_ID",
    "details": "A new connection was established with the same client ID",
    "disconnectReason": "DUPLICATE_CLIENTID"
}

I think this suggests it is not rejoining the existing persistent session but starting a new one? Anyone know what I'm doing wrong?

Also the last will and testament message isn't getting published to the configured topic at any point after disconnect.

Answer 34 · 2021-06-09T18:38:19.000Z

While I've been successful getting connections to complete over MQTTS, HTTPS is still not working with ERROR response from SHCONN.

If I use port 443 for the URL, SHCONN succeeds, but then I get 403 errors to all my requests because they lack SigV4 signing. If I use port 8443, then SHCONN times out. Note that I'm using the same CA , client cert and key for MQTTS, HTTPS (port 443) and HTTPS (port 8443). I can also curl the endpoint with no problem using these certs, so there does appear to be a bug somewhere with HTTPS.

I've had no response from SIMCOM technical support. in over a week.

Answer 35 · 2021-07-28T10:19:17.000Z

@davegravy Thanks for all your updates on getting MQTT working with AWS IoT. Can you please share your final commad log? We currently have an error when we connect. Thanks in advance.

Answer 36 · 2021-08-19T15:47:36.000Z

@davegravy Thanks for all your updates on getting MQTT working with AWS IoT. Can you please share your final commad log? We currently have an error when we connect. Thanks in advance.

Hi James, I have been playing around with this today and below is the list of commands i used to both publish and subscribe.

//connecting the device to data network

//Is the SIM ready? does it require a pin
AT+CPIN?
+CPIN: READY OK

//check network strength Signal Quality report lower the first number the better 99 = unknown
AT+CSQ
+CSQ: 14,99 OK

//Check network registration status, 2nd number should be 1 to show that its registered
AT+CGREG?
+CGREG: 0,1 OK

//show operator selection
AT+COPS?
+COPS: 0,0,"O2 - UK giffgaff",7

//Get Network APN in CAT-M Or NB-IOT
AT+CGNAPN
+CGNAPN: 0,"" OK

//Set prefered modem selection 2 Automatic,13 GSM only,38 LTE only,51 GSM and LTE only
AT+CNMP=13

//select the APN 0,deactive,1 active, 2 auto active
AT+CNACT=1,"giffgaff"
OK
+APP PDP: ACTIVE

//Get ip address of modem
AT+CNACT?
+CNACT: 1,"100.71.118.22" OK

//Set the verisign ca cert into device
at+csslcfg=convert,2,verisignca.pem

//set the device connection to use the device certs
at+csslcfg=convert,1,cert.pem,privatekey.pem

//set ssl version to use at least 1.2
AT+CSSLCFG="sslversion",0,3

//view current connection
AT+SMCONF?

//Set up the certs for the SecureMqtt connection
//rootca and device cert
AT+SMSSL=1,verisignca.pem,cert.pem
AT+SMCONF="URL",axxxxxxxxxu.iot.eu-west-1.amazonaws.com,8883

//Make sure you set the clientid
AT+SMCONF="clientid","SIM7000"

//Connect to the MQTT broker
AT+SMCONN

//Publish a message
//default policy only allows you to publish on clientid topics,number of characters,QOS(only 0 works),Retain (only 0 works)
AT+SMPUB="SIM7000/test",8,0,0

12345678
OK

AT+SMSUB="SIM7000/#",0

AT+SMSUB="SIM7000",1

Answer 37 · 2021-09-14T15:25:58.000Z

Here is a quick list of things I had to do to get the SIM7000A to work with MQTTS:

I had to update the firmware to 1351B04SIM7000A. There has been discussion on whether these matters. Yes, these matters. It will not work with B03 firmware. When downloading the firmware remember to jumper the BOOT_CFG pin with 1.8V_EXIT to get the module in download mode. This is already mentioned elsewhere, but I wanted to reiterate. Also, when in download mode, my module PWRKEY LED did not illuminate, just be aware.
I had to put the ca.crt, client.crt, and client.key in the customer file to get AT+CSSLCFG=”convert”… to work. This is number 3.
Be sure to check the minimum tls support on your mqtt broker, AWS and Mosquito need at least 1.1. This step is not shown in the SIM7000 Series_MQTT(S)_Application Note. Change the protocol to tls version using AT+CSSLCFG=”sslversion”,
If using AWS IoT core, you need to use the legacy Endpoints and the correct CA for it, just like davegravy said in an earlier post. If you do not live in a region that supports the legacy endpoint, you can easily change this in your AWS account in the upper right-hand corner where it says your regain.

I hope this helps someone.

Answer 38 · 2022-03-28T08:30:12.000Z

I've been meaning to implement the process mentioned by @davegravy in a program.
This is the program that I'm using in the void loop part.

`modem.sendAT("+SMCONN");
if (modem.waitResponse(1000L, res) == 1) {
res.replace(GSM_NL "OK" GSM_NL, "");
Serial.println(res);
Serial.println("connection!!!!");
}

res="";

modem.sendAT("+SMPUB = \"BasicPubSub\", 425,0,0");
    if (modem.waitResponse(10000L, res) == 1) 
    {
        res.replace(GSM_NL "OK" GSM_NL, "");
        Serial.println(res);
        Serial.println("datssent");
        res="";
        
        SerialAT.print(jsonBuffer);
      
        
        
    }

modem.sendAT("+SMDISC");
    if(modem.waitResponse(1000L,res) == 1)
    {
        Serial.println(res);
        res="";
        Serial.println("disconnectedf");

    }   
      
    res="";

}`

and this is the message that arrives on the topic

`AT+SMCONN
AT+SMPUB = "BasicPubSub", 425,0,0
AT+SMDISC
AT+SMCONN

AT+SMPUB = "BasicPubSub", 425,0,0

AT+SMDISC

AT+SMCONN

AT+SMPUB = "BasicPubSub", 425,0,0

AT+SMDISC

AT+SMCONN

AT+SMPUB = "BasicPubSub", 425,0,0

AT+SMDISC

AT+SMC `

What is the mistake that I am doing ?

Answer 39 · 2022-07-09T05:21:26.000Z

I have succeeded MQTTS to AWS using SIM7070G (Firmware B11) and hologram network.
Follow @davegravy flow, BUT be sure to update module time manually or using NTP to the current time! It did not work without updating to the correct time!

Answer 40 · 2022-08-05T13:43:15.000Z

Hoping someone can give me a 101 on generating / collecting the Certs because that's where I'm stuck.

I'm using a HiveMQ broker that's an AWS server. Can someone point me to a detailed how-to on creating / saving the certificates? I think I pulled one, put it in the customer folder, but no matter what I do, I can't get
at+csslcfg=convert,2,AWSCert.pem
to get accepted by the SIM7000. Always sends an ERROR code back. I can't tell if it's because the file itself is an issue or something else - like wrong file location.

I've opened up the board I have using QPST, and it doesn't contain any of the folders we see in the CFSWFILE documentation:
0 "/custapp/"
1 "/fota/"
2 "/datatx/"
3 "/customer/"
Tried to load a Cert using CFSWFILE but Putty sucks. Anyone got a better tool to use? I know I should probably not be using Win OS.

I created my own >>customer<< folder and put the Cert there. Also put the Cert in about 10 other folders to see if I could find the location it needs to be in, didn't work. I'm using B04 firmware.

Answer 41 · 2022-08-05T14:01:58.000Z

Once you open up the QPST tool, open start clients> EFS Explorer.
The EFS Explorer will show you the primary file system by default. On the tool bar you will find an option for an
"Alternative File System".
You will find the required folders in this and inside the customer folder you can just drag drop the cert files that you want.
(just make sure that the names and extensions are right when you are running AT commands).

Answer 42 · 2022-08-06T16:01:22.000Z

The EFS Explorer will show you the primary file system by default. On the tool bar you will find an option for an "Alternative File System".

This worked beautifully, thank you! I thought I was going crazy.

Now I just need the right certificates...

Answer 43 · 2022-08-23T21:03:08.000Z

@botletics I tried following the AWS IoT command log you posted however I am getting ERROR at AT+SMCONN.

How can I debug this? The same certificates and key allowed me to use their Python SDK to publish successfully.

AT+CCLK?
+CCLK: "21/05/31,04:30:24+00"

OK
AT+CPIN?
+CPIN: READY

OK
AT+CSQ
+CSQ: 29,99

OK
AT+CGREG?
+CGREG: 0,1

OK
AT+COPS?
+COPS: 0,0,"ROGERS ROGERS",7

OK
AT+CGNAPN
+CGNAPN: 1,"ciot"

OK
AT+CNACT=1,"ciot"
OK

+APP PDP: ACTIVE
AT+CNACT?
+CNACT: 1,"10.237.129.141"

OK
at+csslcfg=convert,2,AmazonRootCA1.pem
OK
at+csslcfg=convert,1,my_thing.cert.pem,my_thing.private.key
OK
AT+CSSLCFG="sslversion",0,3
OK
AT+SMSSL=1,AmazonRootCA1.pem,my_thing.cert.pem
OK
AT+SMCONF=url,##############-ats.iot.us-east-1.amazonaws.com,8883
OK
AT+SMCONF="clientid","basicPubSub"
OK
AT+SMCONF="KEEPTIME",60
OK
AT+SMCONF?
+SMCONF
CLIENTID: "basicPubSub"
URL: "##############-ats.iot.us-east-1.amazonaws.com:8883"
KEEPTIME: 60
USERNAME: ""
PASSWORD: ""
CLEANSS: 0
QOS: 0
TOPIC: ""
MESSAGE: ""
RETAIN: 0

OK
AT+SMCONN
ERROR

I also tried setting a topic and message (same topic that the python client successfully publishes to) with no luck.

You have a working code sample that connects to AWS MQTT via SIM70XX and ESP32 AND listens to a topic?

Answer 44 · 2022-08-23T21:55:00.000Z

AT+CCLK?
+CCLK: "21/05/31,04:30:24+00"

Fix the time. It has to be synchronized. You can use NTP for that.

URL: "##############-ats.iot.us-east-1.amazonaws.com:8883"

Remove '-ats' from the URL. You also need to use legacy certificate.
See "Supported legacy endpoints" section here:
https://docs.aws.amazon.com/general/latest/gr/greengrass.html

the '-ats' has been stripped from the endpoint address.

You have a working code sample that connects to AWS MQTT via SIM70XX and ESP32 AND listens to a topic?

My example here works for me when time is synchronized with NTP:
https://iot.stackexchange.com/questions/6347/connecting-cellular-module-sim7070g-to-aws-mqtt

Answer 45 · 2022-09-08T02:18:27.000Z

Hey guys, figured out how to connect with SSL without verifying certs! Not sure if this would work with AWS though...
Please do the following:

Set "SSL_FONA" to 1 in the .h file (remember to save the file before closing)
Download the latest .cpp file from GitHub
Open the unedited LTE_Demo example sketch and change "http://dweet.io" to "https://dweet.io" on line 1035 (under the '2' option)
Upload to the Arduino
Run the 'G' command to enable data connection, then run '2' to send data to dweet.io using SSL. And voila! 😊

Answer 46 · 2022-09-27T07:12:31.000Z

I finally figured this out!

I now have AT+SMCONN succeeding with AWS (haven't tested anything beyond this yet)

There's 2 important non-obvious steps.

Whatever url AWS gives you for your IoT Core endpoint, you have to strip out the "-ats" from it. So "a1k9ecig9j720o-ats.iot.us-east-1.amazonaws.com" becomes "a1k9ecig9j720o.iot.us-east-1.amazonaws.com"

You need to use the legacy root certificate provided by AWS here under "VeriSign Endpoints (legacy)". NOTE some regions apparently don't support legacy certs, so it seems the SIM7000 won't work in those.

I also updated my SIM7000A to the B04 firmware from B03, not sure if that mattered.

Contrary to what @tomlankhorst posted, it doesn't seem to matter if your RTC is sync'd.

I am using sim7000g.I have used RSA 2048 bit key: [VeriSign Class 3 Public Primary G5 root CA certificate] but still I am unable to connect with aws. AT+SMCONN returns error. my firmware version is also upgraded.Need help.?

Answer 47 · 2022-12-04T03:29:27.000Z

Please see this

Answer 48 · 2023-04-04T03:54:40.000Z

I was stuck at the same situation of my 7080G that AT+SMCONN was failed by error,

I followed the guidance from #58 but still go nowhere.

"Whatever url AWS gives you for your IoT Core endpoint, you have to strip out the "-ats" from it. So "a1k9ecig9j720o-ats.iot.us-east-1.amazonaws.com" becomes "a1k9ecig9j720o.iot.us-east-1.amazonaws.com" You need to use the legacy root certificate provided by AWS here under "VeriSign Endpoints (legacy)". NOTE some regions apparently don't support legacy certs, so it seems the SIM7000 won't work in those."

I also checked AT+CCLK and saw the time was correct.

Any suggestion ?

thanks !

Answer 49 · 2023-08-24T02:42:15.000Z

@wenjun1972 I'm having the exact same issue with the 7080G. Were you able to figure out how to correct it?

Answer 50 · 2023-08-24T02:50:39.000Z

@wenjun1972 I'm having the exact same issue with the 7080G. Were you able to figure out how to correct it?

Did you check that you are online with the network in the 1st place?
Did you check the time and confirm it's synchronized?
Did you upload the certificates and checked that module can find them?

Please copy the whole AT command sequence after you power up.

Answer 51 · 2023-08-24T15:49:59.000Z

Yes, network is online, I know this cause I'm able to connect to other MQTT brokers that don't require SSL.
I did check and set the time manually, I'm not sure if I did it correctly though. I just used the AT+CCLK command and set it to local time.
I did upload the certificates and I know the module could find them since the result says DOWNLOAD and when I use the command AT+CFSGFIS to get the file sizes it returns correct values.

The AT command sequence I use to connect is this:
AT+CSQ
AT+CFUN=0
AT+CGDCONT=1,"IP","hologram"
AT+CFUN=1
AT+CNCFG=0,1,"hologram"
AT+CNACT=0,1
AT+CFSINIT
AT+CFSWFILE=3,"veriSign.crt.pem",0,1758,5000
AT+CFSWFILE=3,"certific.pem.crt",0,1220,5000 Certificate obtained from AWS
AT+CFSWFILE=3,"private.pem.key",0,1675,5000 Private key from AWS
AT+CFSTERM
AT+CSSLCFG="CONVERT",2,"veriSign.crt.pem"
AT+CSSLCFG="CONVERT",1,"certific.pem.crt","private.pem.key"
AT+SMSSL=2,"veriSign.crt.pem","certific.pem.crt"
AT+CSSLCFG="SSLVERSION",0,3
AT+SMCONF="URL",xxxx.iot.ca-central-1.amazonaws.com I did remove -ats from the URL
AT+SMCONF="CLIENTID","test"
AT+SMCONN

Thanks for any help!

Answer 52 · 2023-08-24T16:00:59.000Z

My flow here is different than yours a little bit. For example AT+CSSLCFG command. I also did not do AT+CFSTERM. Doesn't it close access to the file system, so certs cannot be pulled in?

Answer 53 · 2023-08-24T20:17:27.000Z

So I just tried following your flow exactly but still no connection. I also tried using your method of setting the time but when I do so, AT+CCLK doesn't return the correct time so I just set it manually.

Would it be because of the region I'm using? I'm using CA-Central and it's not listed on Supported Legacy Endpoints but when I try connecting to AWS with the Legacy certificate using Mosquitto it does work.

Answer 54 · 2023-08-24T20:29:02.000Z

Regardless of Mosquitto operation, I'd really try switching regions just to be sure.

For the time: why don't use use NTP? Just let the module pull the time on its own using AT+CNTP. Just be sure to select the right parameters, so they match your time zone.

Also, how does your device policy look like in AWS?
I'd recommend using a very loose policy to begin with. Something like:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "iot:*", "Resource": "*" } ] }

Answer 55 · 2023-08-24T20:47:24.000Z

So it was the region that was the issue. I switched it over to one of the ones listed here under Supported Legacy Endpoints. Thanks for the help!

Answer 56 · 2023-09-22T07:50:26.000Z

How did you sign your client certificates @mrbacani ? How do you obtain the certificate and private key from aws, don you have to use the verisign certificate to sign the client certificates?

Answer 57 · 2023-09-22T08:03:13.000Z

Having, similar problems. I am using the verisgn g5 Root CA, and AWS generated cert / key files. I followed @mrbacani 's flow, double checked the time, etc. But still get an error. I also get an error when using a python pahomqtt script with the same certs. My current thought is a mismatch between root ca and generated cert / key. Any ideas?

Answer 58 · 2024-02-06T20:29:14.000Z

After a couple of days banging my head to the docs... it worked. AT based MQTT over SSL that is, with client cert id.
There are many little things that can go wrong, and all that you get is an "ERROR".
One that was problematic (in my case) is that I had a proxy ESP32, so certs loading was error prone. But the last one was hard. The key has to be in "traditional" format ("RSA PRIVATE KEY"). If you create the client cert with openssl 3, the new format does not work and "convert",1 fails. You need that for the AT+SMSSL command... I was not able to do SSL w/o specifying a client cert,
Hope this helps someone...

Answer 59 · 2024-02-07T15:58:28.000Z

I have encountered with this issue. The problem seems related to the SSL version3. My private.pem.key is like this:
-----BEGIN RSA PRIVATE KEY-----
something
-----END RSA PRIVATE KEY-----

and certificate.pem.crt:

-----BEGIN CERTIFICATE-----
somrthing
-----END CERTIFICATE-----

Does anybody know how I can do pre-convert on the "certificate.pem.crt" and "private.pem.key" in terminal, then upload it to SIM7000e?

I tried this command but doesn't work again:

"openssl rsa -in private.key -out private-rsa.key -traditional
"openssl rsa -in private.key -traditional -out private-rsa.key

The existed mosquitto.key and mosquitto.crt works fine in AT+CSSLCFG="CONVERT",1,"mosquitto.crt","mosquitto.key".

I also tried to copy and paste my own cert and key in the existed file, but it ain't work.

any advice?

Answer 60 · 2024-02-08T06:29:05.000Z

I have encountered with this issue. The problem seems related to the SSL version3. My private.pem.key is like this: -----BEGIN RSA PRIVATE KEY----- something -----END RSA PRIVATE KEY-----

and certificate.pem.crt:

-----BEGIN CERTIFICATE----- somrthing -----END CERTIFICATE-----

Does anybody know how I can do pre-convert on the "certificate.pem.crt" and "private.pem.key" in terminal, then upload it to SIM7000e?

I tried this command but doesn't work again:

"openssl rsa -in private.key -out private-rsa.key -traditional

"openssl rsa -in private.key -traditional -out private-rsa.key

The existed mosquitto.key and mosquitto.crt works fine in AT+CSSLCFG="CONVERT",1,"mosquitto.crt","mosquitto.key".

I also tried to copy and paste my own cert and key in the existed file, but it ain't work.

any advice?

I don't think you need to convert the certificates to use them. The .crt and .key are already in the .pem format. You can just use them directly, one good tip would be rename them to something much more simpler like prvt.key and certificate.crt.

Answer 61 · 2024-02-08T10:38:02.000Z

Hi Reddy, I used this at command too:
AT+CSSLCFG="CONVERT",1,"mycert.crt","private.key", but it doesnt work. Im using B01 version should I upgrade the Sim7000e S/W?
I'm using sim7000e rpi hat, with thingsmobile sim card.

Answer 62 · 2024-02-08T11:32:18.000Z

Aryan, if your pem key file has the "RSA" in the header, then it is in traditional format and that part is ok.
I would go step by step verifying what's ok and what's not. Does the "convert" command (AT+CSSLCFG) work with your cert ? (you said no) Does it work with another cert ? (you said yes) Then your cert has some issue. Verify extensions ? "openssl -noout -text -in yourcert.pem" and compare to "openssl -noout -text -in goodcert.pem"
HTH

Answer 63 · 2024-02-08T12:10:56.000Z

Hi Tronar,
I have three certs once I create the thing in AWS IoT core:

AmazonRootCA1.pem
-----BEGIN CERTIFICATE-----
something
-----END CERTIFICATE-----
1294cbc163a64792fca0a883ccaabebe5aa7c4ccd509224d127971***********-certificate.pem.crt
1294cbc163a64792fca0a883ccaabebe5aa7c4ccd509224d127971***********-private.pem.key

For the first one: AT+CSSLCFG="CONVERT",2,"AmazonRootCA1.pem" ----> OK: this one works fine.
While this one: AT+CSSLCFG="CONVERT",1,".....certificate.pem.crt",".....private.pem.key" ----> ERROR: Not working
and this one: AT+CSSLCFG="CONVERT",1,".....certificate.pem",".....private.pem" ----> ERROR: Not working

Should I enable the something before use this? (I don't think so).
Should I upgrade the S/W to B09?

FYI: the certs works fine with the python library!

import AWSIoTPythonSDK.MQTTLib as AWSIoTPyMQTT
myAWSIoTMQTTClient = AWSIoTPyMQTT.AWSIoTMQTTClient(CLIENT_ID)
myAWSIoTMQTTClient.configureEndpoint(ENDPOINT, 8883)
myAWSIoTMQTTClient.configureCredentials(PATH_TO_AMAZON_ROOT_CA_1, PATH_TO_PRIVATE_KEY, PATH_TO_CERTIFICATE)

Answer 64 · 2024-02-08T12:20:08.000Z

Aryan, those are not 3 certs.
The AmazonRootCA1.pem is a cert, a CA cert, used to validate the server cert when connecting using TLS.
The 1294..-certificate.pem.crt is your device cert, and the 1294...private.pem.key is your device key. (private key)
The cert holds a public key, and the corresponding private is in the key file. PEM is a data format.
What is the first line of the key file ? What is the output of "openssl x509 -noout -text -in 1294...certificate.pem.crt" ?

Answer 65 · 2024-02-08T12:59:50.000Z

The first line of the key file::
-----BEGIN RSA PRIVATE KEY-----

the first line of 1294..-certificate.pem.crt:
-----BEGIN CERTIFICATE-----

regarding this: Verify extensions ? "openssl -noout -text -in yourcert.pem" and compare to "openssl -noout -text -in goodcert.pem"

"What is the output of "openssl x509 -noout -text -in 1294...certificate.pem.crt" ?"

mycert (1294..-certificate.pem.crt):
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
f2:e7:58:b4:c5:2c:61:06:6c:e2:7e:9c:80:46:d4:56:4d:9f:2f:96
Signature Algorithm: sha256WithRSAEncryption
Issuer: OU = Amazon Web Services O=Amazon.com Inc. L=Seattle ST=Washington C=US
Validity
Not Before: Nov 1 10:13:51 2023 GMT
Not After : Dec 31 23:59:59 2049 GMT
Subject: CN = AWS IoT Certificate
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:ae:fd:f6:8a:60:5e:23:ef:81:1b:02:37:1c:70:
0f:5b:7c:6f:cb:68:82:43:c9:51:6d:65:df:9d:82:
e6:64:87:14:60:f8:bd:4e:c3:c6:71:2f:c0:07:d4:
ee:53:4d:54:cf:56:86:da:30:3c:db:20:fa:52:65:
9c:33:fd:eb:94:07:d5:8f:77:53:ce:8e:a0:f1:db:
e3:33:a5:09:f4:be:58:c3:06:31:14:e8:12:cc:78:
ef:c1:7a:c3:2a:49:39:18:78:06:4d:b1:a8:a3:dc:
61:f3:62:3e:20:6d:13:57:d8:71:cd:11:61:aa:35:
40:86:ca:e9:55:6d:b8:46:3e:2a:ef:18:cf:d4:48:
a0:9b:c9:c0:35:e0:d5:47:64:83:bc:cd:d4:ae:03:
98:68:19:f6:5b:48:e5:ed:0b:ad:49:a3:ed:f8:91:
5f:27:d0:bb:1f:31:ec:90:e5:9b:4e:a1:b2:e2:5b:
c3:ba:24:c2:99:0d:4e:00:a2:03:5a:e1:c2:40:16:
04:72:e0:6c:23:75:eb:69:b6:e2:ea:32:71:9c:d0:
3a:41:50:d6:d7:69:3a:52:26:73:ad:ad:da:e5:fe:
6c:8e:b5:6e:73:2e:f5:40:f4:61:03:93:09:cc:05:
49:a5:52:58:5e:8f:10:67:16:5d:90:5a:b9:65:63:
fb:21
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Authority Key Identifier:
A0:82:E7:18:27:11:92:AC:E9:28:45:09:6D:D7:37:EA:4A:14:A1:4B
X509v3 Subject Key Identifier:
FD:91:4B:59:0D:D5:F6:73:73:6C:82:7F:D1:D2:D2:0F:7B:0C:FC:DF
X509v3 Basic Constraints: critical
CA:FALSE
X509v3 Key Usage: critical
Digital Signature
Signature Algorithm: sha256WithRSAEncryption
Signature Value:
56:f2:9e:e7:e9:ba:61:3b:b7:a2:70:1f:f5:de:fb:9a:d7:e3:
d0:4d:d4:06:e2:66:91:06:d2:48:1e:b3:77:28:99:d4:45:10:
7b:bb:c3:bd:b1:7a:0d:f0:22:83:be:ca:2d:f5:c5:f8:9d:3f:
a0:d4:ff:59:86:bc:36:7a:ec:af:2a:7a:d1:de:df:65:10:64:
58:a5:62:3c:1a:3c:d5:e1:d0:22:1e:6a:36:8b:23:8e:36:79:
66:da:83:19:83:ef:b5:ef:88:b6:a2:12:f6:7c:60:dc:bf:27:
00:dd:0d:6e:90:26:41:af:b8:60:2a:ea:2d:11:8f:c2:d9:21:
76:4a:f7:87:0d:bf:86:8e:37:03:25:8b:74:98:3a:dc:37:24:
7e:25:49:3c:f8:26:72:48:e1:e4:c0:9c:21:7c:91:0d:3b:5e:
02:61:e0:4c:04:44:3f:d6:d9:59:84:eb:8c:36:98:12:2b:8a:
87:b1:15:35:fb:08:91:7f:ab:03:c2:1b:f5:66:c1:3b:6f:d8:
10:dc:b5:21:a2:54:26:38:d2:92:6d:1f:20:18:ef:b3:f7:f3:
54:84:98:6c:d3:7c:39:da:32:59:c9:49:fb:64:d4:40:51:a2:
de:51:8e:0b:92:6c:a5:76:3a:31:55:83:fe:ad:5a:51:79:00:
a3:ae:6f:69

mosquitto.crt (one that works fine):
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 0 (0x0)
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = GB, ST = United Kingdom, L = Derby, O = Mosquitto, OU = CA, CN = mosquitto.org, emailAddress = roger@atchoo.org
Validity
Not Before: Feb 5 13:06:10 2019 GMT
Not After : May 6 13:06:10 2019 GMT
Subject: C = AU, ST = QLD, L = LABRADOR, O = SmartWorldBox, CN = Tom
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:d7:fe:a3:0c:ed:15:1f:c0:4d:90:7b:e5:34:af:
69:c3:d9:6f:da:b7:8e:47:45:1f:bd:43:a0:81:6d:
aa:5c:db:80:05:3d:d2:f2:1a:9c:c6:ae:05:39:4c:
c3:f7:90:93:a6:e6:29:72:02:c6:78:99:19:3d:03:
57:6b:3d:8e:61:4c:28:b9:97:e2:63:04:03:40:a9:
5d:65:84:5c:54:3c:c3:86:3b:28:ed:8c:51:39:16:
ab:3c:1c:32:82:ce:5a:67:2b:b0:ea:01:67:56:42:
c9:ee:27:32:f5:4e:07:36:08:b8:31:61:f7:96:70:
10:cd:12:c6:55:2e:37:52:af:f6:83:2b:73:89:85:
b3:aa:f6:af:a1:e2:f5:88:f8:6a:0c:20:e8:75:78:
de:bb:e9:05:87:eb:14:9c:a4:d9:34:e6:25:94:2d:
5f:de:73:9d:f4:56:7c:4b:90:7f:71:59:8d:c4:7a:
0e:68:c6:0c:ab:c5:ff:8f:f4:40:e8:d7:37:12:78:
a4:b0:a1:8a:3c:cf:3b:f4:cf:e1:79:68:ff:88:f0:
76:9c:c3:14:c1:5b:3b:41:44:0f:08:a1:88:17:19:
96:57:f2:29:63:21:aa:67:4e:3e:f6:7d:aa:d1:6c:
33:09:71:c8:5e:90:39:bc:1f:df:ae:eb:fe:d5:f5:
50:7f
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
X509v3 Key Usage:
Digital Signature, Non Repudiation, Key Encipherment
Signature Algorithm: sha256WithRSAEncryption
Signature Value:
4c:73:15:6d:cf:ad:97:14:cd:fb:ff:d5:fe:21:f7:c9:26:f6:
ef:52:c6:9a:c0:68:b0:b3:ab:b8:78:53:d0:b2:77:90:2c:5f:
64:63:0f:a2:4e:71:6d:3f:1e:49:93:16:a0:59:bf:c7:e3:6f:
a7:31:6f:92:0d:40:cd:1d:aa:69:aa:c4:9d:7c:7f:f3:ae:77:
17:09:57:c0:a7:88:c6:5f:63:c8:87:94:6c:42:9f:b8:99:01:
32:e9:cf:b0:cd:e0:56:75:c2:59:9e:fa:b8:fb:dc:3c:4d:e4:
d0:b6:65:93:a1:89:f1:92:91:a7:a2:20:1b:57:d8:20:d9:20:
7b:47

Answer 66 · 2024-02-08T13:04:37.000Z

Cert looks fine to me, I guess it's just that the file name is too long. Rename both cert and key to something like client.pem and client.key and try again...

Answer 67 · 2024-02-08T13:07:44.000Z

I did that for 1000 times, gonna to try it again...

GPT tell me this:

Here are the main differences between the two certificates:

Issuer:
The first certificate is issued by Amazon Web Services, while the second one is issued by Mosquitto.org.

Validity:
The first certificate is valid from Nov 1, 2023, to Dec 31, 2049, whereas the second one is valid from Feb 5, 2019, to May 6, 2019.

Subject:
The subjects of the certificates are different. The first one is issued to "AWS IoT Certificate", and the second one is issued to "Tom" by SmartWorldBox.

Public Key Algorithm:
Both certificates use the RSA encryption algorithm for their public keys.

Public Key Modulus:
The modulus values, which represent the public key, are different in both certificates.

X509v3 extensions:
The first certificate includes extensions such as Basic Constraints and Key Usage, whereas the second certificate doesn't include these extensions.

Signature Algorithm:
Both certificates use the sha256WithRSAEncryption signature algorithm.

Serial Number:
The first certificate has a serial number specified, while the second certificate has a serial number of 0 (0x0).

X509v3 Key Usage:
The first certificate has a critical Key Usage extension, while the second certificate does not specify a Key Usage extension.

Subject Key Identifier:
The first certificate has a Subject Key Identifier extension, while the second certificate doesn't include this extension.

I think the issue is related to the firmware of SIM7000E pi hat, mine is 01 but there 09 has been released.

Answer 68 · 2024-02-08T13:12:37.000Z

Oh, wait, I did not pay attention to the time, mosquitto is valid in 2019, what time does you module have ?
Have you disabled time verification ? (AT+CSSLCFG="ignorertctime",0,1)

Answer 69 · 2024-02-08T13:17:05.000Z

The funny thing is the AT+CSSLCFG="CONVERT",1,"mosquitto.crt","mosquitto.key" works fine w/o AT+CSSLCFG="ignorertctime",0,1.

Answer 70 · 2024-02-08T13:18:49.000Z

That might be because the module default time lies in the 2019 timeframe ? Look and see (AT+CCLK?)

Answer 71 · 2024-02-08T13:37:05.000Z

Negative again!
with or without AT+CSSLCFG="ignorertctime",0,1.
the AT+CSSLCFG="CONVERT",1,"mosquitto.crt","mosquitto.key works fine and the AT+CSSLCFG="CONVERT",1,"shorten.crt","shorten.key" doesn't work.

Answer 72 · 2024-02-08T13:40:58.000Z

I don't know then. Sorry,

Answer 73 · 2024-02-08T13:42:09.000Z

I appreciate your consideration. I'm going to update the firmware with 09 and try again.

Answer 74 · 2024-02-08T13:47:03.000Z

Are you sure your certs are uploaded correctly? Did you use Qualcomm tool to upload the certs to the file system or you are trying to do this through the terminal?

Answer 75 · 2024-02-08T13:49:24.000Z

Hia, I used QPST EFS Explorer to upload the certs. I tried via terminal things work fine on uploading certs.

Answer 76 · 2024-02-08T13:55:17.000Z

So the error is that the conversion command doesn't work or you cannot connect to AWS?

Answer 77 · 2024-02-08T14:02:28.000Z

The issue is I can't embark "device cert" and "device private key" while I can do the cert (AmazonRootCA1), hence I can't construct the MQTTS to connect (AT+SMCONN).
also the outcome of AT+SMSSL=1, "AmazonRootCA1","certificate.crt" is OK.
but AT+CSSLCFG="CONVERT",1,"certificate.crt","client.key" doesn't work. (ERROR)

Answer 78 · 2024-02-08T14:24:10.000Z

Something doesn't add up here. certificate.crt that you're trying to use does not exist in the file system screenshot you've provided.
Can you copy the exact sequence in the terminal?

Typically, you should do like this:

AT+CSSLCFG="convert",2,"LegacyRoot.pem"<CR><LF>
AT+CSSLCFG="convert",1,"certificate.crt","private.key"<CR><LF>
AT+CSSLCFG="sslversion",0,3<CR><LF>
AT+SMSSL=1,"LegacyRoot.pem","certificate.crt"<CR><LF>

And then try the connection if your module time is current.

Answer 79 · 2024-02-08T14:32:48.000Z

the screen shot was not updated. its a updated one + log

Answer 80 · 2024-02-08T15:22:07.000Z

A couple of notes:

Your file permissions seem to be not the same for the last 2 files?
You have to strip off -ats when connecting to AWS. Also check if your region supports legacy certificate connections.

Answer 81 · 2024-02-08T15:27:21.000Z

I can feel your pain cause I've been more or less in the same place a couple of days ago. I can try loading your cert if you paste the cert and private, but pasting the private is not something you should do.

Answer 82 · 2024-02-08T15:34:23.000Z

A couple of notes:

Your file permissions seem to be not the same for the last 2 files?

You have to strip off -ats when connecting to AWS. Also check if your region supports legacy certificate connections.

number 2 amended, working on latter one, but i dont think it gonna solve the problem.

Answer 83 · 2024-02-08T15:36:26.000Z

I can feel your pain cause I've been more or less in the same place a couple of days ago. I can try loading your cert if you paste the cert and private, but pasting the private is not something you should do.

Im looking for solve the problem:

private.pem.key
-----BEGIN RSA PRIVATE KEY-----
MIIEoAIBAAKCAQEArv32imBeI++BGwI3HHAPW3xvy2iCQ8lRbWXfnYLmZIcUYPi9
TsPGcS/AB9TuU01Uz1aG2jA82yD6UmWcM/3rlAfVj3dTzo6g8dvjM6UJ9L5YwwYx
FOgSzHjvwXrDKkk5GHgGTbGoo9xh82I+IG0TV9hxzRFhqjVAhsrpVW24Rj4q7xjP
1Eigm8nANeDVR2SDvM3UrgOYaBn2W0jl7QutSaPt+JFfJ9C7HzHskOWbTqGy4lvD
uiTCmQ1OAKIDWuHCQBYEcuBsI3Xrabbi6jJxnNA6QVDW12k6UiZzra3a5f5sjrVu
cy71QPRhA5MJzAVJpVJYXo8QZxZdkFq5ZWP7IQIDAQABAoIBAGLZyZQ/fc509Axq
wvEIiFRYxdo0rilWpu3Sd3BFypoNCEEDIgVFaGr121dRPFPIQllheon0Z9wtE9rJ
1WQ1UwdrKYOCl8/+GAKoAP9igm5DvGZmsAEsW7ovstgr3eWcOWmOG5l1+1qdGqPe
4lN06lFcTmTWJcJ9lHbQVuDQOyFJ+9woSZOf1moth/0KRhm9qlABWXK77x0hIxJs
8/m+YLIlgGCxBrvHd2LEuE4AqtJvTQ+30enLMFGKh696OHaUDQf768QiuHXx+wAm
fmDf+RcWyXIJ6vPWm67IjnwfahDnag2VTU08v4cnleKwrMmuafqwJipjhUERtyny
UZMnIkUCgYEA1F7rfaZi1jvB5uzE+H7DcF4NEB6KRV2oau0EJuGUOtQ0tGj7+e3u
r0ps5pz3J6zO90YnLfeEuaEoNCBleWWqN6bULRYd6Kv+T9r6RM1R5wt87927aRUU
lNEoNU6wGkDnSPiAfViRkf98CMZQPA59vNcueUmqqU8VYq5XqX1ZtoMCgYEA0vE1
DAGsYcklVw73O5TNjXMgfStiNyxFG4WAtBKijx4lJ5uz6IlJV1yrVWMnBH5egJAa
4KqIV+66CG3pWhf7Y/wu9Uh7UAkGWOq3dJQsEvHcCa4AF7Qk5RWkoyU+q09XVMl1
UjpuKrZm4Iu28AzShI/Vy5PeE/UnCBfECszE9osCgYB2J3FBcQbgRlL1FZno3y4B
IHKIG1W2jgsbok6DC1IbAOFp1lcKkFQRdojsLTxc+IoVjRRTQLi5Rm5Fwhhy2BtB
5zF4/CsbvkU5TI2dJdaBgyS5l1WjezT+Lvf94I+dq6qCMK/cDSDAZ7Isd5lAMJfI
LrgOhuvKUtOFGZZwF+uH3QKBgEyrNuiiQxFXlqbJ3bpeH0fmLEzSU+RRxtx17Y2F
qGf0QPTgdsdx/qIuIGfsneXYOGjp95ro4J11O1CNAl+oj8qLglXMfmVcol33Ea7h
nBNWrO8nuwjihPZuo2RYySpisA81GdtFOX10xnee0GL3hhyAWuifWfxPAlzCppJ2
UrB5An9qV/dMbaUWHVWSvjsFoc4+ce1/lvhTN7r9JhpfYz4c+h/MZvRcoAOvokER
v/utt6IiPb/CvMYBUlkf4mVOpoCLb5oi2nf5rO6awCu24iyEXTBbhniKVfP2Mc4g
r8RtmE2Hr6pzDyLIN5vyAtTo6GeDzubYjl+OkbD22MGnnDxl
-----END RSA PRIVATE KEY-----

certificate.pem.crt:
-----BEGIN CERTIFICATE-----
MIIDWjCCAkKgAwIBAgIVAPLnWLTFLGEGbOJ+nIBG1FZNny+WMA0GCSqGSIb3DQEB
CwUAME0xSzBJBgNVBAsMQkFtYXpvbiBXZWIgU2VydmljZXMgTz1BbWF6b24uY29t
IEluYy4gTD1TZWF0dGxlIFNUPVdhc2hpbmd0b24gQz1VUzAeFw0yMzExMDExMDEz
NTFaFw00OTEyMzEyMzU5NTlaMB4xHDAaBgNVBAMME0FXUyBJb1QgQ2VydGlmaWNh
dGUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCu/faKYF4j74EbAjcc
cA9bfG/LaIJDyVFtZd+dguZkhxRg+L1Ow8ZxL8AH1O5TTVTPVobaMDzbIPpSZZwz
/euUB9WPd1POjqDx2+MzpQn0vljDBjEU6BLMeO/BesMqSTkYeAZNsaij3GHzYj4g
bRNX2HHNEWGqNUCGyulVbbhGPirvGM/USKCbycA14NVHZIO8zdSuA5hoGfZbSOXt
C61Jo+34kV8n0LsfMeyQ5ZtOobLiW8O6JMKZDU4AogNa4cJAFgRy4GwjdetptuLq
MnGc0DpBUNbXaTpSJnOtrdrl/myOtW5zLvVA9GEDkwnMBUmlUlhejxBnFl2QWrll
Y/shAgMBAAGjYDBeMB8GA1UdIwQYMBaAFKCC5xgnEZKs6ShFCW3XN+pKFKFLMB0G
A1UdDgQWBBT9kUtZDdX2c3Nsgn/R0tIPewz83zAMBgNVHRMBAf8EAjAAMA4GA1Ud
DwEB/wQEAwIHgDANBgkqhkiG9w0BAQsFAAOCAQEAVvKe5+m6YTu3onAf9d77mtfj
0E3UBuJmkQbSSB6zdyiZ1EUQe7vDvbF6DfAig77KLfXF+J0/oNT/WYa8Nnrsryp6
0d7fZRBkWKViPBo81eHQIh5qNosjjjZ5ZtqDGYPvte+ItqIS9nxg3L8nAN0NbpAm
Qa+4YCrqLRGPwtkhdkr3hw2/ho43AyWLdJg63DckfiVJPPgmckjh5MCcIXyRDTte
AmHgTAREP9bZWYTrjDaYEiuKh7EVNfsIkX+rA8Ib9WbBO2/YENy1IaJUJjjSkm0f
IBjvs/fzVISYbNN8OdoyWclJ+2TUQFGi3lGOC5JspXY6MVWD/q1aUXkAo65vaQ==
-----END CERTIFICATE-----

Answer 84 · 2024-02-08T15:44:45.000Z

Do you think the issue is related to the firmware or the hardware being used?NB-IoT

Answer 85 · 2024-02-08T15:47:19.000Z

If your module SW doesn't have the permission or "see" the files in the filesystem - it will throw an error.

What do these return?

AT+CFSINIT
AT+CFSGFIS=3,"certificate.pem.crt"
AT+CFSGFIS=3,"private.pem.key"

P.S. You might still want to update your firmware to the latest. I See some are here: https://github.com/botletics/SIM7000-LTE-Shield/tree/master/SIM7000%20Documentation/Firmware

Edited: Thanks @tronar

Answer 86 · 2024-02-08T15:57:53.000Z

I loaded your certs in my 7000G and worked, so it's not cert related:
AT+CSSLCFG="convert",1,"aws.pem","aws.key"
OK
I have b08, so it may be firm related, or time related, or...

Answer 87 · 2024-02-08T16:00:50.000Z

I loaded your certs in my 7000G and worked, so it's not cert related: AT+CSSLCFG="convert",1,"aws.pem","aws.key" OK I have b08, so it may be firm related, or time related, or...

Thanks for feedback, I beleive its related to firmware: mine is b01.

Answer 88 · 2024-02-08T16:01:54.000Z

@Scrts, certs go in custommer, so should be AT+CFSGFIS=3, not 0, right ?

Answer 89 · 2024-02-08T16:24:18.000Z

@Scrts, certs go in custommer, so should be AT+CFSGFIS=3, not 0, right ?

You are correct. I've fixed my post. Thank you.

Answer 90 · 2024-02-09T18:49:26.000Z

I did update the firmware to b08, but the error still stands.
I think it might related to something else.

@tronar Did you just copy my certificate and key into a notepad and rename it before uploading? Did you use an Arduino or a Raspberry Pi?

Answer 91 · 2024-02-09T18:55:10.000Z

Yes, I made a couple of files named aws.pem and aws.key, uploaded them using a modified python script and tested by hand.
I'm using a Lilygo ESP-32+SIM7000D.

Answer 92 · 2024-02-09T18:56:47.000Z

Check that the line ends are LF and no CR+LF ?

Answer 93 · 2024-02-09T19:24:16.000Z

They all end by Unix (LF).

Answer 94 · 2024-02-09T19:29:42.000Z

Well, the other thing is the time. I would check if changing the current time renders your mosquitto cert invalid. If yes, then verify correct time. You have a simmilar (working) one, try to create another cert that works (learn the details of openssl :) and work from there. Start by creating a cert for the same keys ?

Answer 95 · 2024-02-09T19:33:39.000Z

I'm going to play with time and read about OpenSSL :)

Answer 96 · 2024-02-09T19:57:06.000Z

Just try the same certificate files on your PC connecting to AWS using Mosquitto. If that works, then your certificate files are OK.
Did you resolve the permissions? Did you try to check if the file system see them and correct size? See my post above.

Answer 97 · 2024-02-09T20:03:16.000Z

Just try the same certificate files on your PC connecting to AWS using Mosquitto. If that works, then your certificate files are OK. Did you resolve the permissions? Did you try to check if the file system see them and correct size? See my post above.

my files works with python library AWSIoTPyMQTT, they aint embark with SIM7000E rpi hat, I check the permission and size too. also the one in https://github.com/tmcadam/sim7000-tools works fine via AT+CSSLCFG="convert",1,"mosquitto.crt","mosquitto.key"

I tried to copy my cert and key to those file and tested, but same issue again.

Answer 98 · 2024-02-09T20:08:18.000Z

By the way, where are you located? In the USA?

Answer 99 · 2024-02-09T20:09:18.000Z

By the way, where are you located? In the USA?

If it helps to solve the problem :) United Kingdom

Answer 100 · 2024-02-09T20:10:04.000Z

Just wanted to be sure, since SIM7000E version is for European frequencies :-)