Connection to re-directed URL failed! - _redirHost: script.googleusercontent.com

[BobRak]

Hi:

Thanks for the excellent work on the HTTPS with redirect library. I am unable to get a simple example to work when redirecting to script.googleusercontent.com. Am suspecting the something has changed on the Google side but I am not getting enough debug information to understand what the problem is. I am hoping you have seen this before and can help.

My output is:

Free heap: 38120

Connecting to wifi: Raker
......
WiFi connected
IP address: 
192.168.86.76
Connecting to script.google.com

GET: To host: script.google.com with url: /macros/s/AKfycbz8KJ-a4OYMjXhZXsxQOmfazq3aYqK7zRq0nmCvMVWNKQiGoig/exec
=========================
GET /macros/s/AKfycbz8KJ-a4OYMjXhZXsxQOmfazq3aYqK7zRq0nmCvMVWNKQiGoig/exec HTTP/1.1
Host: script.google.com
User-Agent: ESP8266



Status code: 302
Reason phrase: Moved Temporarily
_redirHost: script.googleusercontent.com
_redirUrl: /macros/echo?user_content_key=A2K8SyhRJVt-u0QMZUFihMMrY_qlk76bjDjAPsjBTou1y7sPT4ZXVE3pFQFRDNIfmjrgtX8My9ZSY10nvBZBKFkf2D-CVQetm5_BxDlH2jW0nuo2oDemN9CCS2h10ox_1xSncGQajx_ryfhECjZEnOVljp8mS3kEa2FoOegHt6kv1mgV-1PxrV4_cEt9oXVAHc0gNdal2pujvWSqncGR8dveWPraiRQB&lib=Mm1_qotei4b8Ku2zl9ZtU8xxIoJWoTElh

Connection to re-directed URL failed!

GET: Fetch Google Calendar Data:
================================

If I take the redirect host and url and combine them to get:

https://script.googleusercontent.com/macros/echo?user_content_key=WoLbL9_qluslH-nlwIBY_9r_RvlbUBI3HB6_57KRYTYKN0R9A7ai52cSR5Bvp7-N1XVc3n7I7WPL5ICon8jmH4KLBeTS6s6Em5_BxDlH2jW0nuo2oDemN9CCS2h10ox_1xSncGQajx_ryfhECjZEnOVljp8mS3kEa2FoOegHt6kv1mgV-1PxrV4_cEt9oXVAHc0gNdal2pujvWSqncGR8Z-hMl8NVc9aIA_rwgebdRAVRVku1w4JtQ&lib=Mm1_qotei4b8Ku2zl9ZtU8xxIoJWoTElh

and then paste that into a browser window it works and I get back the expected results. Therefore I don't think the problem is with the generated url or with my google app script. According to your documentation is seems like something may have changed in Dec 2018. Could this be related to that?

The test script is a simplified version of your script. Here is the script.

/*  HTTPS on ESP8266 with follow redirects, chunked encoding support
 *  Version 3.0
 *  Author: Sujay Phadke
 *  Github: @electronicsguy
 *  Copyright (C) 2018 Sujay Phadke <electronicsguy123@gmail.com>
 *  All rights reserved.
 *
 *  Example Arduino program
 */

#include <ESP8266WiFi.h>
#include "HTTPSRedirect.h"
#include "DebugMacros.h"

// Fill ssid and password with your network credentials
const char* ssid = "xxxxx";
const char* password = "xxxxx";

const char* host = "script.google.com";
// Replace with your own script id to make server side changes
const char *GScriptId = "AKfycbz8KJ-a4OYMjXhZXsxQOmfazq3aYqK7zRq0nmCvMVWNKQiGoig";

const int httpsPort = 443;

// echo | openssl s_client -connect script.google.com:443 |& openssl x509 -fingerprint -noout
const char* fingerprint = "";
//const uint8_t fingerprint[20] = {};

// Read from Google Calendar
String url = String("/macros/s/") + GScriptId + "/exec";

HTTPSRedirect* client = nullptr;
// used to store the values of free stack and heap
// before the HTTPSRedirect object is instantiated
// so that they can be written to Google sheets
// upon instantiation
unsigned int free_heap_before = 0;

void setup() {
  Serial.begin(115200);
  Serial.flush();
  
  free_heap_before = ESP.getFreeHeap();
  Serial.printf("Free heap: %u\n", free_heap_before);
  
  Serial.println();
  Serial.print("Connecting to wifi: ");
  Serial.println(ssid);
  // flush() is needed to print the above (connecting...) message reliably, 
  // in case the wireless connection doesn't go through
  Serial.flush();

  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());

  // Use HTTPSRedirect class to create a new TLS connection
  client = new HTTPSRedirect(httpsPort);
//  client->setInsecure();
  client->setPrintResponseBody(true);
  client->setContentTypeHeader("application/json");
  
  Serial.print("Connecting to ");
  Serial.println(host);

  // Try to connect for a maximum of 5 times
  bool flag = false;
  for (int i=0; i<5; i++){
    int retval = client->connect(host, httpsPort);
    if (retval == 1) {
       flag = true;
       break;
    }
    else
      Serial.println("Connection failed. Retrying...");
  }

  if (!flag){
    Serial.print("Could not connect to server: ");
    Serial.println(host);
    Serial.println("Exiting...");
    return;
  }

/*  
  if (client->setFingerprint(fingerprint)) {
    Serial.println("Certificate match.");
  } else {
    Serial.println("Certificate mis-match");
  }
*/ 

  Serial.printf("\nGET: To host: %s with url: ", host);
  Serial.println(url);
  Serial.println("=========================");

  // fetch calendar data
  client->GET(url, host);

  Serial.println("\nGET: Fetch Google Calendar Data:");
  Serial.println("================================");

  int statusCode = client->getStatusCode();
  String statusPhrase = client->getReasonPhrase();
  Serial.printf("\nGET: Status code: %s with phrase: ", statusCode);
  Serial.println(statusPhrase);
  Serial.println("=========================");

  String response = client->getResponseBody();
  Serial.printf("\nGET: Response is: ");
  Serial.println(response);
  Serial.println("=========================");
  
 
  Serial.printf("Free heap: %u\n", ESP.getFreeHeap());

  // delete HTTPSRedirect object
  delete client;
  client = nullptr;
}

void loop() {
  delay(5000);
  Serial.println("In loop waiting");
}

BTW - the google app script returns the titles of currently due calendar events. If there are no active calendar events then nothing will be returned. Therefore If you get this to work but don't get any response data that could be OK.

Thanks for your help. Please let me know if you need anymore information.

Bob

[electronicsguy]

Yes indeed things have changed late 2018. The esp8266 library moved to a new version of handling SSL by using BearSSL. In the course of that, I'm still figuring out how to make the fingerprint verification work.

But to avoid fingerprint verification, we have to set the mode to insecure by using "client->setInsecure();"
Readme

Please update to the latest version of esp8266 library, uncomment that line out and try again.

[BobRak]

@electronicsguy
That did it. Thanks very much for the quick response. I had read your Update Dec 2018 in the README.md but didn't get that I needed to add the setInsecure() call.

Good luck in getting the use of certificates figured out. I did read some of the BearSLL.org website and it does seem quite complex. That will be much better than fingerprints since the certs are longer lived.

Bob