Why in Cordova, EventSource and/or fetch do not behave the same way they behave in a browser like chrome, when fetching SseEmitter tokens?
Closed this issue · 8 comments
In Google Chrome for example, EventSource and/or fetch are getting the packages from a SseEmitter endpoint, one by one, so that I can get the ChatGPT tokens generation effect.
But in my Cordova app, in both Android and iOS, the tokens generation effect is not possible, as EventSource and fetch calls, are buffering the backend response, delivering the text to the javascript code in a single push, after the server stopped emitting.
Any idea why this behavior in Cordova? Is it possible to get the same tokens generation effect in Cordova without using a websocket connection?
JS code for EventSource approach:
function handleGptSuggestCallEventSource(params, target) {
let eventSource = mwSuggestEventSource(params)
eventSource.onmessage = function(event) {
if (event.data.includes(FINAL_MESSAGE)) {
eventSource.close(); // Stop the EventSource
return;
} else {
let formattedText = event.data.replace(/\\n/g, '<br>');
formattedText = formattedText.replace(/\n/g, '<br>');
formattedText = formattedText.replace(/ /g, ' ');
updateTargetMessage(formattedText, target);
}
};
eventSource.onerror = function(err) {
eventSource.close();
};
}
function mwSuggestEventSource(params) {
return callGPTEventSource(GPT_SUGGEST_URL.format(params));
}
function callGPTEventSource(url) {
try {
return new EventSource(getFullURL(url));
} catch (e) {
showToastError(getStr(e));
}
}
Backed java code:
public SseEmitter callApiAsync(HttpRequest request) {
SseEmitter emitter = new SseEmitter();
executorService.execute(() -> {
try {
client.sendAsync(request, HttpResponse.BodyHandlers.ofInputStream())
.thenAccept(response -> {
try (InputStream inputStream = response.body()) {
BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
String line;
Matcher matcher;
while ((line = reader.readLine()) != null) {
matcher = GPT_TEXT_PATTERN.matcher(line);
if (matcher.find()) {
line = matcher.group(1);
} else {
line = "";
}
line = line.replace(" ", EMPTRY_SPACE);
emitter.send(SseEmitter.event().data(line)); // Stream each line to client
}
emitter.send(FINAL_MESSAGE);
emitter.complete(); // Complete the emitter after finishing reading
} catch (IOException e) {
emitter.completeWithError(e);
}
});
} catch (Exception e) {
emitter.completeWithError(e);
}
});
return emitter;
}
cordova --version
12.0.0 (cordova-lib@12.0.1)
cordova plugin list
cordova-plugin-android-permissions 1.1.5 "Permissions"
cordova-plugin-geolocation 4.1.0 "Geolocation"
cordova-plugin-inappbrowser 6.0.0 "InAppBrowser"
Any idea why this behavior in Cordova? Is it possible to get the same tokens generation effect in Cordova without using a websocket connection?
To my knowledge, Cordova doesn't do anything to effect the behaviour of server side events (SSE) or web sockets and they should work out of the box providing that the client is configured with the proper CSP policy on the client, and CORS policy on the server side. Cordova is afterall, a packager. It does not implement the browser features like websockets or SSE. Cordova does allow plugins, so unless if you have a plugin installed that overrides the EventSource JS object, then you're using the standard browser feature as implemented in the webview. Cordova itself doesn't implement any Event emitter plugins.
The browser application and the webview are two different components on the device and they could be built with different configurations, so there could be a difference of behaviour, but as long as the server is following the protocol I wouldn't expect any significant difference.
I'm not familiar with the Java library that you're using so I'll speak for a more general sense. For SSE, the server should be:
- Responding with
Content-Type: text/event-streamheader - Use double newline character (
\n) to indicate the end of an event package. If you're dealing with small data chunks, the server might have to explicitly flush the socket to forcefully send data through.
If the server is sending data without events, then the client should be listening for messages on the .onmessage handler/event. Otherwise the client should use .addEventListener("eventName").
For example if the server sends ping:\n\n it will send a ping event, which the client can listen for .addEventListener("ping", ...).
I'd assume that your SseEmitter class would be handling these details, but if you want to rule it out potential issues with the SseEmitter implementation, it's fairly straight forward implementing your own SSE emitter using a raw http server. If it works consistently on both the webviews and browsers with your own emitter server, then it would indicate a problem inside the SseEmitter implementation.
@breautek Thanks for your reply!
I don't believe there is a problem with the back end or with the javascript EventSource handling code, as it is working correctly on standard browsers, on both PC and mobile phones. On javascript side (see above) I am listening on the .onmessage handler/event as I am not setting events on the server side.
The problem is only visible within cordova application, where I am using the exact same javascript code and the exact same backed endpoint. Still the tokens are not being processes one by one in the cordova app.
The "Content-Type: text/event-stream" header is being set too for the connections from cordova.
One thing that I noticed while debugging this problem in cordova using google chrome , is that the stream tab in the network connection only shows up at the end, once the backed signals the completion of the send to client action. In standard browsers this tab shows up at the moment the first package is being sent to the client.
As for the plugins, I only use the plugins mentioned above. And to my knowledge, none of them is altering the internet connection. So I don't think is a plugin problem either.
Based on all these, I tend to believe there might be a webview problem or a limitation...
In that cause I'd recommend seeing if the problem can be reproduced in a sample project.
I'd suggest including both a cordova project and nodejs project.
The NodeJS project can use http-server package to serve a sample site with an SSE endpoint. It could also a HTML document that can showcase it working with a standard browser. The endpoint can be hittable from an android simulator or a physical device that is connected over the same local network.
The cordova project can be a basic "Hello Cordova" project that uses the NodeJS server. For android, you will need to provide a network security policy to allow plaintext connections (so that you can connect to a http server). To do this, you'll need to put an xml file:
<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
<base-config cleartextTrafficPermitted="true"/>
</network-security-config>inside platforms/android/app/src/main/res/xml/networkPolicy.xml
And inside platforms/android/app/src/main/AndroidManifest.xml, the <application> tag needs the following attribute: android:networkSecurityConfig="@xml/networkPolicy".
If a zip can be provided that demonstrates the issue in a basic project, I'll try to take a look in a reasonable timeframe.
If you want to see the behavior you may install my android app, available for download here: https://www.kelteu.com/kelteu.apk. To reproduce the problem, once the app is installed, open it, and tap on the question mark icon (?) associated to one of the products. On the new dialog tap on one of the available prompts, and wait. You can debug the backend calls in the mwutils.js, functions: mwSuggest or mwSuggestEventSource. The JS code is not obfuscated so it should allow you to debug it on your system
You can see the same code running in your webpage if you open www.kelteu.com
The iOS version of the app is available here : https://apps.apple.com/us/app/kelteu/id6670342956?kelteu
That's not really sufficient.
Having the ability to reproduce is one issue and while your application might be able to show case that. Having a project that can be manipulated and experimented with to actually troubleshoot the issue is another requirement.
So it will be better if a sample reproduction project can be provided.
Your APK isn't debuggable but I can still observe logcat messages which shows:
java.io.FileNotFoundException: www/an/gpt/suggest
2024-11-05 11:07:39.746 8226-8266 System.err pid-8226 W at android.content.res.AssetManager.nativeOpenAsset(Native Method)
2024-11-05 11:07:39.746 8226-8266 System.err pid-8226 W at android.content.res.AssetManager.open(AssetManager.java:985)
2024-11-05 11:07:39.746 8226-8266 System.err pid-8226 W at org.apache.cordova.engine.SystemWebViewClient.lambda$new$0$org-apache-cordova-engine-SystemWebViewClient(SystemWebViewClient.java:98)
2024-11-05 11:07:39.746 8226-8266 System.err pid-8226 W at org.apache.cordova.engine.SystemWebViewClient$$ExternalSyntheticLambda2.handle(D8$$SyntheticClass:0)
2024-11-05 11:07:39.746 8226-8266 System.err pid-8226 W at androidx.webkit.WebViewAssetLoader.shouldInterceptRequest(WebViewAssetLoader.java:571)
2024-11-05 11:07:39.746 8226-8266 System.err pid-8226 W at org.apache.cordova.engine.SystemWebViewClient.shouldInterceptRequest(SystemWebViewClient.java:438)
2024-11-05 11:07:39.746 8226-8266 System.err pid-8226 W at WV.H6.a(chromium-TrichromeWebViewGoogle6432.aab-stable-672305838:86)
2024-11-05 11:07:39.746 8226-8266 System.err pid-8226 W at org.chromium.android_webview.AwContentsBackgroundThreadClient.shouldInterceptRequestFromNative(chromium-TrichromeWebViewGoogle6432.aab-stable-672305838:15)
2024-11-05 11:07:39.746 8226-8266 SystemWebViewClient pid-8226 E www/an/gpt/suggest
Your application is trying to hit www/an/gpt/suggest which is likely suppose to be hitting a remote server, but you're requesting against the local device/webview instead.
This can happen if you're not using a fully qualified URL. For example if you request www/an/gpt/suggest the webview will implicitly use the active origin to form a fully qualified URL.
To hit a remote server, it must be explicitly a fully qualified url, e.g. https://kelteu.com/www/an/gpt/suggest
So this suggest its an application issue, not a framework/webview issue.