Problems with UTF-8 characters

Question

Problems with UTF-8 characters

nymoen opened this issue 10 years ago · 10 comments

Hi.
We have tried the Twitter for GeoEvent 10.2.x version.
It works, but the tweets we create does not shows the UTF-8 characters.
We have tried to figure out this problem, but we do not find the source of the problem.
Here you have an example of a tweet created by GeoEvent:

Any idea what we can do to solve this?

Regards
Kjetil N

Answer 1 · 2015-02-25T19:39:09.000Z

Thank you @nymoen - I've exported this issue and we will investigate it.

Answer 2 · 2015-03-02T10:06:27.000Z

@rsunderman : Have you been able to look into this?

Answer 3 · 2015-03-03T15:41:42.000Z

@nymoen I'm unable to reproduce this issue. I was able to connect TCP Input (using Simulator with converted file to UTF-8) to Send a Tweet.

Contents of file used in simulator:

ASCII 169: ©
ASCII 224: à
ASCII 232: è
ASCII 236: ì
ASCII 242: ò
ASCII 249: ù

See below for snapshot of tweets using Send Tweet.

Answer 4 · 2015-03-03T17:49:16.000Z

@cris6097 : In my case I just have a text in the Twitter Output message.
The text in the message is:
Dette er en test med bokstavene: æ,ø å: Æ,Ø,Å

PS. We do use the 10.2.2 version of GeoEvent.

Answer 5 · 2015-03-04T10:33:49.000Z

@cris6097 : I did one more test using the characters you used in your test (©, à, è, ì, ò, ù):
Here is the result in the tweet:

Answer 6 · 2015-03-04T19:18:02.000Z

@nymoen Thank you for the clarification. It seems this issue is in version 10.2.2 and 10.3.0 but it appears to be fixed in 10.3.1.

Answer 7 · 2015-03-04T19:27:22.000Z

@cris6097 So this is an issue in GeoEvent or in the Twitter Connector? We do have a client using this and they are not happy to send tweets with funny characters.

Answer 8 · 2015-03-04T19:49:54.000Z

@nymoen I believe its an issue with GeoEvent. Could you wait for 10.3.1 public release which is ~April 2015? If not, try reaching out to your sales rep for an early adopter program, which by the way I'm unaware if such programs exists.

Answer 9 · 2015-03-04T21:06:31.000Z

@cris6097 If this is an issue with GeoEvent, then the other outputs should give me funny characters also, correct? If I write to a file or send an email, I should also get the funny characters?

PS! I am not sure we can upgrade to 10.3.1 that easy, because this is a production site running 10.2.2.

Answer 10 · 2015-03-04T23:03:45.000Z

I tested the 10.2.2 release as well as the 10.3.0 release and our 10.3.1 release candidate using a Tcp/Text input to send event data to a CSV (system file), a JSON (system file), a GeoEvent Cache, and an e-mail output. The issue was not observed for any of these outputs on any of the releases tested. I do not think you will find that other outputs behave like the twitter-out.

The e-mail outbound connector is probably the closest to your use case. That outbound connector uses the same out-of-the-box MessageFormatter adapter as the twitter-out outbound connector.

Because the same 10.3.x binary JAR file was loaded into both the 10.3 release and 10.3.1 release candidate of GeoEvent - we are assuming the issue is somewhere in the core GeoEvent product rather than the Twitter implementation. However, as I mentioned above, the same out-of-the-box adapter is used for both twitter-out and esri-out-message-email-send. The e-mail output is not exhibiting the behavior on any product release. I cannot explain why we are seeing the issue we are seeing with the twitter-out.

I understand that you are working with a production system which is leveraging the GeoEvent 10.2.2 release, so upgrading will not be easy. Please contact Esri Technical Support and submit this issue as a bug against the 10.2.2 release if you expect that your only path forward will be to request a patch for the 10.2.2 product. Please understand that issue investigation and generation of a patch will have to be prioritized within the product development backlog - and the 10.3.1 release will likely be available before a 10.2.2 patch can be developed, tested, and released to you.