Hoaxy bot scores don't match Botometer when using Twitter search with non-English language

[glciampaglia]

When using Twitter search mode, tthere is mismatch with the results reported in Botometer's web interface. It looks like that when the account has a language other than English, Botometer does show the score from the universal classifier, while Hoaxy shows the score from the English classifier.

How to reproduce this issue:

1. Go on Hoaxy and enter the following query: Mare Jonio
2. Select Italian
3. Click Search
4. Now select only the red nodes, and look for a user called @Max_Tacconi. It should be red (score 4.1).
5. Go on Botometer and enter @Max_Tacconi and click 'check user'. It will show yellow, not red (score 2.6).

Solution: I propose we do the same in Hoaxy: When the user is selecting a language other than English, show scores from the universal classifier.

[ZacMilano]

@yangkcatiu (@Fil here) is user profile 'lang' information available via the bot cache API?

[yang3kc]

We don't store the 'lang' information of the users in the database, so no.

[filmenczer]

Would it be hard to add the 'lang' field from the user profile metadata in the API response? It would allow us to address this issue raised by @glciampaglia ...

[yang3kc]

That would require modification for Botometer-website, Botometer-API, Botometer-database, Hoaxy-Botometer API, so yes, kinda hard.

[yang3kc]

Is it possible for Hoaxy-fontend to work like this: if the user choose language specific search, then Hoaxy will display universal score which is in the current api.

[filmenczer]

For Twitter search it's not a problem; for Hoaxy search, the data is coming from the Hoaxy API, which does not provide 'lang' for nodes.

[yang3kc]

Hoaxy database has the raw tweets stored, so it should be relatively easy to embed the 'lang' into the Hoaxy API.

But I think the news sources Hoaxy is tracking are all English based. So do we even care about the language issue?

[marcmccarty]

I fixed it, at least on Twitter search. The response data from Botometer includes the universal score, as @yangkcatiu said, although I wish I would have seen these comments before figuring it out on my own haha.

At any rate, it was just adding an if-else to check for lang == 'en' || lang == 'en-gb' in 3 places in the frontend. Because of the way the frontend's coded, it should work for Hoaxy as well, because a user that claims their profile language is Japanese could be bilingual and post in English and end up on Hoaxy, but we'd probably still want to consider their universal score as opposed to the english one.

Edit: Accidentally posted from cadre account. Also, if you try to search non-English things on the Hoaxy search, it gives interesting results. Searching 日本語 gave The Standing Sioux Tribe Just Responded to Trump's Dakota Pipeline Order

[marcmccarty]

Completed and will be deployed tomorrow or otherwise ASAP.

[filmenczer]

Thank you all. I understand this is fixed for Twitter search, but am confused how it would work for Hoaxy search. Reopening so we can discuss at next meeting please.

[filmenczer]

For Twitter search, this should be fixed. @marcmccarty will double check that when the search is in a language other than en/en-GB, the front-end will get the universal score from the Botometer DB API.

For Hoaxy search, this is fixed once the user updates the scores. For the initial scores obtained from the Botometer DB API, we will add a new front-end configuration parameter to specify the default language of the Hoaxy installation. If this is anything other than English, the front end will fetch the universal score. Be sure that this is documented for people installing Hoaxy front-end.

[marcmccarty]

This is how it's currently setup, during the axios ajax request:

var botcache_request = axios({
	method: 'post',
	url: configuration.botcache_url,
	responseType: "json",
	data: {
		"user_id": user_id_list_chunk.join(",")
	}
});

botcache_request
.then(
	function(response)
	{
		spinStop("getBotCacheScores");
		console.debug("Got botcache: ", response.data);
		var results = response.data.result;
		for(var i in results)
		{
			var user = results[i];
			if(user)
			{
				var sn = user.user.screen_name;
				var id = user.user.id;
				var score = 0;
				if(user.user.lang == 'en' || user.user.lang == 'en-gb')
				{	
					score = user.scores.english;
				}
				else
				{
					score = user.scores.universal;
				}
				
				botscores[id] = {score: score, old: !user.fresh, time: new Date(user.timestamp), user_id: user.user.id, screen_name: sn };
				updateNodeColor(id, score);
			}
		}
...

[marcmccarty]

Disregard previous comment. user.scores exists, whereas user.user.lang does not. To fix this, we created a global lang variable for Hoaxy (var defaultHoaxyLang = "en";) and set the vue variable lang to equal it.

Then, we created a watch function in Vue that switches the lang back to this value if it's been changed by selecting something on the Twitter search:

"searchBy": function()
        {
          if(this.searchBy == "Hoaxy")
          {
            this.lang = defaultHoaxyLang;
          }
        }

graph.js has this:

returnObj.setLang = function(passedLang){lang = passedLang};

And that setLang() is called multiple times in vue-app.js to ensure that the default language set there is passed to the graph when it's time to evaluate user languages and bot scores.

The only thing left to do is to bring these changes to the non-branded master branch, since these were made on the deployment, Hoaxy-branded branch.