Improve the use of Market API

[diesalbla]

There are several aspects of the use of the Android Market API from the backend server application that have caused errors during beta-testing that should be replaced

• Too Many Requests. Some of the endpoints of the server, such as looking for app recommendations or reading app details, issue a lot of HTTP requests to the Android Market API. The API sometimes replies with a status code 429 Too Many Requests. The server does not stop issuing requests when receiving this status. This must be changed. Upon noticing such a response to one call, the server should stop issuing additional requests.
• Parallelization. In this line of code, the call to traverse executes the issues sequentially, not in parallel. This should be fixed, perhaps by using the Task.gatherUnordered method.

This issue is related to issue #1097, for which a solution is delivered in this PR.

[diesalbla]

@javipacheco Since the goal of the ticket is to improve the method for fetching app details, the goal QA should check 1) that the information is correctly fetched, and 2) that the latency times are effectively reduced.

Setup

Use the setup.sql file in the /assets/postman directory. For the QA, you will need an AndroidId and a Google Play Token, which are known to be accepted by the Android market API. For the androidId to be authorised by the backend app, you need to associate it to the ann user in the setup, which you can do inside a `psql console with the following command:

 insert into installations(`userId`, `deviceToken`, `androidId` ) values 
  ((select id from users where sessiontoken='anntok'), '', '<ANDROID_ID>' ); 

Before the QA, you will also need to clean the local Redis Server.

QA Testing

SInce the goal is to compare same functionality before and after the changes, we will use a same call to the /applications/details endpoint. The call is already stored in the Postman collection (updated in this branch), and the details are in the accompanying environment. You will only need to put your Android Id and the Google Play Token in the Headers.

To make the call more interesting than the default one, we use a longer list of packages, which we give in this attached file: resolvable.txt. This file contains 100 packages, all of which are known to be resolvable, i.e., to refer existing Android applications.

To see the improvements, you should compare the results obtained by running the requests in the master branch and in the PR branch. To measure the number of resolved packages, you can use the following command:

echo "keys *:Resolved" | redis-cli | wc -l

Since the Google Play token may have some quota exhausted, you may want to allow for some time between them. Also, remember to clear the cache before each experiment.

[javipacheco]

Great job!

I have tried 100 new packages using this branch and the time was 19 seconds

In master the same 100 packages, the time was 29 seconds

LGTM!

[diesalbla]

This ticket's work was incorporated during this past week to the master branch.