second-state/smart-contract-search-engine

Add a “quality” field

juntao opened this issue · 8 comments

juntao commented

We should add a “quality” field for each contract.

  • It starts from 50 when the contract is harvested.
  • If the contract can be confirmed to be legit by the admin, it increases to 100.
  • If contract is confirmed by the admin to be scam / spam / test, it drops to 0.

For contracts that do not have admin confirmation, a quality score maybe computed automatically.

The client can filter search results based on the quality score.

tpmccallum commented

The indexer now adds a field called quality to the common index.

                "_index": "testnet",
                "_type": "_doc",
                "_id": "0xc4354BB3D3dD8B3d217e43273f6FaF45aA2ECB0B",
                "_score": 1,
                "_source": {
                    "TxHash": "0x4cf8c9c90c331e309db569ac0b4f78e8092169c10d35ca831c95866b8ef462fc",
                    "abiSha3": "0x6718b332159313dce48c26e0e51c512040a3de930017555a03ba540c5b80235c",
                    "blockNumber": 1832990,
                    "contractAddress": "0xc4354BB3D3dD8B3d217e43273f6FaF45aA2ECB0B",
                    "functionDataId": "0xf91368c25a21a0cfd297e5d2ff45f4e7cc29f49e5ccb447a563953aba3ef1867",
                    "functionData": {
                        "info": {
                            "0": 1,
                            "1": "test",
                            "2": "##### Description\n\nno prizes\n\n",
                            "3": "https://res.cloudinary.com/dgvnn4efo/image/upload/v1559269909/q28pgiws7za9n7xm2vbr.png",
                            "4": 1,
                            "5": 1559270400,
                            "6": "en-US",
                            "7": "",
                            "8": ""
                        },
                        "owner": "0x9dcD4D8480CcE0efC67ED2a2faCE9C503a0Fb4A5"
                    },
                    "requiresUpdating": "yes",
                    "bytecodeSha3": "0x672ab6f7bef2b8f8c34b0dd7d37bef9fcbf0567899ed182278aeee91bb4b7bd5",
                    "abiSha3BytecodeSha3": "0xa1c025708a54ed04595d075658a563e454ac4595eff966113b81447dce3c4340",
                    "quality": "50"
                }
            }
tpmccallum commented

This task can also be performed on an already existing index using the following Python script.
https://github.com/second-state/smart-contract-search-engine/blob/master/python/utilities/batch_update_single_es_field.py
The above script updates all of the items in the index by supplying them with a new field quality which has a default value of 50

juntao commented
tpmccallum commented

Yes, and because the server hosting the search engine has the appropriate credentials (whitelisted by ES/AWS) we can provide this "increase_quality", "decrease_quality" functionality to an Admin via the web page. Here is an example.
Python

# INCREASE Quality Function
@app.route("/api/es_increase_quality", methods=['GET', 'POST'])
def es_increase_quality():
    jsonRequestData = json.loads(request.data)
    itemId = jsonRequestData["contractAddress"]
    doc = {}
    outerData = {}
    outerData["quality"] = "100"
    doc["doc"] = outerData
    theResponse = es.update(index=commonIndex, id=itemId, body=json.dumps(doc))
    return jsonify(theResponse)

# DECREASE Quality Function
@app.route("/api/es_decrease_quality", methods=['GET', 'POST'])
def es_decrease_quality():
    jsonRequestData = json.loads(request.data)
    itemId = jsonRequestData["contractAddress"]
    doc = {}
    outerData = {}
    outerData["quality"] = "0"
    doc["doc"] = outerData
    theResponse = es.update(index=commonIndex, id=itemId, body=json.dumps(doc))
    return jsonify(theResponse)

# RESET Quality Function
@app.route("/api/es_reset_quality", methods=['GET', 'POST'])
def es_reset_quality():
    jsonRequestData = json.loads(request.data)
    itemId = jsonRequestData["contractAddress"]
    doc = {}
    outerData = {}
    outerData["quality"] = "50"
    doc["doc"] = outerData
    theResponse = es.update(index=commonIndex, id=itemId, body=json.dumps(doc))
    return jsonify(theResponse)

Javascript

//INCREASE call from HTML/JS using a single contract address
_data = {"contractAddress": "0xc4354BB3D3dD8B3d217e43273f6FaF45aA2ECB0B"}
var _dataString = JSON.stringify(_data);
r = $.ajax({
    url: "https://cmt-testnet.search.secondstate.io/api/es_increase_quality",
    type: "POST",
    data: _dataString,
    dataType: "json",
    contentType: "application/json",
    success: function(response) {
        console.log(response);
    },
    error: function(xhr) {
        console.log("Set quality failed");
    }
});

// DECREASE call from HTML/JS using a single contract address
_data = {"contractAddress": "0xc4354BB3D3dD8B3d217e43273f6FaF45aA2ECB0B"}
var _dataString = JSON.stringify(_data);
r = $.ajax({
    url: "https://cmt-testnet.search.secondstate.io/api/es_decrease_quality",
    type: "POST",
    data: _dataString,
    dataType: "json",
    contentType: "application/json",
    success: function(response) {
        console.log(response);
    },
    error: function(xhr) {
        console.log("Set quality failed");
    }
});

//RESET call from HTML/JS using a single contract address
_data = {"contractAddress": "0xc4354BB3D3dD8B3d217e43273f6FaF45aA2ECB0B"}
var _dataString = JSON.stringify(_data);
r = $.ajax({
    url: "https://cmt-testnet.search.secondstate.io/api/es_reset_quality",
    type: "POST",
    data: _dataString,
    dataType: "json",
    contentType: "application/json",
    success: function(response) {
        console.log(response);
    },
    error: function(xhr) {
        console.log("Set quality failed");
    }
});

We can decide on an authentication method for the search engine Admin[s] and keep the Admin functions like these (ones listed above) restricted. I feel that it would be nice to just be able to click a button to adjust the quality (the contractAddress could even be automatically passed via the HTML/JS i.e. moderating a list of contractAddresses via a results page in an admin Dashboard)

tpmccallum commented

You can test these out on the testnet via the Javascript if you like. It will be good to discuss search engine users (with elevated privileges). In some cases we could stipulate ownership by interacting with Venus but I guess in the case of quality this may be an independent evaluation of the genuine nature of a giveaway (spam vs genuine)

juntao commented
tpmccallum commented

Fantastic, well in that case it would actually improve simplicity of the code. We would just (in the HTML/JS) pass in an extra value (the quality score) along with the address.
I will add in some validation in the Python code so that the score can not be greater than 100 or less than 0. This can also be double checked in the frontend if/as required.
Python

@app.route("/api/es_update_quality", methods=['GET', 'POST'])
def es_update_quality():
    jsonRequestData = json.loads(request.data)
    itemId = jsonRequestData["contractAddress"]
    qualityScore = jsonRequestData["qualityScore"]
    if int(qualityScore) >= 0 and int(qualityScore) <= 100:
        doc = {}
        outerData = {}
        outerData["quality"] = qualityScore
        doc["doc"] = outerData
        theResponse = es.update(index=commonIndex, id=itemId, body=json.dumps(doc))
        return jsonify(theResponse)

Javascript

//INCREASE call from HTML/JS using a single contract address
_data = {
	"contractAddress": "0xc4354BB3D3dD8B3d217e43273f6FaF45aA2ECB0B",
	"qualityScore": "22"
}
var _dataString = JSON.stringify(_data);
r = $.ajax({
    url: "https://cmt-testnet.search.secondstate.io/api/es_update_quality",
    type: "POST",
    data: _dataString,
    dataType: "json",
    contentType: "application/json",
    success: function(response) {
        console.log(response);
    },
    error: function(xhr) {
        console.log("Set quality failed");
    }
});
tpmccallum commented

This quality field has now been added to the index of the "parked" MainNet instance in readiness for the MainNet's switchover to the new code i.e. data will match code and functionality.