Fast and secure webservice written in C with MongoDB NoSQL database
sudo apt-get install mongodb-clients mongodb-server colorgcc gcc
sudo ln -sf /usr/bin/colorgcc /etc/alternatives/cc
cd /tmp
wget https://github.com/algernon/libmongo-client/archive/master.zip
unzip master.zip
cd libmongo-client-master
autoreconf -i
./configure
make
sudo make (install in /usr/local)
Clients to test are Python scripts
./plurals.py encrypts and Base64 encodes a payload and sends it in a POST request to webservice plurals. This call will be made again after x hours when reply is not successful.
./score.py encrypts and Base64 encodes user_id and and sends it with location in a POST request to webservice score. This call may be made only once every 24 hours, client will govern this.
The server consists of several parts.
Part 1: Receiving
A thread in C code is running in TODO webserver. All it does is:
- process only POSTs with parameter called d with a value
- that value of d is URL-decoded and stored in field encrypted_encoded in MongoDB database blahblah.plurals
- the value is unique in the database, duplicates are ignored Example call is http://hellebaard.nl:8081/blahblah/plurals?d=Lj55GxX%2Beq3KcrOszba793CUA7TLIkx5xx5RqgPWD0w%3D
Part 2: Decoding, decrypting and calculating score
A cron job runs daily a Python script that:
- decodes and decrypts the value of field encrypted_encoded in MongoDB database blahblah.plurals
- stores comma separated values in fields user_id, word_id, synset_id, word_type and data.
- updates user statistics in MongoDB database blahblah.users See data model below for examples.
Part 3: Providing scores
A thread in C code is running in the same webserver. All it does is:
-
receive user chosen location in parameter called l, when this parameter is present
-
this parameter is in form of string containing ISO 3166-2 code and is stored
-
scores for user are being returned Example call is http://hellebaard.nl:8081/blahblah/score?u=sikfr42CDJ8DpSThurbzsw%3D%3D&l=NL-UT and the reply is JSON with score.
{ "plural_submits": 100, "plural_position_overall": 20, "plural_position_country": 10, "plural_position_region": 1, "for_location": "NL-UT" }
When another location is submitted then is known in database, that location will be used next time when score is retrieved. Information message in client when location is changed that it will take 24 hours later before new score is reported for the new location.
MongoDB database has by default a unique ObjectID or _id from which also the insert time can be derived. The following extra fields are defined:
Namespace blahblah.plurals
"Field name"
"Mongo BSON type in C"
"Notes or examples"
_id (ObjectId)
BSON_TYPE_OID
this contains timestamp from moment of insert
encoded_encrypted_data
BSON_TYPE_STRING
e.g. Lj55GxX+eq3KcrOszba793CUA7TLIkx5xx5RqgPWD0w=
user_id
BSON_TYPE_STRING
e.g. NULL or ****
word_id
BSON_TYPE_INT64
e.g. NULL or 41012531
synset_id
BSON_TYPE_INT64
e.g. NULL or 37581
word_type
BSON_TYPE_INT32
e.g. NULL or 1
data
BSON_TYPE_STRING
e.g. NULL or smörgåsbord
Namespace blahblah.users
"Field name"
"Mongo BSON type in C"
"Notes or examples"
_id (ObjectId)
BSON_TYPE_OID
this contains timestamp from moment of insert
encrypted_user_id
BSON_TYPE_STRING
e.g. sikfr42CDJ8DpSThurbzsw%3D%3D
user_id
BSON_TYPE_STRING
e.g. NULL or ****
location
BSON_TYPE_STRING
e.g. NULL or NL-UT
plurals_submitted
BSON_TYPE_INT32
e.g. NULL or 100
plurals_position_overall
BSON_TYPE_INT32
e.g. NULL or 20
plurals_position_country
BSON_TYPE_INT32
e.g. NULL or 10
plurals_position_region
BSON_TYPE_INT32
e.g. NULL or 1
Database can be viewed by running mongodump. Contents can be viewed by running mongoexport -d blahblah -c plurals or ./dump-db.sh. Real-time statistics can be monitored by running mongostats.