Discrepancy between various data sources [IR001]

[mimani68]

Intro

Most of financial activity due the diversity of diversity in data sources and institutional operations come with discrepancy in output result. The process of unifying and checking the result is not only exhausted but also has many drawbacks like human error, inaccuracies and possible duplication.

Demands

1. Continues process for data gathering
2. Determining some functionality for cleaning and normalizing data
3. Show latest status of each transaction

Inputs

1. Excel file (ftp server)
2. Database (query)
3. Online modification
4. Policy file (S3 bucket)
   4.1) bond time of operation
   4.2) database cred

Pipelines

1. Dump polices
2. Clear customer confidential data
3. Checking all inputs and providing result
4. Store updated record into database
5. Send and email consist of list of updated record

Outputs

1. Adding new record in database