asu-cactus/Data-Integration-Demo

Data Integration for Fast-evolving Data

Data Integration Demo

Data Integration for Data with Schema Changes

1. Attribute Renaming
2. Cell Value Reformatting
3. Key Expansion
4. Key Order Changing

Table Cell Position Prediction Using Sequence/Language Model

1. Column (attribute) prediction
2. Key index prediction
3. Aggregation mode prediction (only apply to covid-19 dataset)

Dataset

Covid-19 scenario

• Covid-19 dataset https://github.com/CSSEGISandData/COVID-19

• Google Mobility Report dataset https://www.google.com/covid19/mobility/

Machine log scenario

• Linux log data
• macOS log data
• Android log data

Model

• Pretrained Word Embedding (fastText) + Bi-LSTM
• Pretrained Word Tokenizer (BERT Tokenizer) + Encoder-only Transfomer

Usage

Dataset:

Github Repository

Google Drive

Download dataset:

import gdown
gdown.download('https://drive.google.com/drive/folders/19oLAKktjI0uk8v4lcdBTnRBTyqN-tGeR', output=None, quiet=False)

Run with Google Colab or local Jupyter Notebook.