A light weight ETL engine and smart transformation framework
Build:
From root directory, run: maven install
Run the demo:
- Modify distribution/target/SmartETL-0.0.1-incredible/examples/samplejob.xml to have correct path to the sample input files and output location.
- Go to distribution/target directory, from command line, run: bin/run.bat
Debug:
- Import the projects (libBase, libFormula, smartETL) into Eclipse
- Run configuration
Main class: org.f3tools.incredible.smartETL.JobRunner Argument: path to the samplejob.xml, for example: C:\Work\workspace\incredible\examples\samplejob.xml
Nice features:
- You can skip predefined amount of top and bottom lines when reading a csv file
- You can use lookup as function in formulas
- You can setup a filter for input files and smart trans step
- Smart trans step allows you define mapping rules using a DSL (domain specific language)
- You can filter out a row in formula
- All kind of functions such as string manipulation, date time manipulation ... are supported in the DSL
- You can define variables in smart trans step. you can use both input data, variables, and output data define output mapping rules
2015/04/16
Fixed a bug now variables are case insensitive
2015/3/30
- Added if, then, else to DSL
- Renamed function if to iif
- Support And, Or, Not logical operator
- Fixed a bug in PrefixTerm and PostfixTerm which didn't have initialize methods
- Removed square bracket for variables
- Changed function delimiter from ; to ,
2015/3/21
- Formula tag is supported now
- Two new tools are created. GenJobDef can generate job definition xml from an excel spreasheet. GrammarCheck can find out grammar errors in Excel mapping formulas.