This is a console app that reads IIS Log files and imports them to a SQL Table. If the table does not exist it creates a new once calld Logs
The app.config file has settings for:
ApplicationName: This is the name of application the logs belong to and is recorded in the databaseLogDirectory: This is the location of the IIS Logs- Connectionstring
Logs: this is the database that the logs will be imported to
If there is a problem parsing a log file (e.g. it contains a field that is not configured) a record will recorded in the database with the CanParse flag set to false.
If there are any errors during processing they are recorded (along with the date and time of the log) in log.txt created in the root directory.
The Console window logs as it begins to parse each file, however, the application will stop logging for a long time, during this phase it is catching up and recording the logs in the database. Once complete a message will ask you to "press any key to close".
The most likely changes to this are for adding additional fields. To do this you need to make the following changes:
- Add the new field(s) as properties to
LogFileLine.cs - Near the top of
LogFileParser.csthere is a dictionary calledFieldMapping. This contains an action for each field that populates the correct property inLogFileLine. You must add a dictionary item with corresponding Action to for each field. - Add the new field(s) to the
TABLE_CREATE_SQLandINSERT_LOG_SQLinRepository.cs
To improve performance the reading of the log files and storing of the data is multithreaded. This has been acheived using the Microsoft TPL Dataflow Library.
There are 3 blocks:
- Read the file paths
- Parse the files into
LogFileLineobjects - Save the
LogFileLineobjecs to database
As step 3 is the slowest step it has been given the highest degree of Parallelism