Index per day

[jnus]

I'm migrating a large mssql database with logs, using your sample to move the data. However I would like to create an index per day, Logstash-[date] , like frameworks such as Serilog does. Is this possible with ElasticsearchCRUD?

Thanks
/Jasper

[damienbod]

Hi Jasper
Yes, you could do it like this:

    public class LogToElasticsearchProvider
    {
        private readonly string _connectionString = "http://localhost:9200/";

        public LogToElasticsearchProvider(string connectionString)
        {
            _connectionString = connectionString;
        }

        /// <summary>
        /// Creates a new index per day
        /// If you requiire performance, set the replications etc. and reactivate when finished
        /// =>  new IndexSettings{RefreshInterval="-1", NumberOfReplicas = 0}
        /// You could also use routing per category etc if you have extreme data amounts...
        /// </summary>
        /// <param name="dayDate">Day : 2015.02.23</param>
        /// <param name="logEntries">data from your primary source</param>
        public void CreateLogIndexPerDay(string dayDate, IEnumerable<LogEntry> logEntries)
        {
            IElasticsearchMappingResolver elasticsearchMappingResolver = new ElasticsearchMappingResolver();
            elasticsearchMappingResolver.AddElasticSearchMappingForEntityType(typeof (LogEntry),
                MappingUtils.GetElasticsearchMapping("log_" + dayDate));

            using (var elasticsearchContext = new ElasticsearchContext(_connectionString, elasticsearchMappingResolver))
            {
                // Creates a index log_dayDate, type logentry
                elasticsearchContext.IndexCreate<LogEntry>();

                foreach (var logEntry in logEntries)
                {
                    elasticsearchContext.AddUpdateDocument(logEntry, logEntry.Id);
                }

                // Adds the logs
                elasticsearchContext.SaveChanges();
            }
        }

        /// <summary>
        /// you should delete the alias for old indices
        /// </summary>
        /// <param name="dayDate"></param>
        public void CreateAliasForDayLogIndex(string dayDate)
        {
            IElasticsearchMappingResolver elasticsearchMappingResolver = new ElasticsearchMappingResolver();
            elasticsearchMappingResolver.AddElasticSearchMappingForEntityType(typeof(LogEntry),
                MappingUtils.GetElasticsearchMapping("log_" + dayDate));

            using (var elasticsearchContext = new ElasticsearchContext(_connectionString, elasticsearchMappingResolver))
            {
                elasticsearchContext.AliasCreateForIndex("logentrys", "log_" + dayDate);
            }
        }
    }

I could create an example on gitHub if it would help. If anything is missing let me know, I can extend the library as well if you require extra functionality.

Hope this helps,
Greetings Damien

[jnus]

Hi Daniem

This is excellent and works perfectly!

Thanks.
/Jasper