delegateas/Daxif

How does DataExport work?

RemyDuijkeren opened this issue · 6 comments

I'm trying to use DG.Daxif.Data.Export() method but so far without any luck. I'm calling your lib from C# which I got working for the other methods (be looking at the errors and reading thru the F# code), but this method leaves me blank and wondering how it works.

I don't get any errors, no logging what it is doing or missing, even when LogLevel = Verbose. Is is very fast done, so it looks it is doing nothing. It's probably something simple I'm missing, but I can't figure it out.

Reproduce
I use the following C# code where SourceEnvironment is created somewhere else (no problem with the connection):

DG.Daxif.Data.Export(
    env: SourceEnvironment,
    entityNames: new string[] { "account" },
    pathToOutputFile: @"C:\Data.csv",
    deltaFromDate: null,
    logLevel: DG.Daxif.LogLevel.Verbose);

Expected behavior
I expect some data file to generated, like. csv or something else.

There are some (to my knowledge) broken API's in Daxif that existed in older versions, but were incorrectly implemented. I believe data and solution diff/merge do not work.
Perhaps some of the other maintainers can comment if they have something to add

The original inventor of Daxif had the idea that it should be a toolbox for a lot of stuff for working with CRM. One of the were data export to handle migration or data copying between environments.

I believe we got it to a somewhat working state. But we haven't looked at it in years as we found other tools that better handles the mentioned use cases.

@TomMalow thanks for the info. What kind of tools you (or @magesoe) use voor data? I see a lot of self-made powershells around :-(.

For my it's just basic simple master-data that we need to copy between environments (think country-list, etc.).

Self made migration script are pretty normal as a lot of the standard entities often require special handling. I know @magesoe have previously worked on a series of F# script to handle extensive data migration. (i.e. migrate million of records). However, that was years ago and it is/was a F# script so not that user friendly.

A useful migration tool for such data could be the Data Transfer Tool in XrmToolBox https://github.com/bcolpaert/Colso.Xrm.DataTransporter. This seems to be a tool that has been used by a few people and good rating. However, if it is to be used as part of a DevOps setup, then you might need another tool.

Note:
I myself tries to avoid having meta-data (such as country-list) setup in an environment and migrated down stream. Instead aiming to create a script to setup these meta-data based on a data source (csv, json, etc) that is part of our repo. Then I can simply run that script for all my environments when needed and that customization is now defined as code. But that requires that you don't have people doing manual configuration.

Thanks for the answer! I'm indeed looking for something to use in a DevOps scenario, so I was hoping someone fond THE tool for it :-). Self made script is going to be it then.

For big migration scenarios I often end-up using SSIS.

This is sadly not THE tool for it. Hopefully you find a better solution.

However, I suggest that we test the Daxif data export module. Depending on the result, then either update the wiki if it works or simply remove it completely if it doesn't to avoid future problems or confusion.