Nett.Coma: Merging Inlinetables overwriting?

Question

Nett.Coma: Merging Inlinetables overwriting?

donvreug opened this issue 5 years ago · 20 comments

Hi,

When I try and merge inline tables that have different rows, the second appears to overwrite the first in the config. Is this the expected behaviour? For example:

config = Config.CreateAs()
.MappedToType(() => new ConfigSettings())
.StoredAs(store =>
store.File(defaultUserSettings).AccessedBySource("defaultUser", out defaultUserSource).MergeWith(
store.File(userSettings).AccessedBySource("user", out userSource)))))
.Initialize();

defaultUserSettings toml file:

[User.Variables]
VariablesList = ["fe","sio2"]

[User.Variables.VariablesProperties]
fe = { Check = true, Name = "fe", Caption = "Fe", Unit = "%", Format = "#0.00" }
sio2 = { Check = true, Name = "sio2", Caption = "SiO2", Unit = "%", Format = "#0.00" }

userSettings toml file:

[User.Variables]
VariablesList = ["al2o3"]

[User.Variables.VariablesProperties]
al2o3 = { Check = true, Name = "al2o3", Caption = "Al2O3", Unit = "%", Format = "#0.00" }

And I read the config using...

var variables = config.Get(s => s.User.Variables.VariablesList);
var variableProperties = config.Get(s => s.User.Variables.VariablesProperties);

The result in the config is the table from the userSettings toml file.

I expected the config to hold all three rows. Perhaps I am doing something wrong?

Cheers.

Answer 1 · 2019-10-23T09:10:06.000Z

On latest master I'm not able to reproduce the issue. That is the test case I'm running successfully:

        public class FooCfg
        {
            public UserCfg User { get; set; } = new UserCfg();

            public class UserCfg
            {
                public VarCfg Variables { get; set; } = new VarCfg();
            }

            public class VarCfg
            {
                public Dictionary<string, object> VariablesProperties { get; set; } = new Dictionary<string, object>();
            }
        }


        [Fact]
        public void Foox()
        {
            using var machine = TestFileName.Create("machine", ".toml");
            using var user = TestFileName.Create("user", ".toml");

            // Arrange
            const string machineText = @"
[User.Variables]
VariablesList = [1, 2]
[User.Variables.VariablesProperties]
fe = { Check = true }
sio2 = { Check = true }";
            const string userText = @"
[User.Variables]
VariablesList = [3]
[User.Variables.VariablesProperties]
al2o3 = { Check = false }";
            File.WriteAllText(machine, machineText);
            File.WriteAllText(user, userText);

            // Act
            var cfg = Config.CreateAs()
                .MappedToType(() => new FooCfg())
                .StoredAs(store => store.File(machine)
                    .MergeWith(store.File(user)))
                .Initialize();
            var items = cfg.Get(c => c.User.Variables.VariablesProperties);

            // Assert
            items.Count.Should().Be(3);
        }

Answer 2 · 2019-10-23T12:30:06.000Z

Thanks for your help. I have it working now.

One question: When merging files it appears we can add items, but can we also remove items if they appear in only one file? Say you want to remove redundant items from the config.

Thanks & Regards Don

Answer 3 · 2019-10-23T12:43:10.000Z

To clarify, if you are merging and older toml file with a new one and if there are items in the old one that are not in the new one can they be removed during the merge?

Answer 4 · 2019-10-24T11:30:05.000Z

No this is not possible.

Answer 5 · 2019-10-25T03:11:58.000Z

Ok thanks.

In the Nett.Coma context, If you have defined a TomlTable in your class, such as ...

public TomlTable Settings { get; set; }

and in the user_settings.toml file I have

[Settings]
first = "first"
second = "second"

I can get a value of the key "first" by

var conType = ThisAddIn._config.Get(s => s.Settings).Get("first");

But how do I update the key's value to"fourth" so that my toml file becomes

[Settings]
first = "fourth"
second = "second"

The reason I don't have a specific mapping for the row keys is I may have up to 100 keys in the table.

Thanks & regards

Answer 6 · 2019-10-30T08:02:05.000Z

Here is an example of updating a key:

using var machine = TestFileName.Create("machine", ".toml");
using var user = TestFileName.Create("user", ".toml");

// Arrange
const string machineText = @"x = 1";
const string userText = @"x = 2";
File.WriteAllText(machine, machineText);
File.WriteAllText(user, userText);

// Act
var cfg = Config.CreateAs()
    .StoredAs(store => store.File(machine)
        .MergeWith(store.File(user)))
    .Initialize();

cfg.Set(tbl => tbl.Update("x", 4));

// Assert
File.ReadAllText(user).Trim().Should().Be("x = 4");

Answer 7 · 2019-10-30T08:53:25.000Z

I can't seem to get the syntax correct...

My mapping class is called Configuration and within it I have defined a TomlTable as

public TomlTable Table { get; set; }

the toml file holds

[Table]
a = 1
b = 2
c = 3

And I am trying to update "a" to 4 using

ThisAddIn._config.Set(tbl => tbl.Update("a", 4));

Hmm. I get an error stating 'Configuration' does not contain a definition for 'Update' ...

Any idea where I am going wrong?

Answer 8 · 2019-10-30T09:31:45.000Z

You should not mix custom object graphs and TomlTable. Either your graph consists of TOML objects or of custom objects. A mix of those I never anticipated during the implementation.
Natively this is a not really good supported use case at the moment. I'll may do some improvements regarding that in the future. The syntax I would like to support should look like this:

cfg.Set(c => c.Table["a"], 6);

But currently this will not work, as the selector cannot handle the '["a"]' part and there would by a type mismatch.

Currently you can only set the whole table with some form of workaround where you need to also specify an explicit save target, because a table source cannot be uniquely identified as it can be composed from multiple sources.

        public class FooyCfg
        {
            public Dictionary<string, object> Tbl { get; set; } = new Dictionary<string, object>();
        }
...

    using var machine = TestFileName.Create("machine", ".toml");
    using var user = TestFileName.Create("user", ".toml");

    // Arrange
    const string machineText = @"[Tbl]
x = 1";
    const string userText = @"[Tbl]
x = 2";
    File.WriteAllText(machine, machineText);
    File.WriteAllText(user, userText);

    // Act
    IConfigSource src = null;
    var cfg = Config.CreateAs()
        .MappedToType(() => new FooyCfg())
        .StoredAs(store => store.File(machine)
            .MergeWith(store.File(user).AccessedBySource("user", out src)))
        .Initialize();

    var tbl = cfg.Get(c => c.Tbl);
    tbl["x"] = 4;
    cfg.Set(c => c.Tbl, tbl, src);

    // Assert
    File.ReadAllText(user).ShouldBeSemanticallyEquivalentTo(@"[Tbl]
x = 4");

Answer 9 · 2019-10-30T09:43:20.000Z

Thanks. That is the approach I was taking, but I had hoped (searched your code and examples for) a simpler approach. Now that you have confirmed this is currently the approach to take I will implement it. Thanks for you help,

For your consideration it would also be useful to have the option of specifying the IConfigSource in the Set statement as well, for example cfg.Set(c => c.Table["a"], 6, src).

In your experience do most users of your library use Nett.Coma to create the configuration or do they use Nett directly? Which would you recommend?

Regards Don

Answer 10 · 2019-10-30T10:04:14.000Z

I think most people use Nett directly (but I have no hard data for this assumption), as most people do not have the need to merge different configs into one.

Answer 11 · 2019-10-30T10:07:46.000Z

Ok. Since merging different toml files into one is important for my setup I'll stick with Coma.

Thanks for your help and a great library. Regards Don

Answer 12 · 2019-11-05T08:56:03.000Z

I'm a little stuck so I hope you might be able to help me out.

In my configuration class I have defined a Table as:

public Dictionary<string, object> Table { get; set; } = new Dictionary<string, object>();

I took this approach so it could hold different types. If that is not the best way please advise.

In the toml file I have

[Table]
a = 1
d = "D"
e = ["b","c"]

I can handle the numeric and string types but I am having difficulty with the array type. I need to be able to convert it to a string[].

var tbl = ThisAddIn._config.Get(s => s.Table);
var e = tbl["e"];

where 'e' appears to be an object{object[]} where each item in the array is type string.
I have tried a number of methods to cast/convert it to a string[] including:

var e = (string[])tbl["e"];

var e = tbl["e"] as string[];

var e= tbl["e"];
string[] result = e.Where(x => x != null)
.Select(x => x.ToString())
.ToArray();

and a few others. None have worked. Do you know how to convert 'e' to a string[]?

or do I have to restrict Table to a Dictionary<string, string[]> type?

or strongly type 'e' in my configuration class?

Thanks & Regards Don

Answer 13 · 2019-11-05T11:37:41.000Z

I found an answer using

var result = (e as IEnumerable).Cast()
.Where(x => x != null)
.Select(x => x.ToString())
.ToArray();

Do you know of another way?

Answer 14 · 2019-11-06T11:09:57.000Z

The easiest I can currently can come up with is

var r = ((object[])e).OfType<string>().ToArray();

Also here I hope in the near future something like this will work

var r = cfg.Get<string[]>(c => c.Table["e"]);

Answer 15 · 2019-11-06T11:42:13.000Z

Thanks very much. That is better and the last one would be great.

Answer 16 · 2019-12-03T09:00:21.000Z

I have a datagridview which is the ui control to hold an inline table. I want to be able to add and remove rows. I am finding the following behaviour when removing rows:

With and 'app_settings.toml' and 'user_settings.toml' files where only the 'user' file contains the inline table, i.e.

[User.Variables.VariablesProperties]
fe = { Check = true, Name = "fe", Caption = "Fe", Unit = "%", Format = "#0.00" }
sio2 = { Check = true, Name = "sio2", Caption = "SiO2", Unit = "%", Format = "#0.00" }
al2o3 = { Check = true, Name = "al2o3", Caption = "Al2O3", Unit = "%", Format = "#0.00" }

and merged as follows...

        string appSettings = "C:\\app_settings.toml";
        string userSettings = "C:\\user_settings.toml";
        IConfigSource appSource = null;
        IConfigSource userSource = null;
        Config<Configuration> config = null;
        // merge into config object
        config = Config.CreateAs()
            .MappedToType(() => new Configuration())
            .StoredAs(store =>
                store.File(appSettings).AccessedBySource("app", out appSource).MergeWith(
                store.File(userSettings).AccessedBySource("user", out userSource)))
            .Initialize();

        _app = appSource;
        _user = userSource;
        return config;

It works ok.

However when I have and two additional files, i.e. 'C:\default\app_settings.toml' and 'C:\default\user_settings.toml' and merged as follows:

        config = Config.CreateAs()
            .MappedToType(() => new Configuration())
            .StoredAs(store =>
                store.File(defaultAppSettings).AccessedBySource("defaultApp", out defaultAppSource).MergeWith(
                store.File(defaultUserSettings).AccessedBySource("defaultUser", out defaultUserSource).MergeWith(
                    store.File(appSettings).AccessedBySource("app", out appSource).MergeWith(
                    store.File(userSettings).AccessedBySource("user", out userSource)))))
            .Initialize();

        _defaultApp = defaultAppSource;
        _defaultUser = defaultUserSource;
        _app = appSource;
        _user = userSource;
        return config;

I save the config to 'C:\user_settings.toml' after deletion using:

        ThisAddIn._config.Set(s => s.User.Variables.VariablesProperties, variablesProperties, userSource);

After I delete a row I check it is deleted ok by viewing the file, i.e.

[User.Variables.VariablesProperties]
fe = { Check = true, Name = "fe", Caption = "Fe", Unit = "%", Format = "#0.00" }
sio2 = { Check = true, Name = "sio2", Caption = "SiO2", Unit = "%", Format = "#0.00" }

I then load the config into the datagridview using:

        var variablesProperties = ThisAddIn._config.Get(s => s.User.Variables.VariablesProperties);

But when I load the datagridview again all three rows are loaded. So the deleted row is still held in the config somewhere.

Am I doing something that is not supported or missing a step?

Regards Don

Answer 17 · 2019-12-09T05:56:24.000Z

As a workaround I am reloading the config using the modified toml files.

Answer 18 · 2020-01-10T11:12:18.000Z

Just a small update on the issue. Im still working on it. But with the little time I have available ATM - as other projects of mine need more attention - progress is very very slow.

As soon as #87 is also fixed - hopefully a few weeks max; not that easy as there are some conceptual issues with that - I will release a new Nett version.

Answer 19 · 2020-01-11T03:09:37.000Z

No worries and thanks.

Answer 20 · 2020-03-02T15:17:02.000Z

Fixed in v0.15.0

If more features etc are needed please open a new issue.