Using EntityFramework 6.2 model store breaks filtering

[mcnallys]

• Version 1.4.10.2

Entity framework 6.2 beta introduces a new feature that caches the compiled models; when using this feature "OnModelCreating" is not called.

From: https://blogs.msdn.microsoft.com/dotnet/2017/05/23/announcing-ef-6-2-beta-1/

I was able to get this to work through some "hacks" to the code, but I am curious as to what you think a full solution should be.

I had to mark DynamicFilterDefinition as public, and [Serializable]. The predicate expression would need to be serialized through another mechanism- perhaps a string and using System.Linq.Dynamic. The attribute name also needed to be left as just "DynamicFilter" for the following portion to work.

We have a class that inherits from "DbConfiguration" and in the constructor we can call
SetMetadataAnnotationSerializer("DynamicFilter", () => new DynamicFilterSerializer());

See "Non-String Annotations" from https://entityframework.codeplex.com/wikipage?title=Code%20First%20Annotations

DynamicFilterSerializer is just a simple wrapper around a binary formatter and base64 string. This will make entityframework rebuild the DynamicFilterDefinition object when it recreates the compiled model from the store.

Finally this allows us to reinitialize the filters through another hack like so:

public OurContext(string sqlConnectionString) : base(sqlConnectionString)
{
var mb = new DbModelBuilder();
mb.ResetDynamicFilters();
mb.Filter(DELETED_EVENT_FILTER_NAME, (Type e) => e.IsRemoved, false);
mb.Filter(DELETED_TRAIT_FILTER_NAME, (Type t) => t.IsRemoved, false);
mb.Filter(DELETED_RESOURCE_FILTER_NAME, (Type r) => r.IsRemoved, false);
}

OnModelCreating must still exist for when it has to be compiled the first time.

I am willing to do the legwork on this and submit a pull request but am curious as to the approach you would like to see for this before I committed to some code.

[JonathanMagnan]

Hello @mcnallys ,

Thank you for letting me know.

I will try to check if I can find another solution on my side this weekend.

Best Regards,

Jonathan

[JonathanMagnan]

Hello @mcnallys ,

I must probably do something wrong. I'm not able to reproduce it and my OnModelCreating is invoked every time.

The compiled model is also in the output directory Z.Lab.Form_Request_CompiledModels+EntityContext.edmx so I guess I don't have everything wrong.

Do you think you could provide me a full example or edit the following one?

using System.Data.Entity;
using System.Data.Entity.Infrastructure;
using System.IO;
using System.Linq;
using System.Windows.Forms;
using EntityFramework.DynamicFilters;

namespace Z.Lab
{
    public partial class Form_Request_CompiledModels : Form
    {
        public enum EnumValue
        {
            None,
            A,
            B,
            C
        }

        public Form_Request_CompiledModels()
        {
            InitializeComponent();

            // CLEAR
            using (var ctx = new EntityContext())
            {
                ctx.EntitySimples.RemoveRange(ctx.EntitySimples);
                ctx.SaveChanges();
            }

            // SEED
            using (var ctx = new EntityContext())
            {
                ctx.EntitySimples.Add(new EntitySimple {ColumnString = "Z", ColumnEnum = EnumValue.None});
                ctx.EntitySimples.Add(new EntitySimple {ColumnString = "z", ColumnEnum = EnumValue.A});
                ctx.EntitySimples.Add(new EntitySimple {ColumnString = "a", ColumnEnum = EnumValue.None});
                ctx.SaveChanges();
            }

            // TEST
            using (var ctx = new EntityContext())
            {
                var list = ctx.EntitySimples.ToList();
            }
        }

        [DbConfigurationType(typeof(MyDbConfiguration))]
        public class EntityContext : DbContext
        {
            public class MyDbConfiguration : DbConfiguration
            {
                public MyDbConfiguration() : base()
                {
                    this.SetModelStore(new DefaultDbModelStore(Directory.GetCurrentDirectory()));
                }
            }

            public EntityContext() : base("CodeFirstEntities")
            {
              
            }

            public DbSet<EntitySimple> EntitySimples { get; set; }

            protected override void OnModelCreating(DbModelBuilder modelBuilder)
            {
                modelBuilder.Filter("WithEnum", (EntitySimple d) => d.ColumnEnum == EnumValue.None);

                modelBuilder.Types().Configure(x => x.ToTable(GetType().DeclaringType != null ? GetType().DeclaringType.FullName.Replace(".", "_") + "_" + x.ClrType.Name : ""));

                base.OnModelCreating(modelBuilder);
            }
        }

        public class EntitySimple
        {
            public int ID { get; set; }

            public int ColumnInt { get; set; }
            public string ColumnString { get; set; }

            public EnumValue ColumnEnum { get; set; }
        }
    }
}

Best Regards,

Jonathan

---

Help us to support this library: Donate

[mcnallys]

Thanks for getting back to me, it seems like you have it implemented correctly. The only difference I see is that we construct ours with a sql connection string. I will take your example code and try it out in the next few days.

[mcnallys]

I had time to look at your example, if you run the application once and let it create the file it will call onmodelcreating for that run. Run the application again and you will see the behavior of the model being created - where onmodelcreating does not get called.

[JonathanMagnan]

Hello @mcnallys ,

Thank you, I will give another try this weekend.

I did as you instructed and the onmodelcreating was still called every time. Perhaps I'm missing something.

Best Regards,

Jonathan

---

Help us to support this library: Donate

[mcnallys]

https://drive.google.com/open?id=1YYGqb2Do0oh8j6tip_AvLKhO4I7beSzE

Download that and place a break point in onmodelcreating,

it will hit a few times for the first run.

Close the application, debug again. The breakpoint should not hit.

[JonathanMagnan]

Hello @mcnallys ,

I probably made my test wrong the first time, I can easily reproduce the issue.

Unfortunately, I don't think this request could be made. We have looked a lot at it.

The main problem is EF DynamicFilters is adding some conventions to make the filter work. Since the model is stored already in a file, the OnModelCreating and all custom conventions are no longer required and called.

We tried some solutions but none of them seem to work.

Unfortunately, I believe this library will not be compatible with the DefaultDbModelStore ;(

---

Help us to support this library: Donate

[mcnallys]

I had basic support working, for simple one property soft delete cases, full on other linq statements were another matter. I cannot guarantee a timeline but I can try to get a PR put together that will solve some of it.

[JonathanMagnan]

Sure ;) It may give us some idea how to more easily support it by inspiring ourself from your solution

[AlanMacdonald]

Hi @mcnallys did you figure out a more permanent fix? We've hit this issue with caching a large context. Thanks

[mcnallys]

I have not had time to take a look at getting a cleaner implementation yet. It may be weeks or months before I am able to get back to it.

[JonathanMagnan]

Do you think you could provide me a war implementation of what you have done so far?
info@zzzprojects.com

It's fine if that's not very clean, at least it will give me a rough idea of how we could do it.

Best Regards,

Jonathan

---

Help us to support this library: Donate

[mcnallys]

Slapped together what i have as a war pull request, let me know what other information you need.

Thanks!

[meywd]

any update on this issue?

[mcnallys]

Hi meywd,

Unfortunately this is not a priority for my company at the moment. I can consult with taking my PR further; the last conversation I had with Jonathan was that implementing this were everyone could use it would be difficult.

[meywd]

@mcnallys Thank you for your effort, we have been using this library for 2 years, and suddenly, after adding a couple of classes to our model, EF decided its enough and we started getting timeouts on our APIs, implementing the model store fixed the performance but now all filters are ignored, the only solution I can think of is removing the library and rebuilding all the queries, which is going to a be a pain

[JonathanMagnan]

Hello @meywd ,

Do you think you could provide your model at info@zzzprojects.com

Next month, we are supposing to check if there is anything we can do about the initial load. Having more model that takes a long time might always help.

Depending on the investigation, we might re-visit this issue or try to simply make our own Model Store logic that doesn't EF DynamicFilters libraries.

[mcnallys]

@meywd We use multiple EF models in our solution, is that an option for you?

[meywd]

@JonathanMagnan, Thank you, unfortunately since its not a personal project I can't share it, Although it's not a huge model, around 80-90 entities.

@mcnallys All the entities are related, with many complex queries with multiple joins, I can't separate them without breaking the queries, it would need a redesign

[JonathanMagnan]

No problem,

We will try with the model that people already submitted us. For example, one of them take 4s before the ToList() on empty tables completed the execution. I guess if we fix this one, we will fix pretty much all others one

As said, we should start next month to look at it. So if you want, if we didn't provide any answer yet, just reply to this thread at the end of April to get a status.

[meywd]

Is there any way to add filters programmatically on demand, even as a hack?

[JonathanMagnan]

Hello @meywd ,

There is certainly a way, but none that I'm aware at this moment.

[jwcde]

We've also run into this issue. This was quite the gotcha! Until this was identified as the issue, it seemed like the filters were only working sometimes. As it turns out, it was only working when/if the persistent store was invalidated.