Snips-nlu country and datetime not picking up

[Moon-developer]

Description

I have an intent named "flight" as in the docs with clear utterances. The result of the parsed input returns empty with low probability rate even when the questioned asked is word for word in the utterance.

To Reproduce

I followed the tutorial on the doc site for snips-nlu, pip install snips-nlu and snips-nlu download en after settings up my tutorial and executing it, it complained that that snips/country is not found so I ran snips-nlu download-entity snips/country en. Linking kept failing so I followed the steps again using the admin privilages on command prompt(I am using windows 10). This wasn't a problem till today.

Expected behavior

if I type in what is the flight for tomorrow to botswana? I expected the following output:

{
  "input": "what is the flight for tomorrow to botswana?",
  "intent": {
    "intentName": "holidayQuestion",
    "probability": 0.6913064071842576
  },
  "slots": [
    {
      "range": {
        "start": 19,
        "end": 27
      },
      "rawValue": "tomorrow",
      "value": {
        "kind": "InstantTime",
        "value": "2019-11-22 00:00:00 +02:00",
        "grain": "Day",
        "precision": "Exact"
      },
      "entity": "snips/datetime",
      "slotName": "date"
    },
    {
      "range": {
        "start": 31,
        "end": 39
      },
      "rawValue": "botswana",
      "value": {
        "kind": "Custom",
        "value": "botswana"
      },
      "entity": "country",
      "slotName": "country"
    }
  ]
}

But got:

{
  "input": "what is the flight for tomorrow to botswana?",
  "intent": {
    "intentName": null,
    "probability": 0.46906033378594386
  },
  "slots": []
}

Environment:

• OS: Windows 10 64bit
• python version: 3.7.4
• snips-nlu version: 0.20.1

[Moon-developer]

UPDATE:
The engine does not find 'botswana' as a country unless I run a str.upper() on the sentence to parse. And changing my code from engine = SnipsNLUEngine(resources=load_resources('snips_nlu_en')) to engine = SnipsNLUEngine(config=CONFIG_EN) allows date questions to be found suchs as what is the weather like next week and it finds next week where as it wouldn't before. I am so confused as to why these small changes result in different outputs.

[adrienball]

Hi @Moondog360
The result that you got contains "intentName": null, which means that the NLU failed to recognized your holidayQuestion intent. In that case, it won't even try to extract slots.
So the first thing to understand is why does it fail to recognize your intent.
Here are possible explanations for this:

• You have another intent in your training dataset, which is semantically close to the holidayQuestion intent. This proximity confuses the NLU which is not able to differentiate between the two.
• You have a lot of intents in your training dataset. Confusion arises naturally as the number of intents grows.

Keep in mind that the training of the SnipsNLUEngine is stochastic, meaning that several runs of engine.fit(...) will produce slightly different trained models. This may explain some of the variability that you observe.
In order to remove this effect, you can add a random_seed value in the config that you pass to the engine:

config = deepcopy(CONFIG_EN)
config["random_seed"] = 42
engine = SnipsNLUEngine(config=CONFIG_EN)

Alternatively (and provided you are ok with that), if your dataset is not big, sharing it here would help understand what is happening.

[Moon-developer]

@adrienball Sorry for going quiet on this thread, have been far out of any internet connection. Anyways, So the file wasn't big at all. It gave the same problem just following the turnLightsOn examples on the doc. I was also using the same seed to make sure there wasn't any variation with the engine created.

One thing I did not do was the line you suggested:

from copy import deepcopy
config = deepcopy(CONFIG_EN)

This seems to have fixed the issue for me since originally I was only calling it directly like so:

from snips_nlu.default_configs import CONFIG_EN
nlu_engine = SnipsNLUEngine(config=CONFIG_EN)

So I am not too sure why this was an issue when it was working perfectly fine without it before.