InferDM with snm_1d

[bibionid]

Hello,

Thanks again for your work on dadi-cli!

I have encountered a circular issue with the InferDM module, where if I want to infer the standard neutral model in 1D (snm_1d), upper and lower bounds are required, but I beleive there are no such paramters required for this model?

Error with no --lbounds or --ubounds

dadi-cli InferDM --fs <input>.fs --model snm_1d --output <output>.snm_1d.demo.params --optimizations 1 --nomisid
usage: dadi-cli InferDM [-h] --fs FS [--p0 P0 [P0 ...]] --output-prefix OUTPUT_PREFIX [--optimizations OPTIMIZATIONS] [--check-convergence] [--force-convergence] [--work-queue WORK_QUEUE WORK_QUEUE] [--port PORT] [--debug-wq] [--maxeval MAXEVAL]
                        [--maxtime MAXTIME] [--cpus CPUS] [--gpus GPUS] [--bestfit-p0-file BESTFIT_P0] [--delta-ll DELTA_LL] --model MODEL [--model-file MODEL_FILE] [--grids GRIDS GRIDS GRIDS] [--nomisid] [--constants CONSTANTS [CONSTANTS ...]]
                        --lbounds LBOUNDS [LBOUNDS ...] --ubounds UBOUNDS [UBOUNDS ...] [--global-optimization] [--seed SEED]
dadi-cli InferDM: error: the following arguments are required: --lbounds, --ubounds

Error with --lbounds and --ubounds set

dadi-cli InferDM --fs <input>.fs --model snm_1d --lbounds 0 --ubounds 1 --output <output>.snm_1d.demo.params --optimizations 1 --nomisid
Traceback (most recent call last):
  File "/opt/anaconda_dadi-cli/bin/dadi-cli", line 8, in <module>
    sys.exit(main())
  File "/opt/anaconda_dadi-cli/lib/python3.10/site-packages/dadi_cli/__main__.py", line 1603, in main
    args.runner(args)
  File "/opt/anaconda_dadi-cli/lib/python3.10/site-packages/dadi_cli/__main__.py", line 133, in run_infer_dm
    args.lbounds = _check_params(args.lbounds, args.model, "--lbounds", args.misid)
  File "/opt/anaconda_dadi-cli/lib/python3.10/site-packages/dadi_cli/__main__.py", line 1482, in _check_params
    raise Exception(
Exception:
Found 1 demographic parameters from the option --lbounds; however, 0 demographic parameters are required from the snm_1d model
You might be using the wrong model or need to add --nomisid if you did not use ancestral allele information to polarize the fs.

Error with --lbounds and --ubounds empty

dadi-cli InferDM --fs <input>.fs --model snm_1d --lbounds --ubounds --output <output>.snm_1d.demo.params --optimizations 1 --nomisid
usage: dadi-cli InferDM [-h] --fs FS [--p0 P0 [P0 ...]] --output-prefix OUTPUT_PREFIX [--optimizations OPTIMIZATIONS] [--check-convergence] [--force-convergence] [--work-queue WORK_QUEUE WORK_QUEUE] [--port PORT] [--debug-wq] [--maxeval MAXEVAL]
                        [--maxtime MAXTIME] [--cpus CPUS] [--gpus GPUS] [--bestfit-p0-file BESTFIT_P0] [--delta-ll DELTA_LL] --model MODEL [--model-file MODEL_FILE] [--grids GRIDS GRIDS GRIDS] [--nomisid] [--constants CONSTANTS [CONSTANTS ...]]
                        --lbounds LBOUNDS [LBOUNDS ...] --ubounds UBOUNDS [UBOUNDS ...] [--global-optimization] [--seed SEED]
dadi-cli InferDM: error: argument --lbounds: expected at least one argument

I'm not sure if I am using this model correctly, but I would be most grateful of any advice you can offer on this issue?

Many thanks

[tjstruck]

Hi @bibionid, thanks for bringing this to our attention! I've merged a fix for the standard neutral models, so you shouldn't require boundaries unless you want to fit a misid parameter. Let us know if you run into any other issues.

[bibionid]

Hi @tjstruck,

Thank you very much for your work on this issue.

I have downloaded the new version and have tested the command

dadi-cli InferDM --fs ${input}.fs --model snm_1d --output ${output} --optimizations 1 --nomisid

Unfortunately this generates the error

Traceback (most recent call last):
  File "/opt/anaconda_dadi-cli/bin/dadi-cli", line 8, in <module>
    sys.exit(main())
  File "/opt/anaconda_dadi-cli/lib/python3.10/site-packages/dadi_cli/__main__.py", line 1634, in main
    args.runner(args)
  File "/opt/anaconda_dadi-cli/lib/python3.10/site-packages/dadi_cli/__main__.py", line 139, in run_infer_dm
    model = dadi.Numerics.make_extrap_func(func)([], fs.sample_sizes, args.grids)
  File "/opt/anaconda_dadi-cli/lib/python3.10/site-packages/dadi/Numerics.py", line 374, in extrap_func
    result_l = list(map(partial_func, pts_l))
TypeError: 'NoneType' object is not iterable

I'd be most grateful of any input you might have on this? Am I calling the snm_1d correctly?

Many thanks

[bibionid]

To add to this, an output was created, but only containing a header

[tjstruck]

@bibionid, currently, with how dadi-cli is implementing models from dadi, you will need grid points even for the snm_1d model. You should be able to add something like --grids ns+5 ns+10 ns+15, where ns is the sample size of your data.

Since you aren't optimizing anything with a simple standard neutral model, only one output file will be produced and should have the log-likelihood and theta. The output (for now) should be named: <output>.InferDM.simple.snm.txt.

[bibionid]

Hi @tjstruck,

Thank you for looking at this issue and suggesting this soluiton. I have implemented the --grids flag and can confirm I got the expected output as you suggest.

I wanted to flag that the --grids flag is not mentioned in the documentation of InferDM in the readme, and would suggest this could be a helpful addition if it is necessary for accurate model inferance - I was able to derive model output without the flag in place from all other 1D models.

Again, many thanks for your work on this awesome tool!

[tjstruck]

Thanks for the heads up on the documentation and your experience with other models, @bibionid! I've pushed another up for the snm models to automatically generate grids like the other models.