An error with MOSEK call in AmpliconArchitect.py

[carrie1s]

version:
ampliconsuite 1.3.2
mosek 10.1.31

command:
AmpliconSuite-pipeline.py -s XXX -t 12 --bam ../bam/XX.bam --run_AA --run_AC --ref hg38

Report error when running the step:
AmpliconArchitect.py --ref GRCh38 --downsample 10 --bed XX_AA_CNV_SEEDS.bed --bam ../bam/XX.bam --runmode FULL --extendmode EXPLORE --out XX

ERROR LOG:
INFO:MOSEK:Beginning MOSEK call
DEBUG:MOSEK:Problem
DEBUG:MOSEK: Name :
DEBUG:MOSEK: Objective sense : minimize
DEBUG:MOSEK: Type : CONIC (conic optimization problem)
DEBUG:MOSEK: Constraints : 674
DEBUG:MOSEK: Affine conic cons. : 858 (2574 rows)
DEBUG:MOSEK: Disjunctive cons. : 0
DEBUG:MOSEK: Cones : 0
DEBUG:MOSEK: Scalar variables : 1716
DEBUG:MOSEK: Matrix variables : 0
DEBUG:MOSEK: Integer variables : 0
DEBUG:MOSEK:
DEBUG:MOSEK:Optimizer started.
DEBUG:MOSEK:Presolve started.
DEBUG:MOSEK:Linear dependency checker started.
DEBUG:MOSEK:Linear dependency checker terminated.
DEBUG:MOSEK:Eliminator started.
DEBUG:MOSEK:Freed constraints in eliminator : 2
DEBUG:MOSEK:Eliminator terminated.
DEBUG:MOSEK:Eliminator - tries : 1 time : 0.00
DEBUG:MOSEK:Lin. dep. - tries : 1 time : 0.00
DEBUG:MOSEK:Lin. dep. - primal attempts : 1 successes : 1
DEBUG:MOSEK:Lin. dep. - dual attempts : 0 successes : 0
DEBUG:MOSEK:Lin. dep. - primal deps. : 0 dual deps. : 0
DEBUG:MOSEK:Presolve terminated. Time: 0.01
DEBUG:MOSEK:Optimizer - threads : 24
DEBUG:MOSEK:Optimizer - solved problem : the primal
DEBUG:MOSEK:Optimizer - Constraints : 672
DEBUG:MOSEK:Optimizer - Cones : 856
DEBUG:MOSEK:Optimizer - Scalar variables : 2568 conic : 2568
DEBUG:MOSEK:Optimizer - Semi-definite variables: 0 scalarized : 0
DEBUG:MOSEK:Factor - setup time : 0.00
DEBUG:MOSEK:Factor - dense det. time : 0.00 GP order time : 0.00
DEBUG:MOSEK:Factor - nonzeros before factor : 1769 after factor : 2188
DEBUG:MOSEK:Factor - dense dim. : 0 flops : 1.91e+04
DEBUG:MOSEK:ITE PFEAS DFEAS GFEAS PRSTATUS POBJ DOBJ MU TIME
DEBUG:MOSEK:0 2.6e+00 9.2e+05 1.5e+07 0.00e+00 1.481523133e+07 -6.891673134e+02 1.0e+00 0.01
DEBUG:MOSEK:1 8.6e-01 3.1e+05 8.6e+06 -1.00e+00 1.481247189e+07 -3.374965357e+03 3.3e-01 0.02
DEBUG:MOSEK:2 3.7e-01 1.3e+05 5.6e+06 -1.00e+00 1.480721678e+07 -8.420865348e+03 1.4e-01 0.02
DEBUG:MOSEK:3 1.0e-01 3.6e+04 2.9e+06 -1.00e+00 1.478185341e+07 -3.265364634e+04 3.9e-02 0.03
DEBUG:MOSEK:4 3.0e-02 1.1e+04 1.6e+06 -1.00e+00 1.469091742e+07 -1.189922701e+05 1.2e-02 0.03
DEBUG:MOSEK:5 1.2e-02 4.2e+03 1.0e+06 -9.99e-01 1.445276664e+07 -3.425507493e+05 4.6e-03 0.03
DEBUG:MOSEK:6 5.2e-03 1.8e+03 6.6e+05 -9.96e-01 1.378685075e+07 -9.575357576e+05 2.0e-03 0.04
DEBUG:MOSEK:7 1.6e-03 5.7e+02 3.6e+05 -9.87e-01 1.072088581e+07 -3.763634589e+06 6.2e-04 0.04
DEBUG:MOSEK:8 2.1e-04 7.5e+01 1.2e+05 -9.45e-01 -1.509111365e+07 -2.719934969e+07 8.2e-05 0.04
DEBUG:MOSEK:9 6.3e-05 2.3e+01 4.2e+04 -5.57e-01 -4.934651656e+07 -5.694897247e+07 2.5e-05 0.04
DEBUG:MOSEK:10 4.2e-05 1.5e+01 2.4e+04 2.48e-01 -5.583431514e+07 -6.141871470e+07 1.6e-05 0.05
DEBUG:MOSEK:11 1.6e-05 5.7e+00 6.0e+03 4.71e-01 -6.713890867e+07 -6.960237043e+07 6.2e-06 0.05
DEBUG:MOSEK:12 6.1e-06 2.2e+00 1.4e+03 7.90e-01 -7.234123926e+07 -7.333062025e+07 2.4e-06 0.05
DEBUG:MOSEK:13 1.4e-06 4.9e-01 1.6e+02 9.22e-01 -7.491807945e+07 -7.514736454e+07 5.3e-07 0.06
DEBUG:MOSEK:14 5.2e-07 1.8e-01 3.7e+01 9.84e-01 -7.536359378e+07 -7.545053101e+07 2.0e-07 0.06
DEBUG:MOSEK:15 1.5e-07 5.3e-02 5.8e+00 9.94e-01 -7.555050335e+07 -7.557572224e+07 5.8e-08 0.06
DEBUG:MOSEK:16 2.0e-08 7.1e-03 2.9e-01 9.98e-01 -7.561503075e+07 -7.561838064e+07 7.7e-09 0.07
DEBUG:MOSEK:17 2.6e-09 9.3e-04 1.4e-02 1.00e+00 -7.562296054e+07 -7.562339930e+07 1.0e-09 0.07
DEBUG:MOSEK:18 1.9e-10 6.9e-05 2.8e-04 1.00e+00 -7.562406459e+07 -7.562409717e+07 7.5e-11 0.07
DEBUG:MOSEK:19 1.6e-10 5.3e-05 1.9e-04 9.98e-01 -7.562408461e+07 -7.562410986e+07 5.8e-11 0.08
DEBUG:MOSEK:20 1.6e-10 5.3e-05 1.9e-04 9.98e-01 -7.562408461e+07 -7.562410986e+07 5.8e-11 0.09
DEBUG:MOSEK:21 1.6e-10 5.3e-05 1.9e-04 9.98e-01 -7.562408461e+07 -7.562410986e+07 5.8e-11 0.10
DEBUG:MOSEK:Optimizer terminated. Time: 0.12
DEBUG:MOSEK:
DEBUG:MOSEK:
DEBUG:MOSEK:Interior-point solution summary
DEBUG:MOSEK: Problem status : UNKNOWN
DEBUG:MOSEK: Solution status : UNKNOWN
DEBUG:MOSEK: Primal. obj: -7.5624084608e+07 nrm: 1e+03 Viol. con: 5e-06 var: 0e+00 acc: 0e+00
DEBUG:MOSEK: Dual. obj: -7.5624109859e+07 nrm: 4e+06 Viol. con: 0e+00 var: 2e+00 acc: 0e+00
ERROR:MOSEK:Error when using MOSEK: Failed to solve to optimality. Solution status solsta.unknown
INFO:MOSEK:Saved MOSEK inputs to mosekinput-1.json. Submit that file to support to reproduce the issue.

I have run 6 samples, only this sample report the error. The other 5 samples did not show any issues. The amplicon1 information generated well but amplicon2 encountered this issue. I cannot solve this problem. Is my mosek version wrong or any other specific settings? Please help me.

[jluebeck]

Thank you, would you kindly be able to provide the mosekinput-1.json file? You can email it to jluebeck [at] ucsd.edu. I can work with the MOSEK team to understand the issue.

Thank you,
Jens

[carrie1s]

Thank you, would you kindly be able to provide the mosekinput-1.json file? You can email it to jluebeck [at] ucsd.edu. I can work with the MOSEK team to understand the issue.

Thank you, Jens

OK. I uploaded both the mosekinput-1.json, log file and summary file here. And I have also emailed to jluebeck [at] ucsd.edu.
320dm_shTOP2B_summary.txt
320dm_shTOP2B.log
mosekinput-1.json

Thank you very much,
Carrie

[jluebeck]

Thank you, Carrie. My last question is if you can share the [sample]_amplicon2_graph.txt file with me as well?

Thank you!
Jens

[jluebeck]

Hi Carrie,

I have identified the source of the issue (it is a rare convergence issue with Mosek's optimization) and we can provide a fix for this in the next update to AA. Please give us a few weeks to provide a new release with some other features coming down the pipe.

Thanks,
Jens

[carrie1s]

Thank you, Carrie. My last question is if you can share the [sample]_amplicon2_graph.txt file with me as well?

Thank you! Jens

Sure, Jens. I apologize for not checking my emails in a timely manner. Here are the files generated in output directory "XX_AA_results" . I have packed them up.
320dm_shTOP2B_AA_results.zip

It's great that you found the problem. I hope to see the updated version soon. Many thanks to the maintenance team for your efforts.

Thanks,
Carrie

[carrie1s]

Also the "XX_cnvkit_output" Here:
320dm_shTOP2B_cnvkit_output.zip

[jluebeck]

Thank you - this is very helpful. In the meantime before the fix, I recommend re-running AmpliconSuite-pipeline.py with the following parameter: --AA_insert_sdevs 9

Thanks,
Jens

[jluebeck]

A bypass for the Mosek issue is included in AA v 1.4.r2 (AmpliconSuite v1.3.3), and an error will be logged but it will not crash. However when this occurs it is best for the user to re-run with --AA_insert_sdevs 9.