Mogi example main program fails
EJFielding opened this issue · 15 comments
EJFielding commented
I was also unable to run the main example code for the Mogi model:
models/mogi/examples] fielding% mogi altar: could not resolve 'mogi' into a component that implements protocol 'altar.models'
aivazis commented
Eric,
Both of these are the same error, caused by the fact that the legacy makefile doesn’t build the new library. I will fix asap.
…-- Michael
On Nov 20, 2018, at 7:59 AM, Eric Fielding ***@***.***> wrote:
I was also unable to run the main example code for the Mogi model:
models/mogi/examples] fielding% mogi altar: could not resolve 'mogi' into a component that implements protocol 'altar.models'
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.
EJFielding commented
Michael,
Thanks for the explanation. I was not sure if it was the same cause or not.
++Eric
From: Michael Aivazis <notifications@github.com>
Reply-To: AlTarFramework/altar <reply@reply.github.com>
Date: Tuesday, November 20, 2018 at 8:21 AM
To: AlTarFramework/altar <altar@noreply.github.com>
Cc: "Eric J. Fielding" <Eric.J.Fielding@jpl.nasa.gov>, Author <author@noreply.github.com>
Subject: Re: [AlTarFramework/altar] Mogi example main program fails (#7)
Eric,
Both of these are the same error, caused by the fact that the legacy makefile doesn’t build the new library. I will fix asap.
-- Michael
On Nov 20, 2018, at 7:59 AM, Eric Fielding ***@***.***> wrote:
I was also unable to run the main example code for the Mogi model:
models/mogi/examples] fielding% mogi altar: could not resolve 'mogi' into a component that implements protocol 'altar.models'
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub<#7 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AG6uSBw8D5fxtYCa3nymaudpmYmILsBmks5uxCwKgaJpZM4YraGz>.
EJFielding commented
I am now getting a different error when I try to run the Mogi example main model:
models/mogi/examples] fielding% mogi
altar: unable to find support for <fast> mode
altar: file='/Users/fielding/Documents/progs/Caltech/AlTar-2.0/models/mogi/mogi/Mogi.py', line=146, function='initialize': firewall breached; aborting...
aivazis commented
Eric,
It looks like the C++/CUDA implementations are not available in your installation. What di you use to build altar?
As a check, edit mogi.pfg and switch the “mode” to “native”; this will fall back to the pyre python implementation. Verify that altar behaves.
— Michael
… On Feb 22, 2019, at 11:45 PM, Eric Fielding ***@***.***> wrote:
I am now getting a different error when I try to run the Mogi example main model:
models/mogi/examples] fielding% mogi altar: unable to find support for <fast> mode altar: file='/Users/fielding/Documents/progs/Caltech/AlTar-2.0/models/mogi/mogi/Mogi.py', line=146, function='initialize': firewall breached; aborting...
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub <#7 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGCCyYewY_LV9eZ5868NYNsCTlgSk1Wjks5vQPGzgaJpZM4YraGz>.
EJFielding commented
I am still using the old "make" method to build AlTar 2.0.
I changed the mogi.pfg file to mode "native" and it is running successfully now.
aivazis commented
Ok. I’ll check why make didn’t build the extensions. You are on a mac, right?
— Michael
… On Feb 23, 2019, at 9:20 AM, Eric Fielding ***@***.***> wrote:
I am still using the old "make" method to build AlTar 2.0.
I changed the mogi.pfg file to mode "native" and it is running successfully now.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub <#7 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGCCyRI5rh8eTOOarpsiKZ7SwznLlnCCks5vQXhGgaJpZM4YraGz>.
EJFielding commented
Yes, MacBook with MacPorts.
EJFielding commented
I have updated my Pyre and AlTar installations with git pull. They seem to install successfully. Now when I try to run "mogi" in the examples directory, I get a new error:
Traceback (most recent call last): File "/Users/fielding/Documents/progs/Caltech/AlTar-2.0/altar/builds/bin/mogi", line 32, in <module> status = app.run() File "/Users/fielding/Documents/progs/Caltech/extra_aivazis/pyre/products/packages/pyre/shells/Application.py", line 197, in run return self.shell.launch(self, *args, **kwds) File "/Users/fielding/Documents/progs/Caltech/extra_aivazis/pyre/products/packages/pyre/shells/Script.py", line 49, in launch status = application.main(*args, **kwds) File "/Users/fielding/Documents/progs/Caltech/AlTar-2.0/altar/altar/shells/Application.py", line 53, in main self.controller.initialize(application=self) File "/Users/fielding/Documents/progs/Caltech/AlTar-2.0/altar/altar/bayesian/Annealer.py", line 58, in initialize self.sampler.initialize(application=application) File "/Users/fielding/Documents/progs/Caltech/AlTar-2.0/altar/altar/bayesian/Metropolis.py", line 54, in initialize self.uniform = altar.pdf.uniform_pos(support=(0,1), rng=rng) File "/Users/fielding/Documents/progs/Caltech/extra_aivazis/pyre/products/packages/gsl/pdf.py", line 116, in __init__ super().__init__(**kwds) TypeError: object.__init__() takes exactly one argument (the instance to initialize)
I get the same error whether I have the mode "native" or "fast" in the mogi.pfg file.
lijun99 commented
HI Eric,
The altar.pdf.uniform_pos(support=(0,1), rng=rng) doesn't need support.
Please change it to
self.uniform = altar.pdf.uniform_pos(rng=rng)
in altar/bayesian/Metropolis.py.
I'll ask Michael to update the repo.
Best,
Lijun
…On Fri, May 3, 2019 at 4:25 PM Eric Fielding ***@***.***> wrote:
I have updated my Pyre and AlTar installations with git pull. They seem to
install successfully. Now when I try to run "mogi" in the examples
directory, I get a new error:
Traceback (most recent call last): File
"/Users/fielding/Documents/progs/Caltech/AlTar-2.0/altar/builds/bin/mogi",
line 32, in <module> status = app.run() File
"/Users/fielding/Documents/progs/Caltech/extra_aivazis/pyre/products/packages/pyre/shells/Application.py",
line 197, in run return self.shell.launch(self, *args, **kwds) File
"/Users/fielding/Documents/progs/Caltech/extra_aivazis/pyre/products/packages/pyre/shells/Script.py",
line 49, in launch status = application.main(*args, **kwds) File
"/Users/fielding/Documents/progs/Caltech/AlTar-2.0/altar/altar/shells/Application.py",
line 53, in main self.controller.initialize(application=self) File
"/Users/fielding/Documents/progs/Caltech/AlTar-2.0/altar/altar/bayesian/Annealer.py",
line 58, in initialize self.sampler.initialize(application=application)
File
"/Users/fielding/Documents/progs/Caltech/AlTar-2.0/altar/altar/bayesian/Metropolis.py",
line 54, in initialize self.uniform = altar.pdf.uniform_pos(support=(0,1),
rng=rng) File
"/Users/fielding/Documents/progs/Caltech/extra_aivazis/pyre/products/packages/gsl/pdf.py",
line 116, in __init__ super().__init__(**kwds) TypeError: object.__init__()
takes exactly one argument (the instance to initialize)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#7 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AJHRSGP67KAVKV375NZNEK3PTTCX3ANCNFSM4GFNUGZQ>
.
EJFielding commented
Thanks Lijun.
It runs after I made that change, but the results are all "nan".
altar: step
β: 0
θ: (4096 samples) x (6 parameters)
Σ: 6 x 6
[[ +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00
+0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00
+0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00
+0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00
+0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00
+0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 ]]
** ** ** GSL error: endpoints do not enclose a minimum
altar: step
β: 0.999999999850374
θ: (4096 samples) x (6 parameters)
Σ: 6 x 6
[[ +nan +nan +nan +nan +nan +nan
+nan +nan +nan +nan +nan +nan
+nan +nan +nan +nan +nan +nan
+nan +nan +nan +nan +nan +nan
+nan +nan +nan +nan +nan +nan
+nan +nan +nan +nan +nan +nan ]]
lijun99 commented
Hi Eric and Michael,
I have encountered the same problem. It seems to be the problem with mogi:
if you run other models, such as linear, it should work well. I can take a
look. Michael, could you please check it as well?
Best,
Lijun
…On Fri, May 3, 2019 at 4:59 PM Eric Fielding ***@***.***> wrote:
Thanks Lijun.
It runs after I made that change, but the results are all "nan".
altar: step
β: 0
θ: (4096 samples) x (6 parameters)
Σ: 6 x 6
[[ +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00
+0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00
+0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00
+0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00
+0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00
+0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 ]]
** ** ** GSL error: endpoints do not enclose a minimum
altar: step
β: 0.999999999850374
θ: (4096 samples) x (6 parameters)
Σ: 6 x 6
[[ +nan +nan +nan +nan +nan +nan
+nan +nan +nan +nan +nan +nan
+nan +nan +nan +nan +nan +nan
+nan +nan +nan +nan +nan +nan
+nan +nan +nan +nan +nan +nan
+nan +nan +nan +nan +nan +nan ]]
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#7 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AJHRSGIJGZNPWDEAV4RETHDPTTGV7ANCNFSM4GFNUGZQ>
.
lijun99 commented
Hi Eric and Michael,
I think the problem is, after using the right form of likelihood norm**2/2
(or larger negative values), the framework doesn't handle it well.
To make it work, some of components need to be updated (or brought back
from altar-1,1),
1. the dbeta solver with iterative grid-based searching method, instead of
gsl solver;
2. the posterior needs to recomputed after beta update.
and possibly,
3. condition the covariance matrix, which is not simply symmetrize the
matrix, but makes some parameters less singular if they don't get updated.
(may not to be critical if parameters are well defined).
I think mogi presents an interesting example for testing altar framework:
some parameters have large search ranges while others are small; the rate
of invalid proposals are extremely high (> 99%) in the first few beta
steps. Can someone give me a concrete example as well as the desired
results so that I can make sure the framework works well, not converging to
some local maximum. I also noticed that Grace made some changes in her
pull request; are they relevant?
These updates were already included in my cuda branch,
https://github.com/lijun99/altar/tree/cuda. I will try to prepare a pull
request version once I make sure the mogi results are correct.
Best,
Lijun
…On Fri, May 3, 2019 at 5:29 PM Lijun Zhu ***@***.***> wrote:
Hi Eric and Michael,
I have encountered the same problem. It seems to be the problem with mogi:
if you run other models, such as linear, it should work well. I can take a
look. Michael, could you please check it as well?
Best,
Lijun
On Fri, May 3, 2019 at 4:59 PM Eric Fielding ***@***.***>
wrote:
> Thanks Lijun.
>
> It runs after I made that change, but the results are all "nan".
>
> altar: step
>
> β: 0
>
> θ: (4096 samples) x (6 parameters)
>
> Σ: 6 x 6
>
> [[ +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00
>
> +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00
>
> +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00
>
> +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00
>
> +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00
>
> +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 +0.0000e+00 ]]
>
> ** ** ** GSL error: endpoints do not enclose a minimum
>
>
>
> altar: step
>
> β: 0.999999999850374
>
> θ: (4096 samples) x (6 parameters)
>
> Σ: 6 x 6
>
> [[ +nan +nan +nan +nan +nan +nan
>
> +nan +nan +nan +nan +nan +nan
>
> +nan +nan +nan +nan +nan +nan
>
> +nan +nan +nan +nan +nan +nan
>
> +nan +nan +nan +nan +nan +nan
>
> +nan +nan +nan +nan +nan +nan ]]
>
>
>
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <#7 (comment)>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AJHRSGIJGZNPWDEAV4RETHDPTTGV7ANCNFSM4GFNUGZQ>
> .
>
EJFielding commented
Hi Lijun,
The input data for the Mogi model is generated by the program in the directory below, so you can easily check whether the inversion is getting the synthetic model correct.
Best,
++Eric
lijun99 commented
(Updated the method to pull from another repo. -Lijun)
git remote add develop https://github.com/lijun99/altar.git
git fetch develop pr
git checkout pr
Hi Eric,
Thanks! I used the generator but didn't pay attention to what's in there.
It seems that
dV = altar.properties.float(default=1.e10), not 10e10, judging from the
search range (9,11) in mogi.pfg.
Here are the results I got
parameters (mean, sd):
(0.006367653705854481, 3.9929836489301134)
(0.07916527850719468, 4.548599168862911)
(0.0009419295831507296, 0.056472701051412146)
(7.252818172805752e-05, 0.05711650471564437)
(10.000018077782386, 0.0011839893649379205)
(3000.0094113483433, 6.044272715864086)
which are positions x2, offsets x2, dv, depth. (The order of parameters may
vary during simulation, since there is randomness how parameters are added
to psets.)
So it agrees with the sample generator.
It turns out that conditioning the covariance matrix (to ensure positive
definiteness) is bad for mogi, because the search ranges (or variances) for
different parameters differ a lot. I disabled it for now; but we need to
think about how to do it properly in the future.
So, I have prepared a pull request version and submitted to github. If you
or someone else are interested in testing the modified version before
Michael merges it to master, you could pull my branch at first
git remote add develop https://github.com/lijun99/altar.git
git fetch develop pr
git checkout pr
Then make altar as usual.
Best,
Lijun
…On Sun, May 5, 2019 at 5:34 PM Eric Fielding ***@***.***> wrote:
Hi Lijun,
The input data for the Mogi model is generated by the program in the
directory below, so you can easily check whether the inversion is getting
the synthetic model correct.
Best,
++Eric
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#7 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AJHRSGIG6XAXVLIQZ4I4ZH3PT54HDANCNFSM4GFNUGZQ>
.
EJFielding commented
Lijun,
The mogi version in your "pr" branch works for me and gives good results, but only in the slow "native" mode.
@aivazis Are you still planning to fix the AlTar2 build system to work with the old "mm" and "make" or do we all need to switch to the new "mm"?