Saving chains of pocoMC during the run

[lindede249]

Hello everyone,
I've migrated from an older version of cosmosis to the current one (while keeping the cosmosis-standard-library fork we had the same as before) and therefore to be able to use pocoMC as the new sampler of choice.
Although the migration went flawlessly, and I am able to run the chains, I am not able to save the chains during the run. Mainly, since it gives me the following error (see screenshot below) when it tries to save them into a file (after running the chain for some while). This is happening for multi- and single-process runs. Within the old version I was mainly using emcee and the saving process was never giving any issues.

@minaskar and I were thinking that it might be due to version compatibility of for example Cython (currently using 3.0.10).
But since we don't know, I wanted to ask if anything like this was seen before?

Thank you very much in advance!

Greetings,
Dennis

[joezuntz]

i Hi @lindede249 - could you post the complete output as a text file? There should be more info further up about where this is happening.

[lindede249]

Thank you for the fast response!

Sure, here it is:
cosmosis_error.txt

[joezuntz]

Thanks - could you do this:

python -c 'import dill;print(dill.__version__);import pocomc;print(pocomc.__version__)'

and let us know what it prints, so I can try to replicate this?

[lindede249]

python -c 'import dill;print(dill.version);import pocomc;print(pocomc.version)'
0.3.8
1.2.2

[joezuntz]

Okay, I can replicate this now. The problem is that when pocomc saves its state it also pickles the Prior object, and the Prior object contains a reference to the complete cosmosis pipeline (because it uses it to keep track of the prior ranges). And if the LikelihoodPipeline list of modules has something unpicklable in like a class instance then it fails. Interesting! I'm a bit surprised it ever worked to pickle a pipeline.

I've been able to fix it by having the prior object use a global pipeline variable, just like the sampler object already does. I never really liked this design, but I needed it to be able to get things to work under MPI anyway, so it's a minor change. I think I changed it to this from Minas' original version anyway.

I'll put through a fix to this, but if you don't want to wait for the release process, you can fix it with these changes in the cosmosis/samplers/poco/pocomc_sampler.py file:

 class Prior:
-    def __init__(self, pipeline):
-        self.pipeline = pipeline
-        self.dim = len(self.pipeline.varied_params)
-        self.bounds = np.array([p.limits for p in self.pipeline.varied_params])
+    def __init__(self):
+        self.dim = len(pipeline.varied_params)
+        self.bounds = np.array([p.limits for p in pipeline.varied_params])
 
     def logpdf(self, x):
-        return np.array([self.pipeline.prior(p) for p in x])
+        return np.array([pipeline.prior(p) for p in x])
     
     def rvs(self, size=None):
-        return np.array([self.pipeline.randomized_start() for i in range(size)])
+        return np.array([pipeline.randomized_start() for i in range(size)])
 
 
 class PocoMCSampler(ParallelSampler):
@@ -73,7 +72,7 @@ class PocoMCSampler(ParallelSampler):
 
             # Finally we can create the sampler
             self.sampler = self.pocomc.Sampler(
-                prior=Prior(self.pipeline),
+                prior=Prior(),
                 likelihood=log_likelihood,
                 random_state=seed,
                 n_effective=self.n_effective,

[joezuntz]

Version 3.11.1 is now going through the release process on pypi and conda-forge, and fixes this issue.

[lindede249]

Thanks a lot for the quick help! The chain gets now saved properly!