Load psf waveforms on demand

Question

Load psf waveforms on demand

michaelnt opened this issue 14 years ago · 3 comments

Opening a large (700M) transient results file loads the complete file into memory, which is slow and uses lots of memory.

Would it be possible load the trace names and other meta data on open but then only load the waveforms on demand.

Answer 1 · 2011-09-18T05:05:03.000Z

Actually I have been working on an improved psf reader which use memory mapped
io to improve performance and memory consumption. I just found it in a git
stash and I have now merged it with the latest changes on master and created a
new branch for it. The new branch is called psfmmap.

I don't remember what state it is in, so it might not work at all. The reason
I abandoned was that I chose to spend more time on libpsf instead which has
far better performance than the mmap version of psf.py.

/Henrik

Answer 2 · 2011-09-22T13:07:03.000Z

Thanks, i'll tale a look at that when I get back from my holiday.

Do you think its better to focus on libpsf rather than the pure python
implementation?
On Sep 18, 2011 7:05 a.m., "Henrik Johansson" <
reply@reply.github.com>
wrote:

Actually I have been working on an improved psf reader which use memory
mapped
io to improve performance and memory consumption. I just found it in a git

stash and I have now merged it with the latest changes on master and
created a
new branch for it. The new branch is called psfmmap.

I don't remember what state it is in, so it might not work at all. The
reason
I abandoned was that I chose to spend more time on libpsf instead which
has
far better performance than the mmap version of psf.py.

/Henrik

Reply to this email directly or view it on GitHub:
#13 (comment)

Answer 3 · 2011-09-26T20:01:41.000Z

Yes, I think it's probably better to focus on libpsf because of its better performance. The disadvantage of libpsf is that it's not pure Python and right now more complicated to install.