Caching
Closed this issue · 2 comments
Is there any caching (other than pre-evaluated python output)? It looks like Knitr caches the chunk output automatically, but--unless one has an IPython session with the cumulative Python environment running in the background--I don't see chunks being cached/pickled.
If this is the case, it still solves the problem of sharing a Python environment between chunks in a single session, but not of iterated chunk editing between IPython sessions (unless chunks are independent with respect to evaluation in Python). A solution would seem to involve augmenting Knitr's caching mechanism to somehow include a pickling of the chunk results and reevaluating them, or even caching via IPython. I'm not already familiar with either, but I'd still be willing to help.
Hi Brandon,
Knitron does not do any caching on its own. I'm not so familiar with the automatic cache generation in knitr, but caching a chunk explicitly seems to work just fine.
However, caching a chunk and relying on it in an uncached chunk will not work. I think chunks should not be cached for their side effects. If you really have one chunk that takes a lot of time to generate, you can just pickle your object and load it up again using echo = FALSE.
When I first tried to improve Python support in knitr, I also tried what you suggested concerning pickling the Python environment. Unfortunately, in Python it's not as easy as it is in R to just pickle the entire workspace.
The reason why I finally chose to use IPython is simply because it allows us to send code and receive results asynchronously. This cannot easily be done with vanilla Python in a cross-platform way, because R's abilities to interact with interactive programs such as the Python interpreter are quite limited. Hence, IPython and its parallel API seemed like the easiest solution.
The other effect of being able to share an IPython session in multiple knitr documents is simply a side-effect of how knitron works right now and was not intentional. If this behavior is not desired one can simply use a different profile name.
Please feel free to comment if you see the need for more discussion.