Define pickle protocol used as constant and default to ASCII
mbucc opened this issue · 0 comments
mbucc commented
I saw no speed difference in logging 40,000 transactions using the ASCII pickle protocol.
With an ASCII log file, you have some hope if you need to manually dig through the transaction history. It's also grep'able. In general, text log files are a good thing.
If file size is a concern, we should Gzip log files. (Probably should do that anyway---they get big quick.)
diff -r 687f471135fa topics/tornado/pv/core.py
--- a/topics/tornado/pv/core.py Wed Sep 28 08:02:16 2011 -0400
+++ b/topics/tornado/pv/core.py Wed Sep 28 08:08:27 2011 -0400
@@ -10,6 +10,8 @@
import re
from pv.util import baseN, NUMERALS
+#PICKLE_PROTOCOL = pickle.HIGHEST_PROTOCOL
+PICKLE_PROTOCOL = 0 # ASCII
class Log(object):
"""
@@ -83,12 +85,12 @@
def put(self, value):
self.serialId += 1
- pickle.dump(value, self.logFile, pickle.HIGHEST_PROTOCOL)
+ pickle.dump(value, self.logFile, PICKLE_PROTOCOL)
def putSnapshot(self, root):
# TODO refine error handling
snapshotFile = open(self.makeSnapshotName(self.serialId), 'ab')
- pickle.dump(root, snapshotFile, pickle.HIGHEST_PROTOCOL)
+ pickle.dump(root, snapshotFile, PICKLE_PROTOCOL)
snapshotFile.close()
self.logRotate(self.serialId)