Keyword sometimes gets decoded as non-interned symbol
Closed this issue · 6 comments
I have an application that does a lot of encoding and decoding of objects using Conspack. These activities occur in multiple threads.
In some rare instances, encoded keywords are subsequently decoded as a non-interned symbol. In other words: :|foo| becomes decoded as #:|foo|.
I don't know if it could be related, but the problem always occurs with specific symbols that are unique in the following ways: 1) The names are uncommon. They have likely not been encoded before. 2) They are in lower-case.
Is Conspack thread-safe? Could this be cause by the fact that I am doing lots of encoding and decoding operations at the same time?
I have attempted to reproduce this problem in a controlled environment, but I haven't been able to. This also suggests that there is some kind of race condition.
Any advice would be much appreciated.
This is actually by-design, but it's caught me once or twice as well. The answer is related specifically to "the names are uncommon"---so uncommon they're almost certainly not interned yet! Conspack will not intern symbols by default, because symbols are not easily collected and it's not otherwise easy to control while decoding, etc etc.
The quick solution is, use WITH-INTERNING around your decoding and it should intern everything. I wouldn't recommend this from "untrusted" sources, but that doesn't sound like the case here.
You could also make sure that all symbols you're using are interned ahead-of-time, but that may not be practical in all situations.
WRT thread safety, I can't think off-hand why it wouldn't be; the only state used during decodes is optional and specified via e.g. TRACKING-REFS or similar, which dynamically-scope state. I am under the impression that this is "safe" in most implementations. There should be no potential for race conditions.
Obviously, this only applies to the encoding/decoding logic... no locking is done, so if object structure changes during encoding, you will have issues.
Thank you. I believe that using WITH-INTERNING is the solution.
I note that the PROPERTIES variable seems to be globally modified during
encoding without being protected by a mutex or similar mechanism. Am I
mistaken on this?
On 15 January 2015 at 01:21, Ryan Pavlik notifications@github.com wrote:
WRT thread safety, I can't think off-hand why it wouldn't be; the only
state used during decodes is optional and specified via e.g. TRACKING-REFS
or similar, which dynamically-scope state. I am under the impression that
this is "safe" in most implementations. There should be no potential
for race conditions.Obviously, this only applies to the encoding/decoding logic... no locking
is done, so if object structure changes during encoding, you will have
issues.—
Reply to this email directly or view it on GitHub
#1 (comment).
Oh! Good point. I will fix this either by requiring (or implicitly providing) a WITH-PROPERTIES or simply locking. The former solution would break existing non-ENCODE-OBJECT code using properties, but is probably best. I will fix this shortly.
Sorry for the delay; properties should now be fixed. This may break some code if people are making use of properties outside encode/decode, but the fix should be relatively simple (i.e., use the new block).
Closing.