kiwi-bdd/Kiwi

Rare should equal failure with dictionary comparison [Remastered].

lolgear opened this issue · 7 comments

lolgear commented

Well, as I told before

dictionaries equal doesn't work well.

I would like to narrow down the area for inspection, but I could only give you a clue:

git: yourkarma/jwt
branch: master ( latest release, for example, 2.0.2 )
JWTSpec.m
describe encoding -> context claims set -> it decode claims set and verify it correctly.

steps to reproduce:

  1. download latest release.
  2. open JWT.xcworkspace
  3. cleanup everything.
  4. run tests.
  5. do 4 - 3 - 4 - 4 - 4 - 3 - 4 or whatever order you'd like.
  6. catch results as failure of example mentioned above.
modocache commented

Thanks for the remastered version! 😁

So I guess the problem is still occurring? Is it possible to reduce your test case at all? That is, does a dictionary comparison fail stochastically when not using code from the JWT project?

I ask because Kiwi uses -[NSObject isEqual:] for its equality matcher -- there isn't any special code path when comparing NSDictionary. For two dictionaries to only sometimes not be equal, I'd be forced to suspect either:

  1. All Kiwi equality comparisons fail stochastically, or...
  2. ...Apple's -[NSDictionary isEqual:] sometimes returns different values.

I think the more likely explanation is that something in the JWTSpec_encoding_claims_set_decode_claims_set_and_verify_it_correctly test has a race condition, which causes the two dictionaries to not be equal sometimes.

Of course, it's possible there's some bug I'm overlooking in Kiwi. Reducing the test case would help find that, too!

lolgear commented

@modocache hey, how could I isolate this test?
Is it possible to move it to single spec file?
I think no, because kiwi doesn't allow to run single test :(
Single test project as an option?

modocache commented

Hey @lolgear! Yeah, I think a new Xcode project with just this test, and just the source code it's testing, will probably help isolate the problem better.

lolgear commented

@modocache
check master branch:
https://github.com/lolgear/JWT
Inspection -> SherlockHolmes project. ( In spec you could find steps to reproduce issue )

lolgear commented

@modocache any update?

modocache commented

Not yet, sorry! Will try to find a spare minute during a weekend soon.

modocache commented

Or if you have some spare time before I do, you could take a look at the Kiwi internals to try and figure it out!