Targeting rules using percentage not working

[ilyatovbin-pp]

Describe the bug
When applying split targeting rules with percentages, anything below around 35-40% will always return a False Value. Anything Above it will always return a True value. Very inconsistent depending on whether you are raising percentage or lowering it and your starting point.

Which area does this issue belong to?

• FeatureHub Admin Web app
• SDK
• SDK examples
• Documentation
• Other

To Reproduce
Steps to reproduce the behavior:

1. Create a Feature, set it to OFF.
2. Create a split targeting rule(userkey="splitme"), set it to ON and set 20%
3. Run the script:

from featurehub_sdk.featurehub_config import FeatureHubConfig
import asyncio

edge_url = "http://localhost:8085"
client_eval_key = "MY_CLIENT_KEY"
fh_config = FeatureHubConfig(edge_url, [client_eval_key])
asyncio.run(fh_config.init())


def name_arg():
    if fh_config.new_context().user_key("splitme").build_sync().feature('SUPER_DUPER_FEATURE').get_flag:
        return True
    else:
        return False

import time
i = 0
while True:
    print(name_arg())
    time.sleep(0.2)

You should constantly get False as a result.
4. Now change the split targeting to 40%
5. Now you should still get constant True

you can also start at 40% and get constantly a False result, then raising it to 50 and getting constant True, lower it to 20% (constant False) then raise it back to 40% and still get constant False.

Expected behavior
The split value should return the split value based on actual percentage (1 in 5 in this instance)

Versions

• FeatureHub version [featurehub/party-server:latest]
• SDK version 0.0.2 (python)
• OS: [Ubuntu 20.04LTS + Python3.10]

[rvowles]

Hey there,

I think there is a misunderstanding of how percentages work. We have it in the general documentation, but perhaps its not clear enough or we should replicate it through the different SDKs? Your suggestions would be appreciated given you are hitting this :-) The split percentage is based on the user key (or the session key, it prefers the session key and falls back to the user key) - from this it generates a murmur3 hash - which gives it a spread across 100%. This means the same user key will always generate the same percentage value. The percentage spread is intended for example to spread across different users not the same user over time. A small subset of users is likely to generate a shewed percentage where you don't get exactly what you expect because its the hash is intended to randomly place users in that 100% in an even fashion but it needs a pool size of a decent amount to get it accurate.

Is this any help?

[ilyatovbin-pp]

Might have not read that part (yet), but yes I understand. My brain was of the assumption that each request hitting the split targeting rule will then either return the new value or fall back to the default one based on the percentage. My bad.

My best option in this case then would be simply to generate some form of randomness for each request, and set the rule condition to something like Starts with. This will give me what I want to achieve.

    key = f"user-{uuid.uuid4()}"
    flag = fh_config.new_context().user_key(key).build_sync().feature('SUPER_DUPER_FEATURE').get_string