[Bug]: [group size] group size does not work if searching with offset

[yanliang567]

Is there an existing issue for this?

• I have searched the existing issues

Environment

- Milvus version: master-20240916-526a672b
- Deployment mode(standalone or cluster): standalone
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):  pymilvus 2.5.0rc80

Current Behavior

if searching with offset, group size is not working

Expected Behavior

should work

Steps To Reproduce

1. create a collection with varchar field
2. insert data and build hnsw
3. load
4. search group by varchar field, group_size=5, with offset=10
it returns results just like not setting group_size

Milvus Log

No response

Anything else?

No response

[yanliang567]

/assign @MrPresent-Han
/unassign