faust-streaming/faust

@app.table_route points to the wrong worker

victorjourne opened this issue · 0 comments

Checklist

  • I have included information about relevant versions
  • I have verified that the issue persists when using the master branch of Faust.

Steps to reproduce

Trying to use faust table view with 3 faust workers and 3 topics partitions with 1 Redpanda broker.
Please check my sandbox repository to view clearly what happen and how to precisely reproduce it with Docker!

In a nutshell, here is a dummy app, executed with TOPIC_PARTITIONS: 3 itself in 3 docker containers.

import faust
from faust.web import Request, Response, View
import os

app = faust.App( 'test',
    broker=os.getenv("FAUST_BROKER_URL"),
    topic_partitions=3
)
table = app.Table('info', key_type=str, value_type=str)

@app.agent(key_type=str, value_type=str, concurrency=1)
async def write_info(stream):
    async for key, value in stream.items():
        table[key] = value
        yield

@app.page('/info/{key}/')
@app.table_route(table=table, match_info='key')
async def get_info(web, request: Request, key) -> Response:
    try:
        value = table[key]
        return web.json(value)
    except KeyError:
        raise View.NotFound(f"Cannot find this symbol among [{table.keys()}]")

@app.page("/info")
class info(View): 
    async def get(self, request: Request) -> Response:
        # Downloads infos
        return self.json({symbol: value for symbol, value in table.items()})

    async def post(self, request: Request) -> Response:
        json_data = await request.json()
        await write_info.cast(key=json_data['key'], value=json_data['value'])
        return self.json({"message": f"Post {json_data['key']}"})

After populating the table info with A,B,C,D elements, the faust router not to seem working as expected.

$ curl localhost:6066/info/D/
$ {"error":"Cannot find this symbol among [dict_keys(['A', 'B', 'C'])]"}

Expected behavior

The routing points to the correct worker faust-2 which hold the active partition with the element D.

Actual behavior

The routing points to the worker faust-1 which hold the stand-by but correct partition 0.

Full traceback

Screenshot_test-info-changelog_Redpanda_Console
Screenshot_docker_standby
Screenshot_docker_active

Versions

  • Python version : 3.11
  • Faust version : 0.10.13
  • Operating system : Linux
  • Redpanda version : 23.3.11