launchflow/buildflow

Issues

• BuildFlow parent resource should fail if a child resource fails to create

  #331 opened a year ago by boetro
  0

• Metrics need to be injected as a dependency to work.

  #315 opened a year ago by boetro
  0

• Add support for Kafka IO connectors

  #123 opened 2 years ago by boetro
  1

• add sink for clickhouse

  #209 opened a year ago by boetro
  7

• require less than pydantic 2.0.0

  #213 opened 2 years ago by boetro
  2

• Add better support for processing headers and other components from an HTTP request

  #269 opened 2 years ago by boetro
  0

• Downscaling logic tends to over downscale once the backlog is gone.

  #112 opened 2 years ago by boetro
  0

• Add support for writing to remote object stores

  #102 opened 2 years ago by boetro
  0

• Add support for reading and writing from Amazon SQS

  #117 opened 2 years ago by boetro
  0

• Add support for end points

  #195 opened 2 years ago by boetro
  0

• GCP pub/sub provider should not return 0 instead of None for backlog

  #206 opened 2 years ago by boetro
  0

• update grpcio dependency to >= 1.56.0

  #210 opened 2 years ago by boetro
  0

• Resource allocation + GPU allocation

  #266 opened 2 years ago by uriafranko
  2

• only include the main "buildflow" package in deployment

  #217 opened 2 years ago by boetro
  0

• add better support for using GCS buckets as staging for writing to Snowflake

  #250 opened 2 years ago by boetro
  0

• reduce noisiness of node server logs

  #211 opened 2 years ago by boetro
  0

• update all interface like classes to provide a type var with a bound

  #219 opened 2 years ago by boetro
  0

• When shutdown replicas need to ensure that we kill replicas on the same node

  #114 opened 2 years ago by boetro
  1

• Add configuration options for in-memory retries.

  #207 opened 2 years ago by boetro
  0

• Add Node Server to monitor and destroy state of running node

  #189 opened 2 years ago by boetro
  0

• Add docs for buildflow internals

  #192 opened 2 years ago by boetro
  0

• you should be able to set autoscale options per processor not just at the node level

  #188 opened 2 years ago by boetro
  1

• Better error handling on Infra API

  #193 opened 2 years ago by boetro
  0

• Dump runtime snapshots to disk for historical snapshots

  #202 opened 2 years ago by boetro
  0

• Pulumi gets stuck in lock state which is hard to recover from

  #190 opened 2 years ago by boetro
  0

• dataclasses dont include all private fields in ray remote

  #198 opened 2 years ago by JoshTanke
  0

• Add support for batch processors as part of v2 API

  #194 opened 2 years ago by boetro
  0

• dataclass serialzation issue when running with `python main.py`

  #191 opened 2 years ago by boetro
  0

• Look into using placement groups for scheduling replicas

  #111 opened 2 years ago by boetro
  1

• add option to use BigQuery load jobs when writing to BigQuery

  #173 opened 2 years ago by boetro
  0

• add a `force` option to delete protected resources

  #168 opened 2 years ago by boetro
  0

• we should fail fast if you provided a type that we don't know how to convert

  #167 opened 2 years ago by boetro
  0

• Introduce more dev tools

  #140 opened 2 years ago by aeroaks
  1

• testing discord integration

  #145 opened 2 years ago by boetro
  1

• Add support for writing additional file types beyond parquet

  #101 opened 2 years ago by boetro
  8

• Add batch source for reading in file types (Parquet, JSON, CSV)

  #141 opened 2 years ago by boetro
  0

• Make autoscaler scale up ray nodes faster

  #110 opened 2 years ago by boetro
  0

• If sink or process actor dies the source actor will just spin on the messages.

  #136 opened 2 years ago by boetro
  1

• Add support for chaining together processors

  #135 opened 2 years ago by boetro
  0

• Make it possible to configure the resource allocations of the processor

  #113 opened 2 years ago by boetro
  2

• Usage of argparser for disabling usage stats breaks typer CLIs

  #124 opened 2 years ago by boetro
  0

• Add better autoscaling for streaming pipelines

  #99 opened 2 years ago by boetro
  0

• Add schema validation for local files types that support it

  #104 opened 2 years ago by boetro
  0

• Add support for writing to local files. Initially support parquet files

  #98 opened 2 years ago by boetro
  0

• Add IO template for consuming database change streams

  #91 opened 2 years ago by boetro
  0

• Add support for basic windowing for streaming pipelines

  #90 opened 2 years ago by boetro
  0

• Support source and sink for Azure Synapse Analytics

  #88 opened 2 years ago by boetro
  0

• Support source and sink for Azure Blob Storage

  #89 opened 2 years ago by boetro
  0

• Support source and sink for Azure Web PubSub

  #87 opened 2 years ago by boetro
  0

• Add support for snapshotting / checkpointing between pipeline stages

  #97 opened 2 years ago by boetro
  0