calhounpaul/outlines-vllm-entity-extraction

outlines-vllm-entity-extraction

A demo that uses the Outlines module with the vLLM inference engine to populate JSON dictionaries that conform to precise schema, using minimally-structured input text (the current sample is of controller files from various .NET APIs).

Prerequisites are just python3, docker, and the nvidia container toolkit:

• https://docs.docker.com/engine/install/ubuntu/

• https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html