CoreML stable diffusion image generation

The package is a mediator between Apple's Core ML Stable Diffusion implementation and your app that let you run text-to-image or image-to-image models

SwiftUI example

How to use the package

1. Create GenerativeManager

   let manager = GenerativeManager()

2. Run async method generate

       let images: [CGImage?] = try await manager.generate(
                with: config, 
                by: pipeline
            )

Performance

The speed can be unpredictable. Sometimes a model will suddenly run a lot slower than before. It appears as if Core ML is trying to be smart in how it schedules things, but doesn’t always optimal.

Typical set of files for a model und the purpose of each file

File Name	Description
`TextEncoder.mlmodelc`	Encodes input text into a vector space for further processing.
`Unet.mlmodelc`	Core model handling the transformation of encoded vectors into intermediate image representations.
`UnetChunk1.mlmodelc`	First segment of a segmented U-Net model for optimized processing in environments with memory constraints.
`UnetChunk2.mlmodelc`	Second segment of the segmented U-Net model, completing the tasks started by the first chunk.
`VAEDecoder.mlmodelc`	Decodes the latent representations into final image outputs.
`VAEEncoder.mlmodelc`	Compresses input image data into a latent space for reconstruction or further processing.
`SafetyChecker.mlmodelc`	Ensures generated content adheres to safety guidelines by checking against predefined criteria.
`vocab.json`	Contains the vocabulary used by the text encoder for tokenization and encoding processes.
`merges.txt`	Stores the merging rules for byte-pair encoding used in the text encoder.

Documentation(API)

You need to have Xcode 13 installed in order to have access to Documentation Compiler (DocC)
Go to Product > Build Documentation or ⌃⇧⌘ D

The-Igor/coreml-stable-diffusion-swift