How to remove model from memory after loading

[resolvepalapp]

Im currently unable to remove the model from memory once it's been loaded.
How to free up the memory once the model is no longer needed?

 let eval = await LLMEvaluator()
    do {
      modelContainer = try await eval.load()
      modelContainer = nil
      // Does nothing and model is still held in memory
    } catch {
      print("Failed to laod model", error)
    }
    
    // Loading
  func load() async throws -> ModelContainer {
    let modelContainer = try await loadModelContainer(configuration: modelConfiguration) {
      [modelConfiguration] progress in
      Task { @MainActor in
        self.modelInfo =
        "Downloading \(modelConfiguration.name): \(Int(progress.fractionCompleted * 100))%"
      }
    }
    
    return modelContainer
  }

[awni]

You probably need to use clearCache() to actually release the memory.

MLX has a cache of already allocated RAM for perf reasons. That RAM is free from the perspective of MLX but not the system. So to release it to the system you can use the clearCache method.

[resolvepalapp]

Thank you!