[BUG] Sam model sessions do not respect providers argument

[jalsop24]

Describe the bug
Unlike the other session types (e.g. "u2net") the "sam" model does not respect the providers argument that can be provided to rembg.new_session

To Reproduce

import rembg

sam_session = rembg.new_session("sam", providers=["CPUExecutionProvider"])

print(f"sam encoder providers = {sam_session.encoder.get_providers()}")
# Outputs
# ['AzureExecutionProvider', 'CPUExecutionProvider']

print(f"sam decoder providers = {sam_session.decoder.get_providers()}")
# Outputs
# ['AzureExecutionProvider', 'CPUExecutionProvider']

u2_session = rembg.new_session("u2net", providers=["CPUExecutionProvider"])

print(f"u2net session providers = {u2_session.inner_session.get_providers()}")
# Outputs
# ['CPUExecutionProvider']

Expected behavior
I would expect the encoder and decoder to both use the providers as passed by the providers argument.

e.g.

import rembg

sam_session = rembg.new_session("sam", providers=["CPUExecutionContext"])

print(sam_session.encoder.get_providers())
# Expected output
# ['CPUExecutionProvider']

print(sam_session.decoder.get_providers())
# Expected output
# ['CPUExecutionProvider']

OS Version:
Windows 11 23H2

Rembg version:
v2.0.56

Additional context

pip list output

Package                   Version
------------------------- ---------
attrs                     23.2.0
certifi                   2024.2.2
charset-normalizer        3.3.2
colorama                  0.4.6
coloredlogs               15.0.1
flatbuffers               24.3.25
humanfriendly             10.0
idna                      3.7
imageio                   2.34.1
jsonschema                4.22.0
jsonschema-specifications 2023.12.1
lazy_loader               0.4
llvmlite                  0.42.0
mpmath                    1.3.0
networkx                  3.3
numba                     0.59.1
numpy                     1.26.4
onnxruntime               1.17.3
opencv-python-headless    4.9.0.80
packaging                 24.0
pillow                    10.3.0
pip                       23.0.1
platformdirs              4.2.2
pooch                     1.8.1
protobuf                  5.26.1
PyMatting                 1.1.12
pyreadline3               3.4.1
referencing               0.35.1
rembg                     2.0.56
requests                  2.31.0
rpds-py                   0.18.1
scikit-image              0.23.2
scipy                     1.13.0
setuptools                65.5.0
sympy                     1.12
tifffile                  2024.5.10
tqdm                      4.66.4
urllib3                   2.2.1

[jalsop24]

This is still an issue as of 2.0.57. I think commit 6f1dd31 breaks the fix that I submitted as the logic of a for-else statement means the else clause now always runs.

[LudoBar]

Still an issue in 2.0.59. The current code merges systematically available providers and input providers for the valid providers passed to ORT inference session. So there is no mean to select a specific provider (to force CPU computation for example). And it is quiet annoying because ORT can return non functional providers in the available list depending on the installation environment.