Tensorflow build failed after the ml_dtypes new release

[ShahriyarR]

My current dependencies are:

dependencies = [
    "annoy >=1.17.2",
    "tensorflow >=2.13.0rc0",
    "tf-nightly >=2.14.0.dev20230511",
    "mlflow >=2.5.0",
    "boto3 >=1.26.145",
    "dependency-injector >=4.41.0"
]

The CI run installation versions:
Successfully installed keras-2.14.0 keras-nightly-2.15.0.dev2023092007 ml-dtypes-0.3.0 tb-nightly-2.15.0a20230919 tensorboard-2.14.0 tensorflow-2.14.0rc1 tensorflow-estimator-2.14.0 tf-nightly-2.15.0.dev20230920

And CI build failed as:

AttributeError: module 'ml_dtypes' has no attribute 'float8_e4m3b11'. Did you mean: 'float8_e4m3fn'?
Traceback (most recent call last):
  File "<string>", line 1, in <module>
...
    from tensorflow.python.framework import dtypes
  File "/.venv/lib/python3.10/site-packages/tensorflow/python/framework/dtypes.py", line 39, in <module>
    _np_float8_e5m2 = pywrap_ml_dtypes.float8_e5m2()
TypeError: Unable to convert function return value to a Python type! The signature was
        () -> handle
make: *** [Makefile:28: build] Error 1

Probably the ml_dtypes 0.0.3 is a breaking change for TF.

The solution is to stick with the older version of TF.

[jakevdp]

Hi - thanks for the report. ml_dtypes. float8_e4m3b11 was deprecated in version 0.2.0, and removed in version 0.3.0 (see the CHANGELOG).

I know that tensorflow has been updated in light of this deprecation, so if you use a new enough version you should be fine. I'm unclear on what tensorflow version you're actually using, because your dependencies specify two different tensorflow versions: a tensorflow 2.13 pre-release, and tf-nightly.

[fadara01]

Hello, I'm seeing the same issue with v1.14.0-rc1. The fix was to pin ml_dtype to version 0.2.0.
Master however, is compatible with ml_dtype==0.3.0.
Is there a plan to change the ml_dtype requirement from ml_dtype >= 0.3.0 to ml_dtype ~= 0.2.0 for branches incompatible with 0.3.0?

[hawkinsp]

@fadara01 You'd need to raise that request with TensorFlow. @cantonios FYI.

[cantonios]

@fadara01 You'd need to raise that request with TensorFlow. @cantonios FYI.

Thanks, the nightly and next release will have the version pinned (see tensorflow/tensorflow#62036). It will otherwise be fixed shortly for future versions.

[hawkinsp]

Closing because there's no immediate action needed.

For the more general question of version compatibility, we should make a 1.0 release of ml_dtypes and follow semver in the future, but this issue isn't the right place to discuss that.