mysql setup for ml-metadata
CandiedCode opened this issue · 7 comments
CandiedCode commented
Hello,
I'm hoping someone can provide some guidance. I've tried to setup a quick example of connecting ml-metadata to a mysql database running locally.
I've tried on my mac as well as via docker containers incase it was a mac/mysql c connector issue. Both cases I get the same error message.
Traceback (most recent call last):
File "/code/ml_metadata_playground/metadata.py", line 44, in <module>
store = connect_mysql()
File "/code/ml_metadata_playground/metadata.py", line 17, in connect_mysql
return metadata_store.MetadataStore(connection_config)
File "/usr/local/lib/python3.9/dist-packages/ml_metadata/metadata_store/metadata_store.py", line 103, in __init__
self._metadata_store = metadata_store_serialized.CreateMetadataStore(
RuntimeError: mysql_real_connect failed: errno: , error: For my mac:
I'm using python3.9 nightly build
- "https://pypi-nightly.tensorflow.org/api/package/ml-metadata/ml_metadata-1.8.0.dev20220327-cp39-cp39-macosx_10_14_x86_64.whl#sha256=01f6d117e60a515ac46d93ccff36a2f63972e4c7a81eb35bba40e1ec61313bd3"
- have
mysql-client(replaces mysql-connector-c) via brew install andmariadb-connector-cvia brew as well
The full error via this pytest
self = <ml_metadata.metadata_store.metadata_store.MetadataStore object at 0x7f980ad70be0>
config = mysql {
host: "0.0.0.0"
port: 3306
database: "metadata"
user: "test"
password: "secret"
}
, enable_upgrade_migration = True
def __init__(self,
config: Union[proto.ConnectionConfig,
proto.MetadataStoreClientConfig],
enable_upgrade_migration: bool = False):
"""Initialize the MetadataStore.
MetadataStore can directly connect to either the metadata database or
the MLMD MetadataStore gRPC server.
Args:
config: `proto.ConnectionConfig` or `proto.MetadataStoreClientConfig`.
Configuration to connect to the database or the metadata store server.
enable_upgrade_migration: if set to True, the library upgrades the db
schema and migrates all data if it connects to an old version backend.
It is ignored when using gRPC `proto.MetadataStoreClientConfig`.
"""
self._max_num_retries = 5
if isinstance(config, proto.ConnectionConfig):
self._using_db_connection = True
migration_options = metadata_store_pb2.MigrationOptions()
migration_options.enable_upgrade_migration = enable_upgrade_migration
> self._metadata_store = metadata_store_serialized.CreateMetadataStore(
config.SerializeToString(), migration_options.SerializeToString())
E RuntimeError: mysql_real_connect failed: errno: , error:I've tried to quickly create a dummy repo to share what my images look like via docker-compose via https://github.com/CandiedCode/ml-metadata-playground/blob/mysql_test/ml_metadata_playground/metadata.py#L42
https://github.com/CandiedCode/ml-metadata-playground/blob/mysql_test/docker-compose.yaml
docker compose run meta
[+] Running 1/0
⠿ Container ml-metadata-playground-db-1 Run... 0.0s
Traceback (most recent call last):
File "/code/ml_metadata_playground/metadata.py", line 44, in <module>
store = connect_mysql()
File "/code/ml_metadata_playground/metadata.py", line 17, in connect_mysql
return metadata_store.MetadataStore(connection_config)
File "/usr/local/lib/python3.9/dist-packages/ml_metadata/metadata_store/metadata_store.py", line 103, in __init__
self._metadata_store = metadata_store_serialized.CreateMetadataStore(
RuntimeError: mysql_real_connect failed: errno: , error: Not sure if this is an issue with the nightly 3.9 build or my local MySQL c driver setup.
CandiedCode commented
created #150 to test local wheel package for python 3.9
CandiedCode commented
Full python 3.9 support wasn't part of pybindings until v2.6.0rc1
Changing bazel to use 2.6.0 for pybind11 and running docker compose run manylinux-python39
INFO: Elapsed time: 1387.665s, Critical Path: 732.63s
INFO: 1777 processes: 58 internal, 1719 processwrapper-sandbox.
INFO: Build completed successfully, 1777 total actions
INFO: Build completed successfully, 1777 total actions
+ local bazel_genfiles
+ [[ -d /build/bazel-genfiles ]]
+ bazel_genfiles=bazel-bin
+ cp -f /build/bazel-bin/ml_metadata/proto/metadata_store_pb2.py /build/ml_metadata/proto
+ cp -f /build/bazel-bin/ml_metadata/proto/metadata_store_service_pb2.py /build/ml_metadata/proto
+ cp -f /build/bazel-bin/ml_metadata/proto/metadata_store_service_pb2_grpc.py /build/ml_metadata/proto
+ cp -f /build/bazel-bin/ml_metadata/simple_types/proto/simple_types_pb2.py /build/ml_metadata/simple_types/proto
+ _is_windows
++ uname -s
++ tr A-Z a-z
+ [[ linux =~ (cygwin|mingw32|mingw64|msys)_nt* ]]
+ MLMD_EXTENSION=ml_metadata/metadata_store/pywrap/metadata_store_extension.so
+ cp -f /build/bazel-bin/ml_metadata/metadata_store/pywrap/metadata_store_extension.so /build/ml_metadata/metadata_store/pywrap/metadata_store_extension.so
+ chmod +w /build/ml_metadata/metadata_store/pywrap/metadata_store_extension.so
running build_py
copying ml_metadata/proto/metadata_store_pb2.py -> build/lib/ml_metadata/proto
copying ml_metadata/proto/metadata_store_service_pb2.py -> build/lib/ml_metadata/proto
copying ml_metadata/proto/metadata_store_service_pb2_grpc.py -> build/lib/ml_metadata/proto
copying ml_metadata/simple_types/proto/simple_types_pb2.py -> build/lib/ml_metadata/simple_types/proto
running egg_info
writing ml_metadata.egg-info/PKG-INFO
writing dependency_links to ml_metadata.egg-info/dependency_links.txt
writing namespace_packages to ml_metadata.egg-info/namespace_packages.txt
writing requirements to ml_metadata.egg-info/requires.txt
writing top-level names to ml_metadata.egg-info/top_level.txt
reading manifest file 'ml_metadata.egg-info/SOURCES.txt'
writing manifest file 'ml_metadata.egg-info/SOURCES.txt'
copying ml_metadata/metadata_store/pywrap/metadata_store_extension.so -> build/lib/ml_metadata/metadata_store/pywrap
running build_ext
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/ml_metadata
copying build/lib/ml_metadata/__init__.py -> build/bdist.linux-x86_64/wheel/ml_metadata
copying build/lib/ml_metadata/errors.py -> build/bdist.linux-x86_64/wheel/ml_metadata
creating build/bdist.linux-x86_64/wheel/ml_metadata/metadata_store
copying build/lib/ml_metadata/metadata_store/__init__.py -> build/bdist.linux-x86_64/wheel/ml_metadata/metadata_store
copying build/lib/ml_metadata/metadata_store/metadata_store.py -> build/bdist.linux-x86_64/wheel/ml_metadata/metadata_store
copying build/lib/ml_metadata/metadata_store/metadata_store_test.py -> build/bdist.linux-x86_64/wheel/ml_metadata/metadata_store
copying build/lib/ml_metadata/metadata_store/mlmd_types.py -> build/bdist.linux-x86_64/wheel/ml_metadata/metadata_store
copying build/lib/ml_metadata/metadata_store/mlmd_types_test.py -> build/bdist.linux-x86_64/wheel/ml_metadata/metadata_store
creating build/bdist.linux-x86_64/wheel/ml_metadata/metadata_store/pywrap
copying build/lib/ml_metadata/metadata_store/pywrap/__init__.py -> build/bdist.linux-x86_64/wheel/ml_metadata/metadata_store/pywrap
copying build/lib/ml_metadata/metadata_store/pywrap/metadata_store_extension.so -> build/bdist.linux-x86_64/wheel/ml_metadata/metadata_store/pywrap
copying build/lib/ml_metadata/metadata_store/types.py -> build/bdist.linux-x86_64/wheel/ml_metadata/metadata_store
copying build/lib/ml_metadata/metadata_store/types_test.py -> build/bdist.linux-x86_64/wheel/ml_metadata/metadata_store
creating build/bdist.linux-x86_64/wheel/ml_metadata/proto
copying build/lib/ml_metadata/proto/__init__.py -> build/bdist.linux-x86_64/wheel/ml_metadata/proto
copying build/lib/ml_metadata/proto/metadata_store_pb2.py -> build/bdist.linux-x86_64/wheel/ml_metadata/proto
copying build/lib/ml_metadata/proto/metadata_store_service_pb2.py -> build/bdist.linux-x86_64/wheel/ml_metadata/proto
copying build/lib/ml_metadata/proto/metadata_store_service_pb2_grpc.py -> build/bdist.linux-x86_64/wheel/ml_metadata/proto
creating build/bdist.linux-x86_64/wheel/ml_metadata/simple_types
copying build/lib/ml_metadata/simple_types/__init__.py -> build/bdist.linux-x86_64/wheel/ml_metadata/simple_types
creating build/bdist.linux-x86_64/wheel/ml_metadata/simple_types/proto
copying build/lib/ml_metadata/simple_types/proto/__init__.py -> build/bdist.linux-x86_64/wheel/ml_metadata/simple_types/proto
copying build/lib/ml_metadata/simple_types/proto/simple_types_pb2.py -> build/bdist.linux-x86_64/wheel/ml_metadata/simple_types/proto
copying build/lib/ml_metadata/version.py -> build/bdist.linux-x86_64/wheel/ml_metadata
running install_egg_info
Copying ml_metadata.egg-info to build/bdist.linux-x86_64/wheel/ml_metadata-1.8.0.dev0-py3.9.egg-info
running install_scripts
adding license file "LICENSE" (matched pattern "LICEN[CS]E*")
creating build/bdist.linux-x86_64/wheel/ml_metadata-1.8.0.dev0.dist-info/WHEEL
creating 'dist/ml_metadata-1.8.0.dev0-cp39-cp39-linux_x86_64.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'ml_metadata/__init__.py'
adding 'ml_metadata/errors.py'
adding 'ml_metadata/version.py'
adding 'ml_metadata/metadata_store/__init__.py'
adding 'ml_metadata/metadata_store/metadata_store.py'
adding 'ml_metadata/metadata_store/metadata_store_test.py'
adding 'ml_metadata/metadata_store/mlmd_types.py'
adding 'ml_metadata/metadata_store/mlmd_types_test.py'
adding 'ml_metadata/metadata_store/types.py'
adding 'ml_metadata/metadata_store/types_test.py'
adding 'ml_metadata/metadata_store/pywrap/__init__.py'
adding 'ml_metadata/metadata_store/pywrap/metadata_store_extension.so'
adding 'ml_metadata/proto/__init__.py'
adding 'ml_metadata/proto/metadata_store_pb2.py'
adding 'ml_metadata/proto/metadata_store_service_pb2.py'
adding 'ml_metadata/proto/metadata_store_service_pb2_grpc.py'
adding 'ml_metadata/simple_types/__init__.py'
adding 'ml_metadata/simple_types/proto/__init__.py'
adding 'ml_metadata/simple_types/proto/simple_types_pb2.py'
adding 'ml_metadata-1.8.0.dev0.dist-info/LICENSE'
adding 'ml_metadata-1.8.0.dev0.dist-info/METADATA'
adding 'ml_metadata-1.8.0.dev0.dist-info/WHEEL'
adding 'ml_metadata-1.8.0.dev0.dist-info/namespace_packages.txt'
adding 'ml_metadata-1.8.0.dev0.dist-info/top_level.txt'
adding 'ml_metadata-1.8.0.dev0.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
+ stamp_wheel
++ ls /build/dist/ml_metadata-1.8.0.dev0-cp39-cp39-linux_x86_64.whl /build/dist/ml_metadata-1.8.0.dev0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
+ WHEEL_PATH='/build/dist/ml_metadata-1.8.0.dev0-cp39-cp39-linux_x86_64.whl
/build/dist/ml_metadata-1.8.0.dev0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl'
++ dirname '/build/dist/ml_metadata-1.8.0.dev0-cp39-cp39-linux_x86_64.whl
/build/dist/ml_metadata-1.8.0.dev0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl'
+ WHEEL_DIR='/build/dist/ml_metadata-1.8.0.dev0-cp39-cp39-linux_x86_64.whl
/build/dist'
+ auditwheel repair --plat manylinux2010_x86_64 -w '/build/dist/ml_metadata-1.8.0.dev0-cp39-cp39-linux_x86_64.whl
/build/dist' '/build/dist/ml_metadata-1.8.0.dev0-cp39-cp39-linux_x86_64.whl
/build/dist/ml_metadata-1.8.0.dev0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl'
usage: auditwheel [-h] [-V] [-v] command ...
auditwheel: error: cannot access /build/dist/ml_metadata-1.8.0.dev0-cp39-cp39-linux_x86_64.whl
/build/dist/ml_metadata-1.8.0.dev0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl. No such fileCandiedCode commented
I downgraded to python 3.8 via https://github.com/CandiedCode/ml-metadata-playground/tree/mysql_test and running
docker compose build meta && docker compose run meta
still produces the same error
Traceback (most recent call last):
File "ml_metadata_playground/metadata.py", line 44, in
store = connect_mysql()
File "ml_metadata_playground/metadata.py", line 17, in connect_mysql
return metadata_store.MetadataStore(connection_config)
File "/usr/local/lib/python3.8/dist-packages/ml_metadata/metadata_store/metadata_store.py", line 103, in init
self._metadata_store = metadata_store_serialized.CreateMetadataStore(
RuntimeError: mysql_real_connect failed: errno: , error:
BrianSong commented
Are you using Azure MySQL?
Also, could you try "127.0.0.1" instead of "0.0.0.0" for your local host?
CandiedCode commented
I'm just running a mysql docker container in my same network via docker-compose.
I've tried both 0.0.0.0:3306 when running python locally and then also 'db:3306' when running inside docker compose.
def connect_mysql() -> metadata_store.MetadataStore:
connection_config = metadata_store_pb2.ConnectionConfig()
connection_config.mysql.host = '0.0.0.0' # use 0.0.0.0 when running python file locally, or 'db' when running via docker-compose
connection_config.mysql.port = 3306
connection_config.mysql.database = 'metadata'
connection_config.mysql.user = 'test'
connection_config.mysql.password = 'secret'
return metadata_store.MetadataStore(connection_config)version: "3.9"
services:
meta:
build: .
depends_on:
- db
links:
- db
db:
image: mysql:8.0.28
restart: always
environment:
MYSQL_DATABASE: 'metadata'
MYSQL_USER: 'test'
MYSQL_PASSWORD: 'secret'
MYSQL_ROOT_PASSWORD: 'password'
ports:
- '3306:3306'
expose:
- '3306'
volumes:
- my-db:/var/lib/mysql
volumes:
my-db:
libraries install in dockerfile
RUN apt-get update && apt-get install -y --no-install-recommends \
python3.8 python3.8-dev python3-pip libssl-dev libmysqlclient-dev mysql-clientvia local shell
mysql --host=0.0.0.0 --port=3306 --user=test --password=secret metadata
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 9
Server version: 8.0.28 MySQL Community Server - GPL
Copyright (c) 2000, 2022, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.inside the docker container, i use the service name since '0.0.0.0' doesn't work
# inside the meta docker container
mysql --host=db --port=3306 --user=test --password=secret metadata
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 13
Server version: 8.0.28 MySQL Community Server - GPL
Copyright (c) 2000, 2022, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.Since the error message isn't descriptive, I feel like I'm missing something in my dockerfile for the python script for running the mysql connect c library. 🤷🏽♀️ maybe.
GerardoGR commented
Hey @CandiedCode,
I had a similar issue in a local deployment of kubeflow that uses mysql:8.0.29, I think the error you are seeing:
RuntimeError: mysql_real_connect failed: errno: , error:
is due to #128: by default mysql 8 uses caching_sha2_password . To work around that in your example, you can set the the default-authentication-plugin as a CLI parameter in your docker-compose, e.g:
db:
image: mysql:8.0.28
restart: always
# Taken from docker hub: https://hub.docker.com/_/mysql/
command: --default-authentication-plugin=mysql_native_password
environment:
MYSQL_DATABASE: 'metadata'
MYSQL_USER: 'test'
MYSQL_PASSWORD: 'secret'
MYSQL_ROOT_PASSWORD: 'password'
ports:
- '3306:3306'
expose:
- '3306'
volumes:
- my-db:/var/lib/mysql CandiedCode commented
Thanks @GerardoGR
That did solve my issue.