Test failures in tutorial with python3.12
sblondon opened this issue · 7 comments
sblondon commented
The tests runned with pytest works with python3.11 and not in 3.12:
How to reproduce:
git clone git@github.com:pallets/flask.git
cd flask/examples/tutorial
python3.11 -m venv venv3.11
python3.12 -m venv venv3.12
# install dependencies
# setup db
In both cases, the webservice runs properly with ./venv3.1x/bin/flask --app flaskr run --debug.
However, running the tests show different results. In venv3.11, the 24 tests are green. In venv3.12 virtualenv, there is 6 failures (and 18 green). The failures are due to a DeprecationWarning which becomes an error:
DeprecationWarning: The default timestamp converter is deprecated as of Python 3.12; see the sqlite3 documentation for suggested replacement recipes
The full output is provided at the end of the bug report.
The DeprecationWarning is documented in the 3.12 release: 'default adapters and converters are now deprecated. Instead, use the Adapter and converter recipes and tailor them to your needs.' Copy-pasting blindly the recipes in tests/conftest.py, flaskr/__init__.py and flaskr/db.py does not fix the errors.
The errors can be fixed by adding thoses lines in conftest.py:
def convert_timestamp(val):
"""Convert Unix epoch timestamp to datetime.datetime object."""
return datetime.datetime.strptime(val.decode("utf-8"), "%Y-%m-%d %H:%M:%S").replace(tzinfo=datetime.timezone.utc)
sqlite3.register_converter("timestamp", convert_timestamp)It's probably not the best fix but it's a start.
Pytest output:
(venv3.12) $ ./venv3.12/bin/pytest
========================================= test session starts ==========================================
platform linux -- Python 3.12.4, pytest-8.3.2, pluggy-1.5.0
rootdir: /home/stephane/src/flasktuto/flask/examples/tutorial
configfile: pyproject.toml
testpaths: tests
collected 24 items
tests/test_auth.py ....F... [ 33%]
tests/test_blog.py F...F...F.FF [ 83%]
tests/test_db.py .. [ 91%]
tests/test_factory.py .. [100%]
=============================================== FAILURES ===============================================
______________________________________________ test_login ______________________________________________
client = <FlaskClient <Flask 'flaskr'>>, auth = <conftest.AuthActions object at 0x7f297abc6900>
def test_login(client, auth):
# test that viewing the page renders without template errors
assert client.get("/auth/login").status_code == 200
# test that successful login redirects to the index page
response = auth.login()
assert response.headers["Location"] == "/"
# login request set the user_id in the session
# check that the user is loaded from the session
with client:
> client.get("/")
tests/test_auth.py:50:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:1162: in get
return self.open(*args, **kw)
venv3.12/lib/python3.12/site-packages/flask/testing.py:235: in open
response = super().open(
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:1116: in open
response_parts = self.run_wsgi_app(request.environ, buffered=buffered)
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:988: in run_wsgi_app
rv = run_wsgi_app(self.application, environ, buffered=buffered)
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:1264: in run_wsgi_app
app_rv = app(environ, start_response)
venv3.12/lib/python3.12/site-packages/flask/app.py:1498: in __call__
return self.wsgi_app(environ, start_response)
venv3.12/lib/python3.12/site-packages/flask/app.py:1476: in wsgi_app
response = self.handle_exception(e)
venv3.12/lib/python3.12/site-packages/flask/app.py:1473: in wsgi_app
response = self.full_dispatch_request()
venv3.12/lib/python3.12/site-packages/flask/app.py:882: in full_dispatch_request
rv = self.handle_user_exception(e)
venv3.12/lib/python3.12/site-packages/flask/app.py:880: in full_dispatch_request
rv = self.dispatch_request()
venv3.12/lib/python3.12/site-packages/flask/app.py:865: in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
flaskr/blog.py:24: in index
).fetchall()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
val = b'2018-01-01 00:00:00'
def convert_timestamp(val):
> warn(msg.format(what="timestamp converter"), DeprecationWarning, stacklevel=2)
E DeprecationWarning: The default timestamp converter is deprecated as of Python 3.12; see the sqlite3 documentation for suggested replacement recipes
/usr/lib/python3.12/sqlite3/dbapi2.py:76: DeprecationWarning
______________________________________________ test_index ______________________________________________
client = <FlaskClient <Flask 'flaskr'>>, auth = <conftest.AuthActions object at 0x7f297ad26390>
def test_index(client, auth):
> response = client.get("/")
tests/test_blog.py:7:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:1162: in get
return self.open(*args, **kw)
venv3.12/lib/python3.12/site-packages/flask/testing.py:235: in open
response = super().open(
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:1116: in open
response_parts = self.run_wsgi_app(request.environ, buffered=buffered)
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:988: in run_wsgi_app
rv = run_wsgi_app(self.application, environ, buffered=buffered)
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:1264: in run_wsgi_app
app_rv = app(environ, start_response)
venv3.12/lib/python3.12/site-packages/flask/app.py:1498: in __call__
return self.wsgi_app(environ, start_response)
venv3.12/lib/python3.12/site-packages/flask/app.py:1476: in wsgi_app
response = self.handle_exception(e)
venv3.12/lib/python3.12/site-packages/flask/app.py:1473: in wsgi_app
response = self.full_dispatch_request()
venv3.12/lib/python3.12/site-packages/flask/app.py:882: in full_dispatch_request
rv = self.handle_user_exception(e)
venv3.12/lib/python3.12/site-packages/flask/app.py:880: in full_dispatch_request
rv = self.dispatch_request()
venv3.12/lib/python3.12/site-packages/flask/app.py:865: in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
flaskr/blog.py:24: in index
).fetchall()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
val = b'2018-01-01 00:00:00'
def convert_timestamp(val):
> warn(msg.format(what="timestamp converter"), DeprecationWarning, stacklevel=2)
E DeprecationWarning: The default timestamp converter is deprecated as of Python 3.12; see the sqlite3 documentation for suggested replacement recipes
/usr/lib/python3.12/sqlite3/dbapi2.py:76: DeprecationWarning
_________________________________________ test_author_required _________________________________________
app = <Flask 'flaskr'>, client = <FlaskClient <Flask 'flaskr'>>
auth = <conftest.AuthActions object at 0x7f297ad3e210>
def test_author_required(app, client, auth):
# change the post author to another user
with app.app_context():
db = get_db()
db.execute("UPDATE post SET author_id = 2 WHERE id = 1")
db.commit()
auth.login()
# current user can't modify other user's post
> assert client.post("/1/update").status_code == 403
tests/test_blog.py:34:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:1167: in post
return self.open(*args, **kw)
venv3.12/lib/python3.12/site-packages/flask/testing.py:235: in open
response = super().open(
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:1116: in open
response_parts = self.run_wsgi_app(request.environ, buffered=buffered)
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:988: in run_wsgi_app
rv = run_wsgi_app(self.application, environ, buffered=buffered)
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:1264: in run_wsgi_app
app_rv = app(environ, start_response)
venv3.12/lib/python3.12/site-packages/flask/app.py:1498: in __call__
return self.wsgi_app(environ, start_response)
venv3.12/lib/python3.12/site-packages/flask/app.py:1476: in wsgi_app
response = self.handle_exception(e)
venv3.12/lib/python3.12/site-packages/flask/app.py:1473: in wsgi_app
response = self.full_dispatch_request()
venv3.12/lib/python3.12/site-packages/flask/app.py:882: in full_dispatch_request
rv = self.handle_user_exception(e)
venv3.12/lib/python3.12/site-packages/flask/app.py:880: in full_dispatch_request
rv = self.dispatch_request()
venv3.12/lib/python3.12/site-packages/flask/app.py:865: in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
flaskr/auth.py:27: in wrapped_view
return view(**kwargs)
flaskr/blog.py:90: in update
post = get_post(id)
flaskr/blog.py:48: in get_post
.fetchone()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
val = b'2018-01-01 00:00:00'
def convert_timestamp(val):
> warn(msg.format(what="timestamp converter"), DeprecationWarning, stacklevel=2)
E DeprecationWarning: The default timestamp converter is deprecated as of Python 3.12; see the sqlite3 documentation for suggested replacement recipes
/usr/lib/python3.12/sqlite3/dbapi2.py:76: DeprecationWarning
_____________________________________________ test_update ______________________________________________
client = <FlaskClient <Flask 'flaskr'>>, auth = <conftest.AuthActions object at 0x7f297abc7920>
app = <Flask 'flaskr'>
def test_update(client, auth, app):
auth.login()
> assert client.get("/1/update").status_code == 200
tests/test_blog.py:59:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:1162: in get
return self.open(*args, **kw)
venv3.12/lib/python3.12/site-packages/flask/testing.py:235: in open
response = super().open(
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:1116: in open
response_parts = self.run_wsgi_app(request.environ, buffered=buffered)
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:988: in run_wsgi_app
rv = run_wsgi_app(self.application, environ, buffered=buffered)
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:1264: in run_wsgi_app
app_rv = app(environ, start_response)
venv3.12/lib/python3.12/site-packages/flask/app.py:1498: in __call__
return self.wsgi_app(environ, start_response)
venv3.12/lib/python3.12/site-packages/flask/app.py:1476: in wsgi_app
response = self.handle_exception(e)
venv3.12/lib/python3.12/site-packages/flask/app.py:1473: in wsgi_app
response = self.full_dispatch_request()
venv3.12/lib/python3.12/site-packages/flask/app.py:882: in full_dispatch_request
rv = self.handle_user_exception(e)
venv3.12/lib/python3.12/site-packages/flask/app.py:880: in full_dispatch_request
rv = self.dispatch_request()
venv3.12/lib/python3.12/site-packages/flask/app.py:865: in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
flaskr/auth.py:27: in wrapped_view
return view(**kwargs)
flaskr/blog.py:90: in update
post = get_post(id)
flaskr/blog.py:48: in get_post
.fetchone()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
val = b'2018-01-01 00:00:00'
def convert_timestamp(val):
> warn(msg.format(what="timestamp converter"), DeprecationWarning, stacklevel=2)
E DeprecationWarning: The default timestamp converter is deprecated as of Python 3.12; see the sqlite3 documentation for suggested replacement recipes
/usr/lib/python3.12/sqlite3/dbapi2.py:76: DeprecationWarning
________________________________ test_create_update_validate[/1/update] ________________________________
client = <FlaskClient <Flask 'flaskr'>>, auth = <conftest.AuthActions object at 0x7f297ace5160>
path = '/1/update'
@pytest.mark.parametrize("path", ("/create", "/1/update"))
def test_create_update_validate(client, auth, path):
auth.login()
> response = client.post(path, data={"title": "", "body": ""})
tests/test_blog.py:71:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:1167: in post
return self.open(*args, **kw)
venv3.12/lib/python3.12/site-packages/flask/testing.py:235: in open
response = super().open(
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:1116: in open
response_parts = self.run_wsgi_app(request.environ, buffered=buffered)
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:988: in run_wsgi_app
rv = run_wsgi_app(self.application, environ, buffered=buffered)
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:1264: in run_wsgi_app
app_rv = app(environ, start_response)
venv3.12/lib/python3.12/site-packages/flask/app.py:1498: in __call__
return self.wsgi_app(environ, start_response)
venv3.12/lib/python3.12/site-packages/flask/app.py:1476: in wsgi_app
response = self.handle_exception(e)
venv3.12/lib/python3.12/site-packages/flask/app.py:1473: in wsgi_app
response = self.full_dispatch_request()
venv3.12/lib/python3.12/site-packages/flask/app.py:882: in full_dispatch_request
rv = self.handle_user_exception(e)
venv3.12/lib/python3.12/site-packages/flask/app.py:880: in full_dispatch_request
rv = self.dispatch_request()
venv3.12/lib/python3.12/site-packages/flask/app.py:865: in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
flaskr/auth.py:27: in wrapped_view
return view(**kwargs)
flaskr/blog.py:90: in update
post = get_post(id)
flaskr/blog.py:48: in get_post
.fetchone()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
val = b'2018-01-01 00:00:00'
def convert_timestamp(val):
> warn(msg.format(what="timestamp converter"), DeprecationWarning, stacklevel=2)
E DeprecationWarning: The default timestamp converter is deprecated as of Python 3.12; see the sqlite3 documentation for suggested replacement recipes
/usr/lib/python3.12/sqlite3/dbapi2.py:76: DeprecationWarning
_____________________________________________ test_delete ______________________________________________
client = <FlaskClient <Flask 'flaskr'>>, auth = <conftest.AuthActions object at 0x7f297ada5610>
app = <Flask 'flaskr'>
def test_delete(client, auth, app):
auth.login()
> response = client.post("/1/delete")
tests/test_blog.py:77:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:1167: in post
return self.open(*args, **kw)
venv3.12/lib/python3.12/site-packages/flask/testing.py:235: in open
response = super().open(
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:1116: in open
response_parts = self.run_wsgi_app(request.environ, buffered=buffered)
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:988: in run_wsgi_app
rv = run_wsgi_app(self.application, environ, buffered=buffered)
venv3.12/lib/python3.12/site-packages/werkzeug/test.py:1264: in run_wsgi_app
app_rv = app(environ, start_response)
venv3.12/lib/python3.12/site-packages/flask/app.py:1498: in __call__
return self.wsgi_app(environ, start_response)
venv3.12/lib/python3.12/site-packages/flask/app.py:1476: in wsgi_app
response = self.handle_exception(e)
venv3.12/lib/python3.12/site-packages/flask/app.py:1473: in wsgi_app
response = self.full_dispatch_request()
venv3.12/lib/python3.12/site-packages/flask/app.py:882: in full_dispatch_request
rv = self.handle_user_exception(e)
venv3.12/lib/python3.12/site-packages/flask/app.py:880: in full_dispatch_request
rv = self.dispatch_request()
venv3.12/lib/python3.12/site-packages/flask/app.py:865: in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
flaskr/auth.py:27: in wrapped_view
return view(**kwargs)
flaskr/blog.py:121: in delete
get_post(id)
flaskr/blog.py:48: in get_post
.fetchone()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
val = b'2018-01-01 00:00:00'
def convert_timestamp(val):
> warn(msg.format(what="timestamp converter"), DeprecationWarning, stacklevel=2)
E DeprecationWarning: The default timestamp converter is deprecated as of Python 3.12; see the sqlite3 documentation for suggested replacement recipes
/usr/lib/python3.12/sqlite3/dbapi2.py:76: DeprecationWarning
======================================= short test summary info ========================================
FAILED tests/test_auth.py::test_login - DeprecationWarning: The default timestamp converter is deprecated as of Python 3.12; see the sqlite...
FAILED tests/test_blog.py::test_index - DeprecationWarning: The default timestamp converter is deprecated as of Python 3.12; see the sqlite...
FAILED tests/test_blog.py::test_author_required - DeprecationWarning: The default timestamp converter is deprecated as of Python 3.12; see the sqlite...
FAILED tests/test_blog.py::test_update - DeprecationWarning: The default timestamp converter is deprecated as of Python 3.12; see the sqlite...
FAILED tests/test_blog.py::test_create_update_validate[/1/update] - DeprecationWarning: The default timestamp converter is deprecated as of Python 3.12; see the sqlite...
FAILED tests/test_blog.py::test_delete - DeprecationWarning: The default timestamp converter is deprecated as of Python 3.12; see the sqlite...
===================================== 6 failed, 18 passed in 8.05s =====================================
``
sblondon commented
For info, the tests with python3.11 are still green with the dirty patch.
sblondon commented
Trying to write aware datetime by converting the datetime inside the SQL request fails. For example, in blog.index() function:
posts = db.execute(
"SELECT p.id, title, body, datetime(p.created, 'localtime'), author_id, username"
" FROM post p JOIN user u ON p.author_id = u.id"
" ORDER BY datetime(p.created, 'localtime') DESC"
).fetchall()it fails because the key created in post items is replaced by the key datetime(p.created, 'localtime').
I think the right way is to use a convert function like in the first post (and in the documentation). However, it would be better to reuse the local time instead of forcing to UTC.
VMD281 commented
I see you've been working on this issue, and I’d like to contribute if there's any part I can help with. Have you made any progress, or is there a specific task I can assist with?
sblondon commented
I think the next step is to write a PR with a convert function providing the local timezone.
Feel free to do it if you want: I'm currently on another topic so, perhaps you can do it before I will be back on this issue.
pawr commented
By adding a default converter in the db.py, based on initial example and linked recipes, all tests pass in both venv3.11 venv3.12 using the following:
def convert_timestamp(val):
return datetime.datetime.fromisoformat(val.decode())
def get_db():
if "db" not in g:
sqlite3.register_converter("timestamp", convert_timestamp)
...
The convert_timestamp could apply astimezone(tz) with an optional tz=None but figured that may be out of scope for the tutorial. Otherwise, it works with fromisoformat(val.decode())
torkian commented
this small code change fix my issue :
file one change: /examples/tutorial/run_tests.py
@@ -0,0 +1,5 @@
+import unittest
+if __name__ == "__main__":
+ testsuite = unittest.TestLoader().discover('tests')
+ unittest.TextTestRunner(verbosity=1).run(testsuite)
file two change: /examples/tutorial/tests/conftest.py
@@ -1,13 +1,22 @@
import os
import tempfile
-
import pytest
+import sqlite3
+import datetime
from flaskr import create_app
from flaskr.db import get_db
from flaskr.db import init_db
+# Register a custom converter for timestamps
+def convert_timestamp(val):
+ """Convert Unix epoch timestamp to datetime.datetime object."""
+ return datetime.datetime.strptime(val.decode("utf-8"), "%Y-%m-%d %H:%M:%S").replace(tzinfo=datetime.timezone.utc)
+
+sqlite3.register_converter("timestamp", convert_timestamp)
+
# read in SQL for populating test data
+
with open(os.path.join(os.path.dirname(__file__), "data.sql"), "rb") as f:
_data_sql = f.read().decode("utf8")
sblondon commented
I suggest you to send a PR so the maintainers can check it and accept it directly if they agree with the patch.
I advise you to add a link to this issue so they can read the analysis.