palewire/django-calaccess-raw-data

django.db.utils.DataError on load of Cvr2SoCd

Closed this issue · 5 comments

hcharley commented

During a run of downloadcalaccessrawdata in the load step, I encountered this database error. Did I muck something up on my database setup or is it on the calaccess side?

Loading data files
 Loading CvrSoCd
 Loading Cvr2SoCd
Traceback (most recent call last):
  File "manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/downloadcalaccessrawdata.py", line 121, in handle
    self.load()
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/downloadcalaccessrawdata.py", line 251, in load
    call_command("loadcalaccessrawfile", model.__name__)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/__init__.py", line 115, in call_command
    return klass.execute(*args, **defaults)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 503, in handle
    label_output = self.handle_label(label, **options)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/loadcalaccessrawfile.py", line 22, in handle_label
    self.load(label)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/loadcalaccessrawfile.py", line 41, in load
    self.load_postgresql(model, csv_path)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/loadcalaccessrawfile.py", line 160, in load_postgresql
    self.cursor.execute(temp_insert)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/backends/utils.py", line 81, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/backends/utils.py", line 63, in execute
    return self.cursor.execute(sql)
django.db.utils.DataError: value too long for type character varying(19)
CONTEXT:  COPY temporary_table, line 37468, column TRAN_ID: "9A0E4A6D-5C20-49AE-B"

I reproduced by calling loadcalaccessrawfile directly:

10:11:21 $ python manage.py loadcalaccessrawfile Cvr2SoCd
 Loading Cvr2SoCd
Traceback (most recent call last):
  File "manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/core/management/base.py", line 503, in handle
    label_output = self.handle_label(label, **options)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/loadcalaccessrawfile.py", line 22, in handle_label
    self.load(label)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/loadcalaccessrawfile.py", line 41, in load
    self.load_postgresql(model, csv_path)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/calaccess_raw/management/commands/loadcalaccessrawfile.py", line 160, in load_postgresql
    self.cursor.execute(temp_insert)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/backends/utils.py", line 81, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/Users/cbodkin/.virtualenvs/hcbodkin.com/lib/python2.7/site-packages/django/db/backends/utils.py", line 63, in execute
    return self.cursor.execute(sql)
django.db.utils.DataError: value too long for type character varying(19)
CONTEXT:  COPY temporary_table, line 37468, column TRAN_ID: "9A0E4A6D-5C20-49AE-B"
palewire commented
palewire commented

I just ran the complete update process on the master branch of this repo and did not get an error. So there must be some difference between our two configurations. Are you using a PyPI release? If so, which version?

hcharley commented

I was using pypi release, but then I had switched to master branch because of a separate problem. I'm going to close until I can reproduce reliably using pypi version.

palewire commented

As a second measure, I just created a blank virtualenv and installed the latest PyPI release according to our official instructions. Everything loaded fine on a first run through updatecalaccessrawdata. So something else must be different between us. I am using PostgreSQL 9.5 on Ubuntu Linux 16.04 with django-calaccess-raw-data 1.4.6.

hcharley commented

Believe I just overcame this issue. Not sure exactly what was going on--probably on an out of sync tree or something screwy with my DB, but thank you @palewire for assisting.

On PostgreSQL 9.4.5 on Mac OS 10.11.6 (15G31)

screen shot 2016-12-19 at 11 10 07 am
screen shot 2016-12-19 at 11 10 14 am