auroraresearchlab/netbox-dns

Adding record via API that would generate automatic reverse fails

mikat-gh opened this issue · 3 comments

mikat-gh commented

Hi,

While writing a script to import our existing DNS data I noticed that using API to add a record which results in automatic reverse generation fails with "500 Internal server error". Netbox log shows:

Internal Server Error: /api/plugins/netbox-dns/records/
Traceback (most recent call last):
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "netbox_dns_record_pkey"
DETAIL:  Key (id)=(773) already exists.


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/viewsets.py", line 125, in view
    return self.dispatch(request, *args, **kwargs)
  File "/opt/netbox/netbox/netbox/api/viewsets/__init__.py", line 118, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/mixins.py", line 19, in create
    self.perform_create(serializer)
  File "/opt/netbox/netbox/netbox/api/viewsets/__init__.py", line 159, in perform_create
    instance = serializer.save()
  File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 212, in save
    self.instance = self.create(validated_data)
  File "/opt/netbox/netbox/netbox/api/serializers/features.py", line 56, in create
    instance = super().create(validated_data)
  File "/opt/netbox/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 962, in create
    instance = ModelClass._default_manager.create(**validated_data)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/query.py", line 514, in create
    obj.save(force_insert=True, using=self.db)
  File "/opt/netbox/venv/lib/python3.10/site-packages/netbox_dns/models.py", line 737, in save
    super().save(*args, **kwargs)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 806, in save
    self.save_base(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 857, in save_base
    updated = self._save_table(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 1000, in _save_table
    results = self._do_insert(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 1041, in _do_insert
    return manager._insert(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/query.py", line 1434, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1621, in execute_sql
    cursor.execute(sql, params)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: duplicate key value violates unique constraint "netbox_dns_record_pkey"
DETAIL:  Key (id)=(773) already exists.

Creating identical record via GUI works and adds the reverse.

Netbox 3.3.5
Netbox-dns 0.13.1

peteeckel commented

Throwing a 500 error is definitely wrong ... that needs to be addressed.

But could you please provide the API call including data that you used to try to create the record? I occasionally create records using the API (including reverse pointer creation) and never saw that problem. Having your request so I can try to reproduce the failure would help a lot.

peteeckel commented

Update: I can reproduce it. I think I know what's causing the problem (the save() method for the record is called twice, and the second time causes the exception), but that isnt't (and shouldn't be) an issue when the record is created using the GUI, nor when the tests are calling the methods directly, just with the API.

peteeckel commented

This is interesting. I just added some logging for debugging purposes and created a new record with the GUI (successful) and then with the API (failed). The issue indeed seems to be the double super().save() call:

save() called for object test2.zone5.example.com [A]
Calling update_ptr_record() for object test2.zone5.example.com [A]
update_ptr_record() called for object test2.zone5.example.com [A]
save() called for object 116.4.0.10.in-addr.arpa [PTR]
Calling super().save() for object 116.4.0.10.in-addr.arpa [PTR]
Back from super().save() for object 116.4.0.10.in-addr.arpa [PTR], pk is 107
save() called for object 4.0.10.in-addr.arpa [SOA]
Calling super().save() for object 4.0.10.in-addr.arpa [SOA]
Back from super().save() for object 4.0.10.in-addr.arpa [SOA], pk is 15
Calling super().save() for object test2.zone5.example.com [A]
Back from super().save() for object test2.zone5.example.com [A], pk is 108
Back from update_ptr_record() for object test2.zone5.example.com [A]
Calling super().save() for object test2.zone5.example.com [A]
Back from super().save() for object test2.zone5.example.com [A], pk is 108
save() called for object zone5.example.com [SOA]
Calling super().save() for object zone5.example.com [SOA]
Back from super().save() for object zone5.example.com [SOA], pk is 5

Called via the GUI, the super().save() call for the A record from update_ptr_record() returns the new pk 108, then the second super().save() call from save() returns correctly and the record and the associated PTR record are created as expected.

save() called for object test3.zone5.example.com [A]
Calling update_ptr_record() for object test3.zone5.example.com [A]
update_ptr_record() called for object test3.zone5.example.com [A]
save() called for object 116.4.0.10.in-addr.arpa [PTR]
Calling super().save() for object 116.4.0.10.in-addr.arpa [PTR]
Back from super().save() for object 116.4.0.10.in-addr.arpa [PTR], pk is 109
save() called for object 4.0.10.in-addr.arpa [SOA]
Calling super().save() for object 4.0.10.in-addr.arpa [SOA]
Back from super().save() for object 4.0.10.in-addr.arpa [SOA], pk is 15
Calling super().save() for object test3.zone5.example.com [A]
Back from super().save() for object test3.zone5.example.com [A], pk is 110
Back from update_ptr_record() for object test3.zone5.example.com [A]
Calling super().save() for object test3.zone5.example.com [A]

Called via the API, the super().save() call for the A record from update_ptr_record() returns the new pk 110, then the second super().save() call from save() fails with the 'duplicate pkey' error.

The solution should be pretty simple: Avoid the duplicate super().save() on new records. On the other hand I would like to know why the API behaves differently from the GUI (and from calling the classes directly, which works as well - not surprising as that's what the GUI does).