[bug] IntegrityError being generated from write_device_metrics

[nemesifier]

I have observed a lot of errors like the following, coming from openwisp_monitoring.device.tasks.write_device_metrics:

Metric.DoesNotExist: Metric matching query does not exist.
  File "openwisp_monitoring/monitoring/base/models.py", line 154, in _get_or_create
    metric = cls.objects.get(**lookup_kwargs)
  File "django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "django/db/models/query.py", line 435, in get
    raise self.model.DoesNotExist(
UniqueViolation: duplicate key value violates unique constraint "monitoring_metric_key_field_name_content_t_bcaee2c5_uniq"
DETAIL:  Key (key, field_name, content_type_id, object_id, main_tags)=(memory, percent_used, 24, 0cf10289-013c-4c87-adf9-92ce2e8ef354, {}) already exists.

  File "django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
IntegrityError: duplicate key value violates unique constraint "monitoring_metric_key_field_name_content_t_bcaee2c5_uniq"
DETAIL:  Key (key, field_name, content_type_id, object_id, main_tags)=(memory, percent_used, 24, 0cf10289-013c-4c87-adf9-92ce2e8ef354, {}) already exists.

  File "celery/app/trace.py", line 734, in __protected_call__
    return self.run(*args, **kwargs)
  File "openwisp_monitoring/device/tasks.py", line 112, in write_device_metrics
    device_data.writer.write(data, time, current)
  File "openwisp_monitoring/device/writer.py", line 152, in write
    self._write_memory(
  File "openwisp_monitoring/device/writer.py", line 330, in _write_memory
    metric, created = Metric._get_or_create(
  File "openwisp_monitoring/monitoring/base/models.py", line 164, in _get_or_create
    metric.save()
  File "django/db/models/base.py", line 739, in save
    self.save_base(using=using, force_insert=force_insert,
  File "django/db/models/base.py", line 776, in save_base
    updated = self._save_table(
  File "django/db/models/base.py", line 881, in _save_table
    results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)
  File "django/db/models/base.py", line 919, in _do_insert
    return manager._insert(
  File "django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "django/db/models/query.py", line 1270, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "django/db/models/sql/compiler.py", line 1416, in execute_sql
    cursor.execute(sql, params)
  File "django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "django/db/backends/utils.py", line 79, in _execute
    with self.db.wrap_database_errors:
  File "django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)

We could try catching this error and try the read query again, maybe the object has already been created by another background task which ran in parallel.