"AttributeError: 'dict' object has no attribute 'type'" when running pygeometa metadata import for iso19139 schema

Question

"AttributeError: 'dict' object has no attribute 'type'" when running pygeometa metadata import for iso19139 schema

ShaneMill1 opened this issue 2 years ago · 5 comments

Hi! I am noticing an issue after updating pygeometa:

(pygeometa-test) [shane.mill@ip-205-156-8-80 PygeoMeta-Investigation-3-10-23]$ pygeometa metadata import metar-wis1-xmlformat.xml --schema iso19139
/home/shane.mill/mambaforge/envs/pygeometa-test/lib/python3.10/site-packages/owslib/iso.py:120: FutureWarning: the .identification and .serviceidentification properties will merge into .identification being a list of properties.  This is currently implemented in .identificationinfo.  Please see https://github.com/geopython/OWSLib/issues/38 for more information
  warnings.warn(
/home/shane.mill/mambaforge/envs/pygeometa-test/lib/python3.10/site-packages/owslib/iso.py:596: FutureWarning: The .keywords and .keywords2 properties will merge into the .keywords property in the future, with .keywords becoming a list of MD_Keywords instances. This is currently implemented in .keywords2. Please see https://github.com/geopython/OWSLib/issues/301 for more information
  warnings.warn(
/home/shane.mill/mambaforge/envs/pygeometa-test/lib/python3.10/site-packages/owslib/iso.py:321: FutureWarning: The .keywords_object attribute will become .keywords proper in the next release. .keywords_object is a list of ibstances of the Keyword class. See for https://github.com/geopython/OWSLib/pull/765 more details.
  warnings.warn(
Traceback (most recent call last):
  File "/home/shane.mill/mambaforge/envs/pygeometa-test/bin/pygeometa", line 33, in <module>
    sys.exit(load_entry_point('pygeometa==0.13.1', 'console_scripts', 'pygeometa')())
  File "/home/shane.mill/mambaforge/envs/pygeometa-test/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/shane.mill/mambaforge/envs/pygeometa-test/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/shane.mill/mambaforge/envs/pygeometa-test/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/shane.mill/mambaforge/envs/pygeometa-test/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/shane.mill/mambaforge/envs/pygeometa-test/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/shane.mill/mambaforge/envs/pygeometa-test/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/shane.mill/mambaforge/envs/pygeometa-test/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/shane.mill/mambaforge/envs/pygeometa-test/lib/python3.10/site-packages/pygeometa-0.13.1-py3.10.egg/pygeometa/core.py", line 491, in import_
    content = schema_object.import_(metadata_file.read())
  File "/home/shane.mill/mambaforge/envs/pygeometa-test/lib/python3.10/site-packages/pygeometa-0.13.1-py3.10.egg/pygeometa/schemas/iso19139/__init__.py", line 124, in import_
    'type': value.type,
AttributeError: 'dict' object has no attribute 'type'

I suspect that it may be this commit that introduced the error: 7398dde

After checking out the commit prior to the referenced commit, things seem to work:

(pygeometa-test) [shane.mill@ip-205-156-8-80 pygeometa]$ git checkout 716ab0e146cf2f3aa9cce59d1247dc7f5c104b6b
Note: checking out '716ab0e146cf2f3aa9cce59d1247dc7f5c104b6b'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at 716ab0e... OARec/WCMP2: add default creation date, safeguard temporal (#209)
(pygeometa-test) [shane.mill@ip-205-156-8-80 pygeometa]$ ls
build  CODE_OF_CONDUCT.md  CONTRIBUTING.md  debian  dist  docs  LICENSE.md  MANIFEST.in  pygeometa  pygeometa.egg-info  README.md  requirements-dev.txt  requirements.txt  sample-wmo-wigos.yml  sample.yml  SECURITY.md  setup.py  tests
(pygeometa-test) [shane.mill@ip-205-156-8-80 pygeometa]$ python setup.py build
(pygeometa-test) [shane.mill@ip-205-156-8-80 pygeometa]$ python setup.py install
(pygeometa-test) [shane.mill@ip-205-156-8-80 PygeoMeta-Investigation-3-10-23]$ pygeometa metadata import metar-wis1-xmlformat.xml --schema iso19139
/home/shane.mill/mambaforge/envs/pygeometa-test/lib/python3.10/site-packages/owslib/iso.py:120: FutureWarning: the .identification and .serviceidentification properties will merge into .identification being a list of properties.  This is currently implemented in .identificationinfo.  Please see https://github.com/geopython/OWSLib/issues/38 for more information
  warnings.warn(
/home/shane.mill/mambaforge/envs/pygeometa-test/lib/python3.10/site-packages/owslib/iso.py:596: FutureWarning: The .keywords and .keywords2 properties will merge into the .keywords property in the future, with .keywords becoming a list of MD_Keywords instances. This is currently implemented in .keywords2. Please see https://github.com/geopython/OWSLib/issues/301 for more information
  warnings.warn(
/home/shane.mill/mambaforge/envs/pygeometa-test/lib/python3.10/site-packages/owslib/iso.py:321: FutureWarning: The .keywords_object attribute will become .keywords proper in the next release. .keywords_object is a list of ibstances of the Keyword class. See for https://github.com/geopython/OWSLib/pull/765 more details.
  warnings.warn(
contact:
  pointOfContact:
    address: 5830 University Research Court
    administrativearea: null
    city: College Park
    country: USA
    email: ncep.pmb.dm@noaa.gov
    fax: null
    individualname: null
    organization: National Weather Service
    phone: 301-683-0756
    positionname: null
    postalcode: '20740'
    url: https://www.weather.gov
distribution: {}
identification:
  abstract: A statement for reporting surface or upper air observations that do not
    conform to an established standard. These reports are in text form.
  accessconstraints: otherRestrictions
  dates:
    publication: '2018-12-14'
    revision: '2018-12-14'
  extents:
    spatial:
    - bbox:
      - -66
      - 24
      - -124
      - 49
    temporal:
    - {}
  keywords:
    keywords-0:
      keywords:
      - Irregularly
      type: temporal
    keywords-1:
      keywords:
      - miscellanous
      - upper air
      - plain language
      - text
      type: theme
    keywords-2:
      keywords:
      - Meteorological geographical features
      type: null
    keywords-3:
      keywords:
      - GlobalExchange
      type: dataCentre
  status: null
  title: TEXT
  topiccategory:
  - climatologyMeteorologyAtmosphere
mcf:
  version: '1.0'
metadata:
  datestamp: '2018-12-14T14:33:07Z'
  hierarchylevel: dataset
  identifier: urn:x-wmo:md:int.wmo.wis::SXNE66KWAL

I've attached the test file as well
metar-wis1-xmlformat.txt

Answer 1 · 2023-04-06T11:58:33.000Z

Hi @ShaneMill1 I'm unable to reproduce; pygeometa generates the MCF accordingly. Are you using pygeometa latest master branch?

Answer 2 · 2023-04-06T12:53:03.000Z

@tomkralidis At the time of the creation of this issue, I had. I'll start fresh and attempt to reproduce the issue again today and follow up. Thanks!

Answer 3 · 2023-04-06T14:31:45.000Z

@tomkralidis I ran this on a fresh ec2 instance so I know caching or anything like that isn't coming into play.

Same issue:

(my-env) (base) pygeometa metadata import metar-wis1-xmlformat.xml --schema iso19139
/home/ec2-user/my-env/lib/python3.9/site-packages/OWSLib-0.28.1-py3.9.egg/owslib/iso.py:120: FutureWarning: the .identification and .serviceidentification properties will merge into .identification being a list of properties.  This is currently implemented in .identificationinfo.  Please see https://github.com/geopython/OWSLib/issues/38 for more information
  warnings.warn(
/home/ec2-user/my-env/lib/python3.9/site-packages/OWSLib-0.28.1-py3.9.egg/owslib/iso.py:596: FutureWarning: The .keywords and .keywords2 properties will merge into the .keywords property in the future, with .keywords becoming a list of MD_Keywords instances. This is currently implemented in .keywords2. Please see https://github.com/geopython/OWSLib/issues/301 for more information
  warnings.warn(
/home/ec2-user/my-env/lib/python3.9/site-packages/OWSLib-0.28.1-py3.9.egg/owslib/iso.py:321: FutureWarning: The .keywords_object attribute will become .keywords proper in the next release. .keywords_object is a list of ibstances of the Keyword class. See for https://github.com/geopython/OWSLib/pull/765 more details.
  warnings.warn(
Traceback (most recent call last):
  File "/home/ec2-user/my-env/bin/pygeometa", line 33, in <module>
    sys.exit(load_entry_point('pygeometa==0.13.1', 'console_scripts', 'pygeometa')())
  File "/home/ec2-user/my-env/lib/python3.9/site-packages/click-8.1.3-py3.9.egg/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/ec2-user/my-env/lib/python3.9/site-packages/click-8.1.3-py3.9.egg/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/ec2-user/my-env/lib/python3.9/site-packages/click-8.1.3-py3.9.egg/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ec2-user/my-env/lib/python3.9/site-packages/click-8.1.3-py3.9.egg/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ec2-user/my-env/lib/python3.9/site-packages/click-8.1.3-py3.9.egg/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ec2-user/my-env/lib/python3.9/site-packages/click-8.1.3-py3.9.egg/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/ec2-user/my-env/lib/python3.9/site-packages/click-8.1.3-py3.9.egg/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/ec2-user/my-env/lib/python3.9/site-packages/pygeometa-0.13.1-py3.9.egg/pygeometa/core.py", line 491, in import_
    content = schema_object.import_(metadata_file.read())
  File "/home/ec2-user/my-env/lib/python3.9/site-packages/pygeometa-0.13.1-py3.9.egg/pygeometa/schemas/iso19139/__init__.py", line 124, in import_
    'type': value.type,
AttributeError: 'dict' object has no attribute 'type'

confirming the status of the git repo (most recent commit) local to my machine:

commit ec03d4acd86a5e1a4af6a8d6b8e9f5699f85905e (HEAD -> master, origin/master, origin/HEAD)
Merge: 027e096 a77d524
Author: Tom Kralidis <tomkralidis@gmail.com>
Date:   Thu Mar 16 07:06:38 2023 -0400

    Merge pull request #215 from geopython/oaproc-providers

Environment:

(my-env) (base) uname -r
5.10.167-147.601.amzn2.x86_64
(my-env) (base) python --version
Python 3.9.12

Answer 4 · 2023-04-06T14:42:26.000Z

Good catch Shane. Reproduced. This is fixed in OWSLib master branch, and will be available on next release.

In the meantime, can you update OWSLib per below:

pip3 install https://github.com/geopython/OWSLib/archive/master.zip

...and try again?

Answer 5 · 2023-04-06T14:50:24.000Z

@tomkralidis

That solved the issue!

(my-env) (base) pygeometa metadata import metar-wis1-xmlformat.xml --schema iso19139
contact:
  pointOfContact:
    address: 5830 University Research Court
    city: College Park
    country: USA
    email: ncep.pmb.dm@noaa.gov
    organization: National Weather Service
    phone: 301-683-0756
    postalcode: '20740'
    url: https://www.weather.gov
distribution: {}
identification:
  abstract: A statement for reporting surface or upper air observations that do not
    conform to an established standard. These reports are in text form.
  accessconstraints: otherRestrictions
  dates:
    publication: '2018-12-14'
    revision: '2018-12-14'
  extents:
    spatial:
    - bbox:
      - -66
      - 24
      - -124
      - 49
    temporal:
    - {}
  keywords:
    keywords-0:
      keywords:
      - Irregularly
      type: temporal
    keywords-1:
      keywords:
      - miscellanous
      - upper air
      - plain language
      - text
      type: theme
    keywords-2:
      keywords:
      - Meteorological geographical features
      type: null
    keywords-3:
      keywords:
      - GlobalExchange
      type: dataCentre
  status: null
  title: TEXT
  topiccategory:
  - climatologyMeteorologyAtmosphere
mcf:
  version: '1.0'
metadata:
  datestamp: '2018-12-14T14:33:07Z'
  hierarchylevel: dataset
  identifier: urn:x-wmo:md:int.wmo.wis::SXNE66KWAL
  language: eng

Thanks a lot!