LibraryOfCongress/concordia

Importer issue - populate elasticsearch indices task failing with socket timeout (>10)

jkueloc opened this issue · 1 comments

jkueloc commented

concordia.tasks.populate_elasticsearch_indices

elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeoutError(HTTPSConnectionPool(host='...-prod-...): Read timed out. (read timeout=10))

Related needs....

  • May need to bump up EBS again.

  • Software Service version updated need (dev, test, stage in- progress)Update now done. prod still needed.

  • Software update is available from OpenSearch 1.1 (ES 6.8) to 1.2

  • Configuration change - Instance type:

Open Search Instance Types Update in test, stage and prod are on previous instances:
Domain is running on previous generation instances. Latest generation instance types listed here.
M4 are previous generation instance types. We recommend using a latest generation instance type for better performance at a lower cost.

jkueloc commented
  • I updated the instance types on test, stage, and prod from previous generation m4.xlarge.search or similar all to the lastest similar generation m5.xlarge.search.
  • Applied the OpenSearch Service software version to R20220323-P2 (latest)
  • I did not add to EBS storage.

These changes appear to have resolved the ReadTimeoutError(HTTPSConnectionPool(host='...-prod-...): Read timed out. (read timeout=10)) on the nightly concordia.tasks.populate_elasticsearch_indices update.