Problem: es-reindex.sh fail - /usr/bin/curl: Argument list too long
mamedin opened this issue · 1 comments
mamedin commented
On Ubuntu16.04:
Index list before reindexing:
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open transfers ClkAfO1lSHKBWUP4hqd_Eg 5 1 0 0 1.2kb 1.2kb
yellow open transferfiles 5kie5zleTCiV1F5HTOmuxw 5 1 0 0 1.2kb 1.2kb
yellow open aips LTWqFAfxSNaIlBIk3z9OZQ 5 1 16 0 1.3mb 1.3mb
yellow open aipfiles ngDzBSvNRDq8iWAMkV4_0w 5 1 178 0 3.5mb 3.5mb
Please back up your Elasticsearch indices before proceeding further.
Do you want to continue? [y/N] y
Deleting aips_new...
Deleting aipfiles_new...
Deleting transfers_new...
Deleting transferfiles_new...
Creating aips_new with old mappings...
es-reindex.sh: line 39: /usr/bin/curl: Argument list too long
The curl arguments seem very long:
root@am-XXXX:/tmp# curl -s localhost:9200/aips | jq '.'"${index}"' | del(.settings.index.provided_name, .settings.index.creation_date, .settings.index.uuid, .settings.index.version)' |wc -l
7614
root@am-XXXX:/tmp# curl -s localhost:9200/aipfiles | jq '.'"${index}"' | del(.settings.index.provided_name, .settings.index.creation_date, .settings.index.uuid, .settings.index.version)' |wc -l
7591
root@am-XXXX:/tmp# curl -s localhost:9200/transfers | jq '.'"${index}"' | del(.settings.index.provided_name, .settings.index.creation_date, .settings.index.uuid, .settings.index.version)' |wc -l
78
root@am-XXXX:/tmp# curl -s localhost:9200/transferfiles | jq '.'"${index}"' | del(.settings.index.provided_name, .settings.index.creation_date, .settings.index.uuid, .settings.index.version)' |wc -l
119
mamedin commented
Using a json file fixes the issue:
# Create tmp indices with old mappings
for index in $index_list; do
echo -e "Creating ${index}_new with old mappings..."
curl -s ${es_url}/${index} | jq '.'"${index}"' | del(.settings.index.provided_name, .settings.index.creation_date, .settings.index.uuid, .settings.index.version)' > /tmp/mapping_file.json
curl -s -XPUT ${es_url}/${index}_new \
-H 'Content-Type: application/json' \
-d @/tmp/mapping_file.json > /dev/null
done
echo -e "\n"
```