Getting HTTP 504 error when running wash helper

[sbanerj2]

I keep on getting the following error while trying to download some demographics information about participants. Also some of the default functionalities of the helper command fails with the same error message:
e3db.exceptions.APIError: Error during API operation: HTTP Error: 504

Code that I added:

1. commands.py:

    def example_iterate_age(self, namespace):

        gender_q_id = "f9b45d1b-e2a6-41bb-bc24-1752bfb3b225"
        hand_q_id = "3f364a9b-a6cf-49ef-b095-40d028ad9544"
        birth_q_id = 'd60dd485-af5b-4154-bf82-e5819d876564'
        income_q_id = '0cd6efe7-00e4-4788-97a3-3cfb0d524844'
        edu_q_id = '7a4cb5ba-1695-4975-bbaa-755431b94571'
        employ_q_id = 'be62e57d-bee8-41ec-b68f-63441109a390'

        query = Search(count=200, include_data=False,
                       include_all_writers=True, next_token=0).match(
                           strategy="WILDCARD",
                           plain={hand_q_id: "*"})  # Example wildcard for handedness ending in 't'
        q = QueryIterate(self.client, query)
        q.raiseIfNone()  # throw error if no results
        print("Total results: " + str(q.total_results))
        print("Has records: " + str(q.has_data))
        i = 0
        
        users = []
        print("Demographic data about the population are being written to file: demographics.txt ")

        demographics_file = open(
           'demographics.txt', 'w')
        strng_header = "Count" + " " + "Writer ID " + " " + " Record ID: " + " " + "Gender" + " " + "Hand" + " " + "Age " + " " + "Employment Status" + " " + " Income Range" + " " + "Education" + "\n"
        demographics_file.write(strng_header)
        for record in q:
            gender = record.meta.plain.get(gender_q_id, "NONE")
            hand = record.meta.plain.get(hand_q_id, "NONE")
            birth_date = record.meta.plain.get(birth_q_id, "NONE")
            income = record.meta.plain.get(income_q_id, "NONE")
            education = record.meta.plain.get(edu_q_id, "NONE")
            emply_stat = record.meta.plain.get(employ_q_id, "NONE")
            strng = str(i) + " " + str(record.meta.writer_id) + " " + str(record.meta.record_id) + " " + gender + " " + hand + " " + birth_date + " " + emply_stat + " " + income + " " + education + '\n'
            print (strng)
            if str(record.meta.record_id) not in users:
                users.append(str(record.meta.record_id))
                demographics_file.write(strng)
            i +=1
        demographics_file.close()

2. wash-helper.py

   baseline_age = "Query about age demographics in the baseline survey and iterate results."
    baseline = subparser.add_parser(
        "example-age", help=baseline_age, description=baseline_age)
    baseline.set_defaults(func=wash.example_iterate_age)
#

Commands that fail:

1. ./wash-helper.py example-baseline (command already present)
2. ./wash-helper.py example-altitude (command already present)
3. ./wash-helper.py example-age (new command added by me)

Strangely./wash-helper.py smoke works.

[Mrhea]

Hello, thank you for bringing this issue to our attention. The team is going to attempt to reproduce your error and get back to you as soon as possible.

[Mrhea]

After investigating the issue we determined that the 504 was due to a known behavior in the storage service that the commands queries. We were able to initially recreate your issue, and see subsequent success after running the commands again.

Occasional 504 timeout errors can occur if commands which query for large datasets have not been ran in some time. If a 504 error occurs, it is recommended to run the command a couple more times to see success.