aws-solutions/distributed-load-testing-on-aws

TypeError: testTaskConfigs is not iterable

nijazz opened this issue · 8 comments

nijazz commented

Describe the bug
Trying to create a new test scenario using this API call:
https://docs.aws.amazon.com/solutions/latest/distributed-load-testing-on-aws/distributed-load-testing-api.html#post-scenarios

To Reproduce
Set up infrastructure via latest CloudFormation template.
Run the code below to receive HTTP error 400 with response "TypeError: testTaskConfigs is not iterable".
We haven't run this code since 17 Oct 2022 but then all worked well.
I also noticed that testId parameter was removed from the payload. Did the logic change without us knowing?

Python client code:

def post_scenarios(self, script_name):
        """
        Start a new test
        """
        test_id = ''.join(choice(string.ascii_letters + string.digits) for i in range(10))
        test_name = datetime.now().strftime('Perf_Test_%Y-%m-%d_%H_%M_%S')

        # script name on s3 must match test id => make a copy to cater for it
        script_name = self._copy_jmeter_config(script_name, test_id)

        ramp_min = os.environ.get('PERF_TEST_RAMP_UP_MINUTES', '0')
        hold_min = os.environ.get('PERF_TEST_HOLD_FOR_MINUTES', '1')

        doc = json.dumps({
            'testId': test_id,
            'testName': test_name,
            'testTaskConfigs': script_name,
            'testDescription': 'Endpoint performance tests using Apache JMeter',
            'taskCount': int(os.environ.get('PERF_TEST_TASK_COUNT', '1')),
            'testScenario': {
                'execution': [{
                    'concurrency': int(os.environ.get('PERF_TEST_CONCURRENCY', '1')),
                    'ramp-up': f'{ramp_min}m',
                    'hold-for': f'{hold_min}m',
                    'scenario': test_name
                }],
                'scenarios': {
                    test_name: {
                        'script': script_name
                    }
                }
            },
            'testType': 'jmeter',
            'fileType': 'script'
        })

        url = urljoin(urlunparse(self.url), 'scenarios')
        response = requests.post(url, data=doc, auth=self.auth, timeout=60)
        if response.status_code != 200:
            print(f'invalid response from the API - {response.status_code}: {response.text}')

        return response.text

Expected behavior
Expected to return HTTP code 200 and a response as per documentation.

Please complete the following information about the solution:

  • [3.2.1 ] Version: [e.g. v1.1.0]
  • [ap-southeast-2] Region: [e.g. us-east-1]
  • [ No] Was the solution modified from the version published on this repository?
  • If the answer to the previous question was yes, are the changes available on GitHub?
  • [No] Have you checked your service quotas for the services this solution uses?
  • [Yes] Were there any errors in the CloudWatch Logs? Yes, but only stated the fact that request returned 400.

Screenshots

Additional context

nijazz commented

It's also woth mentioning that we ran revision v2.0.2 before. Lately we had to get the latest CloudFormation script due to the breaking changes in S3 ACL policies.

Does anyone know how to fix this or whether its being addressed in a future release?

Based on the latest release (v3.2.2), these are the supported parameters for POST requests. Please refer to the request payload example below:

{
   "testId":"J7h5cRlcqp",
   "testName":"test_1",
   "testDescription":"test_1",
   "testTaskConfigs":[
      {
         "concurrency":"5",
         "taskCount":"2",
         "region":"us-east-1"
      }
   ],
   "testScenario":{
      "execution":[
         {
            "ramp-up":"10s",
            "hold-for":"5s",
            "scenario":"test_1"
         }
      ],
      "scenarios":{
         "test_1":{
            "script":"J7h5cRlcqp.jmx"
         }
      }
   },
   "showLive":false,
   "testType":"jmeter",
   "fileType":"script",
   "regionalTaskDetails":{
      "us-east-1":{
         "vCPULimit":4000,
         "vCPUsPerTask":2,
         "vCPUsInUse":0,
         "dltTaskLimit":2000,
         "dltAvailableTasks":2000
      }
   }
}

Hi @bassemwanis,

Thanks, we will update and try again. Is there any updated documentation on the API and its payloads as the documentation here seems out of date:
https://docs.aws.amazon.com/solutions/latest/distributed-load-testing-on-aws/distributed-load-testing-on-aws.pdf

Thanks

Updated the documentation.