Many `crud_tests.py` tests fail due to code 429 - Request rate is large

[RedCraig]

Hey there,

I have:

• a fresh checkout of azure-documentdb-python
• provisioned a brand new docdb instance (and haven't changed any settings)
• entered my credentials in tests/crud_tests.py
• added azure-document-db to the path, so tests can import and use it

Running:

python tests/crudtests.py

gives:

Ran 72 tests in 412.725s

FAILED (errors=40)

One of the errors is:

{"code":"Unauthorized","message":"The input authorization token can't serve the request. 
Please check that the expected payload is built as per the protocol, 
and check the key being used. 
Server used the following payload to sign: 
'get\nmedia\nfrunapr8tqabaaaaaaaaahm8gry=\ntue, 07 jun 2016 14:37:35 gmt\n\n'
\r\nActivityId: 24c813cf-cb23-49db-8015-4c5001c1eb8e"}

The other 39 errors are all code:429, "Request rate is large".

A second test run has 34 of 72 fail.
This time there's one:
"code":"BadRequest","message":"Cross partition query is required but disabled. Please set x-ms-documentdb-query-enablecrosspartition to true"
and the rest are: code:429, "Request rate is large".

Is this expected?
Do I have to change any of my docdb settings to account for a higher request rate?

Thanks,
Craig

[RedCraig]

Ah, I just noticed issue #30 which reports the same.

[rnagpal]

@RedCraig : 429s are expected in this case since each test does a database/collection create. I'll be going to change all these test cases to re-use the database/collection to avoid throttling and also to avoid getting billed for each collection create when running the tests.

I'm going to take a look at the unauthorized exception and the BadRequest exception and get back to you soon.

[RedCraig]

@rnagpal reducing the number of requests is a good idea. I'm new to docDB, if you reuse the same db/collection are there any consistency issues you could run into when re-using across test cases?

[rnagpal]

@RedCraig : The tests now pass as we now have the default retry policy in Python that retries every request on getting a 429. I just ran it on a production instance and all tests passed. Let me know if you still run into any issue running the tests. I couldn't repro the Unauthorized or BadRequest issue and may be that's fixed. If you see it anytime, send us the ActivityId and we will investigate.

Since all the python tests execute sequentially, I don't see any issue reusing the database/collection as long as the test cleans up the resources that it creates. Anyways, I haven't made that change yet. At least that would make the test run faster(avoids database/collection creates with every request).

[RedCraig]

@rnagpal thanks! I'll give the tests a whirl.