No Azure credentials were found
smile-II opened this issue · 3 comments
Error: Could not find any credentials that grant access to storage account: 'openaipublic' and container: 'neuron-explainer'
Access Failure: message=Could not access container, request=, status=404, error=ResourceNotFound, error_description=The specified resource does not exist.
RequestId:ef3dbd4b-701e-00d0-03cc-87e199000000
Time:2023-05-16T07:59:35.0263720Z, error_headers=Content-Length: 223, Content-Type: application/xml, Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0, x-ms-request-id: ef3dbd4b-701e-00d0-03cc-87e199000000, x-ms-version: 2019-02-02, x-ms-error-code: ResourceNotFound, Date: Tue, 16 May 2023 07:59:34 GMT
No Azure credentials were found. If the container is not marked as public, please do one of the following:
- Log in with 'az login', blobfile will use your default credentials to lookup your storage account key
- Set the environment variable 'AZURE_STORAGE_KEY' to your storage account key which you can find by following this guide: https://docs.microsoft.com/en-us/azure/storage/common/storage-account-keys-manage
- Create an account with 'az ad sp create-for-rbac --name ' and set the 'AZURE_APPLICATION_CREDENTIALS' environment variable to the path of the output from that command or individually set the 'AZURE_CLIENT_ID', 'AZURE_CLIENT_SECRET', and 'AZURE_TENANT_ID' environment variables
The demo cannot run. Is there a more detailed configuration tutorial available?
If you want to pull individual data files, I wrote this code:
import json
import urllib.request
def load_az_json(url):
with urllib.request.urlopen(url) as f:
neuron_record = f.read()
return json.loads(neuron_record)
def load_neuron(layer_index, neuron_index):
url = f"https://openaipublic.blob.core.windows.net/neuron-explainer/data/explanations/{layer_index}/{neuron_index}.jsonl"
return load_az_json(url)
# program entry point
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser(description='Load neuron explanation from Azure')
parser.add_argument('layer_index', type=int, help='layer index')
parser.add_argument('neuron_index', type=int, help='neuron index')
args = parser.parse_args()
neuron = load_neuron(args.layer_index, args.neuron_index)
explanation = neuron["scored_explanations"][0]["explanation"]
score = neuron["scored_explanations"][0]["scored_simulation"]["ev_correlation_score"]
print(f"Layer {args.layer_index}, Neuron {args.neuron_index}: [{explanation}] with score [{round(score, 2)}]")
This can be extended to other files, by the following convention: for the below paths, replace az://openaipublic/
with https://openaipublic.blob.core.windows.net/
az://openaipublic/neuron-explainer/data/collated-activations/{layer_index}/{neuron_index}.json
az://openaipublic/neuron-explainer/data/explanations/{layer_index}/{neuron_index}.jsonl
az://openaipublic/neuron-explainer/data/related-neurons/weight-based/{layer_index}/{neuron_index}.json
az://openaipublic/neuron-explainer/data/related-tokens/activation-based/{layer_index}/{neuron_index}.json
This can be extended to download all of the files if you wish by wrapping it in some loops (48 layers, 6400 neurons per layer, 0 indexed).
Should be fixed for load_neuron as of #15