No Azure credentials were found

[smile-II]

Error: Could not find any credentials that grant access to storage account: 'openaipublic' and container: 'neuron-explainer'
Access Failure: message=Could not access container, request=, status=404, error=ResourceNotFound, error_description=The specified resource does not exist.
RequestId:ef3dbd4b-701e-00d0-03cc-87e199000000
Time:2023-05-16T07:59:35.0263720Z, error_headers=Content-Length: 223, Content-Type: application/xml, Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0, x-ms-request-id: ef3dbd4b-701e-00d0-03cc-87e199000000, x-ms-version: 2019-02-02, x-ms-error-code: ResourceNotFound, Date: Tue, 16 May 2023 07:59:34 GMT

No Azure credentials were found. If the container is not marked as public, please do one of the following:

• Log in with 'az login', blobfile will use your default credentials to lookup your storage account key
• Set the environment variable 'AZURE_STORAGE_KEY' to your storage account key which you can find by following this guide: https://docs.microsoft.com/en-us/azure/storage/common/storage-account-keys-manage
• Create an account with 'az ad sp create-for-rbac --name ' and set the 'AZURE_APPLICATION_CREDENTIALS' environment variable to the path of the output from that command or individually set the 'AZURE_CLIENT_ID', 'AZURE_CLIENT_SECRET', and 'AZURE_TENANT_ID' environment variables

[smile-II]

The demo cannot run. Is there a more detailed configuration tutorial available?

[0xSMT]

If you want to pull individual data files, I wrote this code:

import json
import urllib.request

def load_az_json(url):
    with urllib.request.urlopen(url) as f:
        neuron_record = f.read()    

    return json.loads(neuron_record)
    
def load_neuron(layer_index, neuron_index):
    url = f"https://openaipublic.blob.core.windows.net/neuron-explainer/data/explanations/{layer_index}/{neuron_index}.jsonl"

    return load_az_json(url)

# program entry point
if __name__ == "__main__":
    import argparse
    parser = argparse.ArgumentParser(description='Load neuron explanation from Azure')
    parser.add_argument('layer_index', type=int, help='layer index')
    parser.add_argument('neuron_index', type=int, help='neuron index')
    args = parser.parse_args()

    neuron = load_neuron(args.layer_index, args.neuron_index)

    explanation = neuron["scored_explanations"][0]["explanation"]
    score       = neuron["scored_explanations"][0]["scored_simulation"]["ev_correlation_score"]

    print(f"Layer {args.layer_index}, Neuron {args.neuron_index}: [{explanation}] with score [{round(score, 2)}]")

This can be extended to other files, by the following convention: for the below paths, replace az://openaipublic/ with https://openaipublic.blob.core.windows.net/

• az://openaipublic/neuron-explainer/data/collated-activations/{layer_index}/{neuron_index}.json
• az://openaipublic/neuron-explainer/data/explanations/{layer_index}/{neuron_index}.jsonl
• az://openaipublic/neuron-explainer/data/related-neurons/weight-based/{layer_index}/{neuron_index}.json
• az://openaipublic/neuron-explainer/data/related-tokens/activation-based/{layer_index}/{neuron_index}.json

This can be extended to download all of the files if you wish by wrapping it in some loops (48 layers, 6400 neurons per layer, 0 indexed).

[williamrs-openai]

Should be fixed for load_neuron as of #15