Optimizely Enriched Events Export gives you secure access to your Optimizely event data so you can analyze your experiment results with greater flexibility. The export includes a useful combination of events attributes:
- Raw metadata (event names, user IDs, etc) that you pass to Optimizely, without additional processing
- Enriched metadata that Optimizely adds such as experiment IDs, variation IDs, and session IDs
For more information, see the Enriched Events data specification.
This repo contains oevents
, a simple CLI tool for loading enriched event data.
oevents
is written in bash and should run on OS X and most GNU/Linux distributions. You'll need to install the following prerequisites:
- jq
- curl (Note: All versions of OS X starting with Jaguar come with curl installed)
- the Amazon AWS CLI (v2+)
You can download a zipped directory containing the oevents
CLI from this repo or you can download just the shell script here
oevents
is a bash script. To run it, you'll need to make sure it's executable. Assuming oevents
is in your working directory, you can do this with the following command:
$ chmod +x oevents
To run oevents
, you can specify the script's path explicitly like this (again, assuming oevents
is in your working directory):
$ ./oevents help
Or you can add it to a directory in your PATH
environment variable.
$ sudo cp oevents /usr/local/bin/
$ oevents help
For usage instructions, use the help
command:
$ oevents help
Optimizely Enriched Event data is served via Amazon S3. oevents
uses the Amazon AWS CLI to interact with S3, which requires authentication. You can authenticate to AWS in three ways:
-
(Recommended) Pass your Optimizely Personal Access Token to
oevents
via theOPTIMIZELY_API_TOKEN
environment variable:$ export OPTIMIZELY_API_TOKEN=<token> $ oevents ls
-
Pass your Optimizely Personal Access Token and Optimizely Account ID via the
--token
and--account-id
command line arguments:$ oevents ls --token <token> --account-id 12345
-
Provide your AWS credentials directly. See the AWS user guide for instructions.
Note: If you chose methods 1 or 2, oevents
will acquire temporary AWS credentials using the Optimizely Authentication API. You can use the oevents auth
command to display those credentials or write them to a file. This is handy if you prefer to access Enriched Event by some other means (e.g. using the AWS CLI directly, or boto3):
$ oevents auth --token <optimizely personal access token>
export AWS_ACCESS_KEY_ID=<key id>
export AWS_SECRET_ACCESS_KEY=<secret access key>
export AWS_SESSION_TOKEN=<session token>
export AWS_SESSION_EXPIRATION=1594953226000
export S3_BASE_PATH=s3://optimizely-events-data/v1/account_id=12345/
Enriched Events are partitioned into two top-level datasets, decisions (type=decisions
) and conversions (type=events
). Each of these datasets is partitioned by date and experiment (for decisions) or event type (for conversions).
You can use oevents ls
to list all of the experiments that produced decision data on a given date:
$ oevents ls --type decisions --date 2020-05-10
PRE experiment=10676850402/
PRE experiment=14386840295/
PRE experiment=14821050982/
PRE experiment=15117030650/
PRE experiment=17517981213/
PRE experiment=17535310125/
PRE experiment=8997901009/
You can also use oevents
to list all of the event types collected on a given day:
$ oevents ls --type events --date 2020-05-10
PRE event=search_query/
PRE event=search_results_click/
PRE event=add_to_cart/
PRE event=purchase/
You can use oevents load
to download your Enriched Event data in Apache Parquet format. Command line arguments can be used to specify a progressively narrower subset of your data.
The following command will download all enriched event data associated with your Optimizely account. Data will be downloaded into the current working directory. You can configure the destination using the --output
command line argument or the OPTIMIZELY_DATA_DIR
environment variable.
$ oevents load
Use the --type
argument to specify decisions or events The following will download all decision data associated with your Optimizely account:
$ oevents load \
--type decisions
The following will download all decision data collected between July 1st and 5th, 2020 (inclusive):
$ oevents load \
--type decisions \
--start 2020-07-01 \
--end 2020-07-05
The following will download all decision data for experiment 12345
collected between July 1st and 5th, 2020:
$ oevents load \
--type decisions \
--start 2020-07-01 \
--end 2020-07-05 \
--experiment 12345
The following will download all conversion data for event add_to_cart
collected between July 1st and 5th, 2020:
$ oevents load \
--type events \
--start 2020-07-01 \
--end 2020-07-05 \
--event add_to_cart
You can run the oevents
test suite by installing BATS and running test.bats
in this directory. Note: the oevents
test suite requires bash v4.4+.
This software depends on code from the following wonderful open source projects:
Copyright (c) 2012-2020 Amazon.com, Inc. or its affiliates. License (Apache 2.0)
Copyright (c) 2012 Stephen Dolan. License (MIT)
Copyright (c) 1996 - 2020, Daniel Stenberg, daniel@haxx.se, and many contributors. License (MIT)
Copyright (c) 2014 Sam Stephenson License (MIT)