Post-processing Details

[nebgnahz]

Accuracy

Each client reports the frame number and the level (degradation in use):

I0814 22:21:59.142653  6046 awstream_operators.cc:88] Emitting 0 (frame) 187013 (size) 10 (level) with configuration 1600, 0, 20
I0814 22:21:59.176369  6051 awstream_operators.cc:88] Emitting 1 (frame) 27093 (size) 10 (level) with configuration 1600, 0, 20
I0814 22:21:59.209719  6051 awstream_operators.cc:88] Emitting 2 (frame) 35179 (size) 10 (level) with configuration 1600, 0, 20

Use grep and awk, one can extract frame number as $6 and level as $10.

<stderr.log grep  "(frame)" | awk '{print $6}' | pbcopy

Copy and paste into spreadsheet and save as CSV, latency.bw.csv.

Bandwidth and Latency

The aggregation point measures image quality, has log:

1502749321936 5022891 bytes. 90 images. 37 (mean) 34 (median) 62 (95th) 241 (global-99.9) 102473 (src_dev;global) 63 (max) 3 nodes
BYNODE: 4625054 4407676 4407676
1502749322937 4857873 bytes. 87 images. 34.6897 (mean) 33 (median) 62 (95th) 241 (global-99.9) 143534 (src_dev;global) 64 (max) 3 nodes
BYNODE: 6300141 5972834 6025304

Use grep and awk, one can extract throughput as $2 and (average) latency as $6.

<image_quality.out grep "bytes" | awk '{print $6}' | pbcopy

R

We then use R to further process the log (doing windowed average mainly).

library(zoo)

## Latency and BW, every 5 seconds
js <- read.csv("latency.bw.csv")
bw <- rollapply(js$bytes, width=5, by=5, FUN=mean, align="left")
latency <- rollapply(js$latency, width=5, by=5, FUN=mean, align="left")

## three stats
js <- as.data.frame(cbind(bw / 1000 * 8 / 3, latency))
write.csv(js, "js.latency.bw.csv", row.names=F)

## Accuracy

js <- read.csv("accuracy.csv")
## all levels (degradation accuracy), this is only an approximation
accuracies <- c(0.25962339610065, 0.299560853199498, 0.449945295404814,
                0.545181134654819, 0.663186741485058, 0.72295378259747,
                0.759557982979804, 0.810061349693251, 0.868728191553363,
                0.900673400673401, 0.922836538461538)

accuracy.raw <- accuracies[js$level]
accuracy <- rollapply(accuracy.raw, width=150, by=150, FUN=mean, align="left")
write.csv(accuracy, "js.accuracy.csv", row.names=F)