carbonalyser/Carbonalyser

Improve measure of the number of bytes by adding cost of HTTP headers.

AAABBBCCCAAAA opened this issue · 4 comments

Correct me if i'm wrong but HTTP header is not taken in account. It is important because they contains site depends data that is useful to compare sites consumption.

My point is that we could take in account HTTP header contained in the request:
for all response header get length of the header title + content.

It is not peanut. Example Cookie header (371 bytes).
This data is not held in datacenter but generated dynamically so we just have cost of transit over network. Maybe you already take this cost in account ?

Original code:

  let duration = localStorage.getItem('duration');
  duration = null === duration ? 0 : duration;

  const kWhDataCenterTotal = stats.total * kWhPerByteDataCenter;
  const GESDataCenterTotal = kWhDataCenterTotal * defaultCarbonIntensityFactorIngCO2PerKWh;

  const kWhNetworkTotal = stats.total * kWhPerByteNetwork;
  const GESNetworkTotal = kWhNetworkTotal * defaultCarbonIntensityFactorIngCO2PerKWh;

  const kWhDeviceTotal = duration * kWhPerMinuteDevice;
  const GESDeviceTotal = kWhDeviceTotal * carbonIntensityFactorIngCO2PerKWh[userLocation];

  const kWhTotal = Math.round(1000 * (kWhDataCenterTotal + kWhNetworkTotal + kWhDeviceTotal)) / 1000;
  const gCO2Total = Math.round(GESDataCenterTotal + GESNetworkTotal + GESDeviceTotal);

  const kmByCar = Math.round(1000 * gCO2Total / GESgCO2ForOneKmByCar) / 1000;
  const chargedSmartphones = Math.round(gCO2Total / GESgCO2ForOneChargedSmartphone);

So you add size of header generated by website:

  const kWhNetworkTotal = (stats.content.total + stats.headers.total) * kWhPerByteNetwork;
  const GESNetworkTotal = kWhNetworkTotal * defaultCarbonIntensityFactorIngCO2PerKWh;

What about size of underlying header they seem to be not taken in account in the compute:
20 bytes for ip
20 bytes for tcp
some bytes for TLS if thereis

  let length = 0;
  for(var a = 0; a < requestDetails.responseHeaders.length; a =a +1) {
    var obj = requestDetails.responseHeaders[a];
    length += (obj.name + ": " + obj.value).length;
  }

A first version is implemented by 04885a4.

For instance on https://www.w3.org/Icons/w3c_home here the total number of bytes for the request:
We get 9826 bytes with the old version.

With the new version we got 10891 (total) on an expected value of 10985 (2 * (20 + 20) + (headers http) + (content http).

Measure of http part size is down via devtools. We do not get as much as theorically expected because of others elements of the http header that we cannot handle (http error status response for instance).

In a more advanced version we may count headers sent by the client (at the webRequest.onSendHeaders). This amount should be attributed to sites visited also and not to clients because clients are slave of the website.

This is next part is done at 708c1a8.

For instances on https://www.w3.org/Icons/w3c_home there is two request.

we measure (bytes) 472 and 409 bytes versus 507 and 441 in devtools as previous this is due to other http protocal elements.