Validate HTTP response header

[benner]

Check for specific HTTP response header and value

There is possibility to add request headers but no validation for returned ones. Having this kind of validator I can increase our probing fidelity.

I can make pull request but registered this issue first to open discussion about best implementation to fit not our needs but for all community. By looking into code it seems that best place is to colocate optional settings into http_validator. In our case it is OK to have exact key:value match but I make it regex too.

[manugarg]

Hello @benner,

Thanks for filing this. I think response header validator (as part of http_validator) makes a lot of sense. Please feel free to send the pull request.

In our case it is OK to have exact key:value match but I make it regex too.

It probably makes sense to use regex for value match. Sometimes, there can be small variations on some header values, all of which are okay.

[benner]

Add partial (only header existence) suggestion: #553

As mention in commit: to add value validation some things should be consider. It can be array with multiple values (rare case). If so - what to do for success/fail check. Some ideas: only and only value which is specified in cfg should exists; any of values should meet specified criteria. Should we add multiple values and so on :-)

[manugarg]

@benner How about we use regex for value check?

success_header {
  name: .. 
  value_regex: ..
}

[benner]

Sure, I already tried to do this but then realized that value is array. What to do with it? I mean some values in array can match this regex some - don't. Do we need another setting for logic "and" or "or"? Ether way if we merge this I promise to do value validation in agreed way :-)

[benner]

From stack overflow (:D):

In Golang this means it's a []string array.

[benner]

In my opinion it this good solution to have header name required and header value (or value_regex) optional. And then if only header name is present - we only validate header existence and if value/regex is present - we validate values too. We just need agree to how to deal with multiple values on same header map key.

[manugarg]

Ah, good point regarding multiple possible values of an HTTP header. How about we pass the check if any value matches the regex. Similarly fail the check if any value matches "fail header" value regex?

Something like:

if fail.. {
  if header exists {
    if no value_regex {
     fail
    }
    if any value matches value_regex {
      fail
    }
  }
}

if pass.. {
  if header exists {
    if no value_regex {
     pass
    }
    if any value matches value_regex {
      pass
    }
  }
}

[benner]

Agree. I will extend PR. Also then it makes sense to have dedicated function like status code lookup :-)

[manugarg]

@benner I was wondering if you wanted to finish the PR :)

Also, curious if cloudprober working out for you?

[benner]

Sorry for delay, I'm OOO. I will continue on this next week

[benner]

Also, curious if cloudprober working out for you?

Answering this: few additional things are needed:

• other (more general) discovery mechanism (thinking about consul)
• some flag which makes time.Duration on probe non blocking for fist time (of configurable first time delay)
• prometejus surfacer metric filtering will be nice too (e.g. for just exposing cloudprober internal metrics)

After HTTP header validation I can make more PRs for these.

[manugarg]

Also, curious if cloudprober working out for you?

Answering this: few additional things are needed:

• other (more general) discovery mechanism (thinking about consul)

I filed a bug for consul support long time ago, but didn't get a chance to work on it: #316

Also, curious if you have seen "file" based targets for super general discovery mechanism:
https://github.com/google/cloudprober/blob/master/targets/file/proto/config.proto

File can be on GCS (or local filesystem) and cloudprober will re-read the file at regular interval.

• some flag which makes time.Duration on probe non blocking for fist time (of configurable first time delay)
• prometejus surfacer metric filtering will be nice too (e.g. for just exposing cloudprober internal metrics)

After HTTP header validation I can make more PRs for these.

[benner]

Yes I played and filed mini fix :-)