default dtype in Trieste

[hstojic]

this is not a very detailed description of the issue, but here it is

dtypes are hardcoded in several places but overall things are adapted to GPflow and defaults to float64, so if you stick to gpflow models things are working fine, but as soon as you try to use a float32 model (e.g. DeepEnsemble) we run into float incompatibility errors

what we ended up doing is being very careful about defining the search space and observers so that they correspond to the model at hand, I think acquisition functions mostly worked out fine, but perhaps there were some issues there as well

we could a few things at least to address it

• declare a default somewhere (perhaps akin to gpflow's default_dtype?) and then use it throughout
• make it more obvious that the user should pay attention at how dtype of search space and observers when they are created, or perhaps provide methods to switch between dtypes (imagine if you have a single problem but need to switch the dtype depending on the model)
• make dtype checks at critical points (data - models - acq functions all match) to verify that all is good to go

I'm also thinking that perhaps instead of declaring a default we could make it somehow dependent on models, adapt acq fncs to whatever the models are using? but make sure the search space / observers match the model dtype