sentinel-hub/sentinelhub-py

[FEAT] Enable timestamp filtering from interval end towards interval start

ColinMoldenhauer opened this issue · 1 comments

ColinMoldenhauer commented

What is the problem? Please describe.

For the training of a recurrent network, we require time-series data with a certain time difference between samples. Additionally, we require the final timestamp to be at a certain timestamp t_ref. Unfortunately, sentinelhub.time_utils.filter_times() filters the specified time_interval starting from the interval start asserting the specified time_difference. Therefore, it is almost impossible to find a suitable time_interval, which will produce an observation at t_ref.

Here's the solution

filter_times should support inverting the querying direction, i.e. starting with the end of time_interval backwards in time, respecting constraints of time_difference.

A possible implementation below:

In sentinelhub.time_utils

  • added reverse parameter
  • reverse the sorted timestamps if reverse=True
  • use absolute value of timestamp difference
  • reverse filtered timestamps again in order to recover order of filtered timestamps
# an adapted version of filter_times, allowing for reverse filtering
def filter_times(timestamps: Iterable[TimeType],
                 time_difference: dt.timedelta,
                 reverse: bool = False) -> List[TimeType]:
    """Filters out timestamps within time_difference, preserving only...
        ... the oldest timestamp if reverse = False
        ... the most recent timestamp if reverse = True.

    :param timestamps: A list of timestamps.
    :param time_difference: A time difference threshold.
    :param reverse: Start filtering from the oldest (False) or most current (True) timestamp.
    :return: An ordered list of timestamps `d_1 <= d_2 <= ... <= d_n` such that `d_(i+1)-d_i > time_difference`.
    """
    timestamps = sorted(set(timestamps))
    if reverse: timestamps = timestamps[::-1]

    filtered_timestamps: List[TimeType] = []

    for current_timestamp in timestamps:
        if not filtered_timestamps or abs(current_timestamp - filtered_timestamps[-1]) > time_difference:
            filtered_timestamps.append(current_timestamp)

    if reverse: filtered_timestamps = filtered_timestamps[::-1]
    return filtered_timestamps

Related Feature Request for eolearn

Best,
Colin