microsoft/vscode-data-wrangler

Datetimeindex not available for operations (filtering, sorting, etc.)

Opened this issue · 1 comments

Environment data

  • VS Code version: 1.91.1
  • Data Wrangler Extension version (available under the Extensions sidebar): 1.4.2
  • Jupyter Extension version (available under the Extensions sidebar): 2024.6.0
  • Python Extension version (available under the Extensions sidebar): 2024.10.0
  • OS (Windows | Mac | Linux distro) and version: Windows 10
  • Pandas version: 2.0.1
  • Python and/or Anaconda version: 3.11.5
  • Type of virtual environment used (N/A | venv | virtualenv | conda | ...): venv

Expected behaviour

I am currently working with timeseries with datatimeindex.
Data Wrangle don't allow me to work with datetimeindexes (statistics, filter, sort, etc.)

To do so, I have to duplicate the datetimeindex as a dedicated column.
Given the possibilities for manipulating datetimeindexes with pandas, it would be preferable to be able to work with datetimeindex with Data Wrangler.

Actual behaviour

image

Observe the difference between timestamp_index and timestamp_col.

Steps to reproduce:

import pandas as pd
import numpy as np

# Generate a time series for today's date with an hourly frequency
date_range = pd.date_range(start=pd.Timestamp('today').normalize(), periods=24, freq='H')

# Create the DataFrame with the index named "timestamp_index"
df = pd.DataFrame(index=date_range)
df.index.name = 'timestamp_index'

# Add a "Sensor" column with random values from 0 to 20.5
df['Sensor'] = np.random.uniform(0, 20.5, size=len(df))

# Add a "timestamp_col" column which is a duplicate of the index
df['timestamp_col'] = df.index

# Reorder the columns so that "timestamp_col" is the first column
df = df[['timestamp_col', 'Sensor']]

# Display the DataFrame to verify
df

It seems Data Wrangler doesn't allow filtering on the index in general, regardless of the data type. I have to do df2 = df.reset_index() to be able to filter on the index. I really hope we can get the ability to filter on the index as well.