Gmousse/dataframe-js

[BUG] Dataframe loaded from file doesn't identify missing values

Opened this issue · 0 comments

Describe the bug
When data is loaded from a file locally, DataFrame treats all values as string types (empty string) due to which "missing values" are not differentiated/tracked, due to which fillMissingValues() doesn't work as expected.

However, this works when the DataFrame is prepared on the fly (using new DataFrame(...))

To Reproduce
Steps to reproduce the behavior:

  1. Copy the content below to a file & save (for e.g. test.csv)
name,age
Adam,10
Amy,
  1. Run the code below (assuming DataFrame dependency is added):
const DataFrame = require('dataframe-js').DataFrame;

(async() => {
  const df = await DataFrame.fromCSV('test.csv');
  df.fillMissingValues(0).show();
})();

Expected behavior
Second row should have been updated with 0 as the age.

Screenshots

Desktop (please complete the following information):

  • OS: Ubuntu

Additional context
This works as expected when the data is prepared in-house, i.e.

Code:

const DataFrame = require('dataframe-js').DataFrame;

(async() => {
  const df = await new DataFrame([
    { name: 'Adam', age: 10 },
    { name: 'Amy' }
  ]);
  df.fillMissingValues(0).show();
})();

Output: