jfilter/clean-text

doesn't work with dataframe (csv file)

Closed this issue · 1 comments

Hi,
i have a csv file with multiple columns: post_id, post_text
and im trying to clean the post_text which is a dataframe and i read it from a csv file. The problem that the clean method doesn't take all the text but it tooks some word from every line of the dataframe !
please help.
you can find a txt file (csv are not allowed here) and two screenshots, one is for real data and the other is for the output of clean()

clean
data

posts.txt

Change the first line to:
d=data['Post Message'].tolist()

This is not a problem of this library but a 'feature' of pandas. I personally also find it unintuitive.