jezcope/pyrefine

What do you plan to use as table model?

Opened this issue · 2 comments

Hi!

This project is great and I'd be interested to get involved. I am wondering which data model you plan to use? Something home-made maybe?

Ideally it would be fantastic if the data model was abstract (defined by a bunch of abstract classes) and people could subclass it with their own data model. But maybe it would be overkill and it is fine to impose the data model in the library. In that case, may I suggest pandas (http://pandas.pydata.org/)?

Cheers,
Antonin

@wetneb Hey, thanks for your interest! I'll be glad of any contributions.

Documenting some design decisions like this is on my todo list. In the meantime, what I've written so far expects data to be passed in as a pandas DataFrame, and as I'm pretty heavily reliant on the existing functionality pandas offers to get the project up and running quickly I don't expect that to change for a while. I like the idea of abstracting out the data model, but it would be useful to have an idea of what else one might try to plug in first.

Wonderful! I didn't notice you were using pandas, that's great.
Ideally I was thinking about an SQL backend, but that would probably require a lot of work.