[New Feature] Development List TO DO
Opened this issue · 2 comments
peeyee commented
To do list for new Contributors
Contributions are welcome, here is a list for new contributors:
No | Plugin | Content | Priority | Difficulty | Contributer | Status | Issue/PR |
---|---|---|---|---|---|---|---|
1 | clickhouse | low | medium | ||||
2 | hdf5 | add docs about the perfect way of loading data and use cases. | medium | low | |||
3 | httpClient | support for windows | high | medium | |||
4 | kafka | support for kerberos | low | medium | |||
5 | kafka | support for windows | high | medium | |||
6 | kdb | support for using loadTextEx | medium | medium | |||
7 | kdb | support for reading compressed data of type q IPC (1), snappy (3), L4Z (4) | high | high | |||
8 | mqtt | support for arm | low | medium | |||
9 | mqtt | publish stream engine calculation results directly to the mqtt server | medium | medium | |||
10 | mysql | add docs about return type | medium | low | |||
11 | mysql | support for type blob | high | high | |||
12 | mysql | support for type decimal | high | high | |||
13 | oracle | high | high | ||||
14 | parquet | support for reading column of type repeated | high | high | |||
15 | parquet | support for windows | high | medium | |||
16 | signal | support for rfft, irfft | high | high | |||
17 | xgboost | upgrade the library of xgboost to version 1.7 | medium | medium | |||
18 | zip | support for arm | low | medium | |||
19 | zip | support for compressing folders | high | medium |
FlyingOE commented
Item No.6 kdb+'s tasks: Is it referring to direct file access (similar to loadFile
), bypassing q process (similar to loadTable
)?
peeyee commented
Yes, the way of loadTable/File is totally loading the data into DolphinDB's in-memory table. Thus comes a problem if we do not have enough memory,especially for community users.
So we want to have a method loadTextEx that directly loading data into DolphinDB's DFS table, meanwhile reduce the memory usage, for example, each batch costs 512MB.