dataframe.js and dataframe-min.js bug

Question

dataframe.js and dataframe-min.js bug

Closed this issue 7 years ago · 10 comments

hello Gmousse , maybe i found a bug about import data frame.js .

when i import dataframe.js , i join two data frame , performance is too slower , caught browser core dump,

when ii import dataframe-min.js , program give a error "TypeError: t expected as one of [DataFrame], not as a object | t " .

i think , maybe it is a bug .

Gmousse , thank you for a powerful js package to support DataFrame .

Answer 1 · 2017-04-26T07:43:09.000Z

Hi @wustevens,

About the slowness of the join, I will work on a new optimize version of the library (but when ?).

Can you give me the piece of code you used to import the dataframe-min.js ?
Can you give me the complete error log (and where the error is throw) ?

Thank you

Answer 2 · 2017-04-26T07:48:58.000Z

hi @Gmousse , when i import dataframe-min.js

and then ,

result = ( new dfjs.DataFrame( symbol , _.keys( symbol[ 0 ] ) ) ).join( result , [ 'exchange' , 'symbol' ] , 'inner' ) ;

result is DataFrame type data .

system give me a error !

Answer 3 · 2017-04-26T07:56:43.000Z

let result = _.reduce( indicators , function( prev , item , name ) {
// 过滤数据

			let data = _.filter( item.data , row => ( row[ name ] >= Number( item.min ) && row[ name ] <= Number( item.max ) ) );

			// 做 关联处理

			if ( data === null || data.length <= 0 ) {
				return prev;
			} else if ( prev === undefined || prev === null ) {
				return new dfjs.DataFrame( data , _.keys( data[ 0 ] ) );
			} else {
				return prev.join( ( new dfjs.DataFrame( data , _.keys( data[ 0 ] ) ) ) ,
				                  [ 'exchange' , 'symbol' , 'annual_date' , 'annual_type' , 'trade_date' ] ,
				                  'inner' );
			}
		} , null );

Answer 4 · 2017-04-26T07:57:44.000Z

@wustevens I will investigate in few hours.
If the bug doesn't appear with the non-minify version, it's maybe due to a webpack missconfiguration.

thanks for this report

Answer 5 · 2017-04-26T08:08:32.000Z

@Gmousse , thanks for your work .

Answer 6 · 2017-04-26T08:51:11.000Z

hi , @Gmousse

when i import dataframe.js , non-min package , join operate performance is slow .
i join left df ( rows is 3 ) and right df ( rows is 205 ), but no return df , always running .

@Gmousse , Do you think where is a problem with my program?

Here is my code :

typeof symbol
< "object"
_.first( symbol , 2 )
< Array (2) = $2
0 {delist_date: "Mon, 01 Jan 1900 00:00:00 GMT", exchange: "SH", ipo_date: "Mon, 01 Jan 1900 00:00:00 GMT", notes: "主板", security_name: "浦发银行", …}
1 {delist_date: "Mon, 01 Jan 1900 00:00:00 GMT", exchange: "SH", ipo_date: "Mon, 01 Jan 1900 00:00:00 GMT", notes: "主板", security_name: "白云机场", …}

“Array”原型

symbol.length
< 1125
typeof result
< "object"
result.dim()
< [205, 9] (2)
result.show(2)
< "| annual... | annual... | compan... | currency | curren... | exchange | symbol | trade_... | 总市值/净利润 |

| Wed, 3... | 1 | B | CNY | Millons | SH | 600000 | Thu, 1... | 7.108 |
| Wed, 3... | 1 | G | CNY | Millons | SH | 600004 | Thu, 1... | 12.762 |"

( ( new dfjs.DataFrame( _.first( symbol , 3 ) , _.keys( symbol[ 0 ] ) ) ).join( result , [ 'exchange' , 'symbol' ] , 'inner' ) ).show()

Thanks you .

Answer 7 · 2017-04-26T21:19:30.000Z

@wustevens hi
Indeed, I have realized some tests about joins, indeed it's really slow.
I m little busy this time but I plan to fix this performance issue this week-end.

For the -min issue, it's probably due to the uglify plugin and the checktype module I use.
I will take a look tomorrow .

Answer 8 · 2017-04-27T03:10:17.000Z

hi @Gmousse , Looking forward to your solution , Thanks you.

Answer 9 · 2017-07-09T14:29:34.000Z

hi @wustevens.
Sorry to be late, I couldn't work on this project the last 2 months.

Yesterday I have fixed your issues and publish them on the tag 1.2.3.
The join method worked but had a strange behavior leading on performance issues.
Now it should work correctly and faster than ever.

For your issues on dataframe-min, It's related with type checking and uglification.
I have created a specific issue #23 and will work on it soon.

Answer 10 · 2017-07-09T17:20:49.000Z

@wustevens The dataframe-min.js build is repaired (effective from the 1.2.3 and available). It was a uglifyjs relative issue and I have repaired it by a custom configuration in my webpack.config.js. #23

Please, be free to use the -min bundle.