parquet.ParquetReader.openUrl gives error not valid parquet file
thomasroshin opened this issue · 0 comments
I am using parquetjs-lite to read a parquet file from a url and while testing I am encountering an issue. I have put the file on my local and verified access to the file - http://localhost:8000/stock_plans_v2.parquet. From my JS code when I call
parquet.ParquetReader.openUrl(request,'http://localhost:8000/stock_plans_v2.parquet');
It always fails with 'not valid parquet file'. But the same file has no issues when open as a local file:
reader = await parquet.ParquetReader.openFile('stock_plans_v2.parque');
Reading through some issues on the net, I thought that maybe the reason was because of schema and that writing the file from parquetjs-lite (rather than pyarrow earlier used to generate the parquet), I re-created the parquet from the parquetjs-lite associating the schema.
// declare a schema
var schema = new parquet.ParquetSchema({
StockName: { type: 'UTF8' },
CountryofListing: { type: 'UTF8' },
Ticker: { type: 'UTF8' },
MarginRate: { type: 'INT64' },
GoShort: { type: 'BOOLEAN' },
LimitedRiskPremium: { type: 'DOUBLE' },
});
// create new ParquetWriter
var writer = await parquet.ParquetWriter.openFile(schema, 'stock_plans_v3.parquet');
// append a few rows to the file
await writer.appendRow({StockName: '3i Group Plc', CountryofListing: 'UK', Ticker: 'III.L', MarginRate: 25, GoShort: true, LimitedRiskPremium: 0.7});
:
:
//close writer
await writer.close();
But reading this parquet also has the same issue. Works with parquet.ParquetReader.openFile but fails with parquet.ParquetReader.openUrl as 'not valid parquet file'.
Any pointers would be helpful.