Serialization error when importing from MongoDB
alallema opened this issue ยท 3 comments
Description
I tried to import documents with meilisync
from the MongoDB free plan: https://cloud.mongodb.com/
and got a serialization error:
TypeError: Object of type ObjectId is not JSON serializable
Data format:
I used the sample data proposed by MongoDB which looked like:
{
"_id": { "$oid": "64c91330c2a74205feef694b" },
"account_id": { "$numberInt": "278603" },
"limit": { "$numberInt": "10000" },
"products": ["Commodity", "InvestmentStock"]
}
Configuration file
debug: true
plugins:
- meilisync.plugin.Plugin
progress:
type: file
source:
type: mongo
host: mongodb+srv://cluster_IP.mongodb.net
port: 27017
username: userName
password: passWord
database: sample_analytics
meilisearch:
api_url: http://localhost:7700/
api_key: 'masterKey'
insert_size: 100
insert_interval: 10
sync:
- table: accounts
index: accounts
full: true
Screenshots & Logs:
I put my logs in a file just in case:
output_error.txt
Note:
I tried to remove full: true
by:
fields:
account_id:
limit:
products:
But I received another error that I will describe in another issue.
Hi @alallema ๐ . Did you prepare the data from Mongo to send or did Meilisync do it itself (sorry, I'm only familiar with meilisync at a very high level). The issue here is the ObjectId
needs to be converted to a string before it can be serialized to JSON. I can make some suggestions how to do that once I know where the data came from.
Hi @sanders41,
Thank you for replying ๐. No, I haven't prepared anything. I've used the samples proposed by MongoDb to populate my database, and it seems that all _id
fields are created in this format by default on MongoDb Cloud.
I've also created a collection using the movie dataset and I've got another error described here #17
fixed