Loopback `autoupdate` is deleting all indexes from our production server. (Postgres)
thisguymartin opened this issue · 9 comments
Questions:
I am not sure why however loopback autoupdate does not seem to be reliable it seems to wipe all of our index every time it is runned. For a ORM this crucial feature is a bug and huge red flag now in my opinion.
Description/Steps to reproduce
If bug: Our servers are hosted on Heroku and can be duplicated by just running loopback autoupdate command.
Just FYI this in regards to the postgres database.
Link to reproduction sandbox
Expected result
No wiping database index everytime that that autoupdate is runned
Additional information
I accept that perhaps I am writting our models incorrrectly so here is JSON block.
{
"name": "Event",
"plural": "events",
"base": "PersistedModel",
"idInjection": true,
"options": {
"validateUpsert": true
},
"hidden": ["api_key"],
"protected": ["api_key"],
"properties": {
"account_id": {
"type": "number",
"required": true
},
"stripe_public_key": {
"type": "string",
"default": ""
},
"stripe_secret_key": {
"type": "string",
"default": ""
},
"stripe_endpoint_secret": {
"type": "string",
"default": ""
},
"stripe_verified": {
"type": "boolean",
"required": true,
"default": false
},
"api_key": {
"type": "string",
"index": true
},
"format": {
"type": "string",
"required": true
},
"duration": {
"type": "string",
"required": true
},
"progress": {
"type": "string",
"required": true,
"default": "concept"
},
"facilitator_id": {
"type": "number",
"required": true
},
"expert_id": {
"type": "number",
"required": true
},
"title": {
"type": "string",
"required": true
},
"image": {
"type": "string"
},
"description": {
"type": "string"
},
"survey": {
"type": "object"
},
"start_time": {
"type": "date"
},
"location_name": {
"type": "string"
},
"location_address": {
"type": "string"
},
"city": {
"type": "string"
},
"state": {
"type": "string"
},
"province": {
"type": "string"
},
"country": {
"type": "string",
"default": "United States"
},
"zip_code": {
"type": "string"
},
"seats_available": {
"type": "number"
},
"venue_id": {
"type": "number"
},
"ticket_price": {
"type": "number"
},
"ticket_currency": {
"type": "string",
"default": "USD"
},
"ticket_locale": {
"type": "string",
"default": "en"
},
"access_code": {
"type": "string",
"required": true
},
"archived": {
"type": "boolean",
"required": true,
"default": false
},
"name_slug": {
"type": "string",
"default": ""
},
"event_duration": {
"type": "number",
"required":true,
"default": 0
},
"agenda_days":{
"type": ["object"]
},
"website_scripts": {
"type": "string",
"default": ""
},
"payment_gateway_id": {
"type": "number"
},
"domain_id": {
"type": "number",
"required": false
},
"funnel_id":{
"type": "number"
}
},
"validations": [],
"relations": {
"venue": {
"type": "belongsTo",
"model": "Venue",
"foreignKey": "venue_id",
"options": {
"nestRemoting": true
}
},
"facilitator": {
"type": "belongsTo",
"model": "Person",
"foreignKey": "facilitator_id",
"options": {
"nestRemoting": true
}
},
"expert": {
"type": "belongsTo",
"model": "Person",
"foreignKey": "expert_id",
"options": {
"nestRemoting": true
}
},
"agenda_items": {
"type": "hasMany",
"model": "AgendaItem",
"foreignKey": "event_id",
"options": {
"nestRemoting": true
}
},
"tasks": {
"type": "hasMany",
"model": "Task",
"foreignKey": "event_id",
"options": {
"nestRemoting": true
}
},
"attendees": {
"type": "hasMany",
"model": "Attendee",
"foreignKey": "event_id",
"options": {
"nestRemoting": true
}
},
"answers": {
"type": "hasMany",
"model": "SurveyAnswer",
"foreignKey": "event_id",
"options": {
"nestRemoting": true
}
},
"account": {
"type": "belongsTo",
"model": "Account",
"foreignKey": "account_id",
"options": {
"nestRemoting": false
}
},
"payment_history": {
"type": "hasOne",
"model": "PaymentHistory",
"foreignKey": "event_id",
"options": {
"nestRemoting": true
}
},
"payment_gateway": {
"type": "hasOne",
"model": "PaymentGateway",
"foreignKey": "event_id",
"options": {
"nestRemoting": true
}
},
"domain": {
"type": "hasOne",
"model": "CustomDomain",
"foreignKey": "event_id",
"options": {
"nestRemoting": true
}
}
},
"acls": [],
"methods": {}
}
@deepakrkris, could you please take a look? Thanks.
@mpatino117 can you also upload other model json files (like venue, facilitator, etc referred in the relations) for your given example ?
@mpatino117 , indexes are getting recreated when doing an auto-update if the model.json has a separate "indexes" section as in https://loopback.io/doc/en/lb3/Model-definition-JSON-file.html#indexes, example:
"indexes": {
"title_index": {
"keys": {"title": 1},
"options": {
"unique": true
}
}
}
@mpatino117 , I also tried configuring indexes directly in the "properties" section of model.json and that works fine as well. The only caveat is that the indexes are always created with a "_idx" suffix.
From your example,
"api_key": {
"type": "string",
"index": true
}
will create an index by name "event_api_key_idx".
My inference is that if the index had been created externally with a different name, it would have been recreated as "event_api_key_idx" . Please check and get back to us.
So there are two things I have found :
- auto-update for postgres recreates indexes and works fine for both property level index configuration (index is created with suffix "_idx" ) and separate "indexes" section having all index configurations. There is no issue here.
- model discovery with ds.discoverSchemas() returns model definition without any index configurations for postgres connector. not sure if this is a known issue already.
ds.discoverSchemas('todo', {relations: true}, function(err, models) {
console.log(models);
});
@mpatino117 can you check and update if you are still facing this issue ?
@deepakrkris Here is a additional model that could help:
{
"name": "Person",
"plural": "people",
"base": "PersistedModel",
"idInjection": true,
"options": {
"validateUpsert": true
},
"hidden": [
"authcode",
"authcode_expiration"
],
"protected": [
"authcode",
"authcode_expiration"
],
"properties": {
"account_id": {
"type": "number",
"required": true
},
"person_type": {
"type": "string",
"required": true,
"default": "Facilitator"
},
"tester": {
"type": "boolean",
"required": true,
"default": "false"
},
"phone": {
"type": "string",
"required": true
},
"authcode": {
"type": "string"
},
"authcode_expiration": {
"type": "date"
},
"email": {
"type": "string",
"required": true
},
"first_name": {
"type": "string",
"required": true
},
"last_name": {
"type": "string",
"required": true
},
"title": {
"type": "string"
},
"company": {
"type": "string"
},
"photo_url": {
"type": "string"
},
"notes": {
"type": "string"
},
"preferences": {
"type": "object"
},
"bio": {
"type": "string"
},
"response": {
"type": "string",
"required": false,
"default": ""
},
"subdomain": {
"type": "string",
"required": false
},
"created_at": {
"type": "date",
"required": false
},
"updated_at": {
"type": "date",
"required": false
}
},
"validations": [],
"relations": {
"events": {
"type": "hasMany",
"model": "Event",
"foreignKey": "facilitator_id",
"options": {
"nestRemoting": true
}
},
"expert_events": {
"type": "hasMany",
"model": "Event",
"foreignKey": "expert_id",
"options": {
"nestRemoting": true
}
},
"tasks": {
"type": "hasMany",
"model": "Task",
"foreignKey": "owner_id",
"options": {
"nestRemoting": true
}
},
"facilitator_account": {
"type": "hasOne",
"model": "Account",
"foreignKey": "owner_id",
"options": {
"nestRemoting": true
}
},
"account": {
"type": "belongsTo",
"model": "Account",
"foreignKey": "account_id",
"options": {
"nestRemoting": true
}
}
},
"acls": [],
"methods": {}
}
I seem to have been able to got index created in one modal. I have to keep trying to implement across the app to call it good on my side.
Thank you for your assistance @deepakrkris
@mpatino117 from your last update, was the index created on one model after db.autoUpdate(). In that case I dont think the issue still exists.
@mpatino117 the model.json in #397 (comment) does not have any index definition.