Cleaning up Study templates
Closed this issue · 34 comments
I just now performed another sync from DEV to PROD, but had to leave out the content synchronization for our GL-templates and Study types due to various problems that need to be addressed/fixed:
- 1) Get rid of wrong UUIDs
- 2) Fix incoherent GL-template assignments to Study types
- 3) Fix wrongly changed Study type descriptions
- 4) Update Adv. Screening: Urban Infrastructure Study type
- 5) create a dummy Study on DEV for each active Study type to validate template configuration
1) Wrong UUIDs?
6 of our GL-templates have malformed UUIDs and exactly for those 6 the synchronization to PROD fails (content is looked up based on its UUID and I suppose these wrong looking UUIDs cannot be matched to the appropriate content on PROD)
I have no idea how something like that could even happen. @fgeyer16 believes that it might be a relic of the issue we had with the Business Rules module, which at some point stopped cloning new GL-steps out of the templates and instead started overwriting them.
2) Assignments of GL-templates to Study types is incoherent and needs to be fixed.
Each Study type should IMO only contain GL-templates that were build explicitly for that particular Study type. Sharing one and the same GL-template among e.g. Expert and Screening Studies is dangerous can lead to unwanted side-effects should we ever change that template.
Right now only Adv. Scr.: Transport Infrastructure is correct, since it contains only GL-templates that are meant only for this particular Study type. So, changing any of those templates won't affect other Study types and changing any of the other templates won't affect this Study type. For all other Study types this needs to be fixed.
AFAIK our different Expert Studies will differ only in the number of EU-GL methodology steps that they feature (Expert: General has only the first 5 EU-GL steps, while the other two Study types feature also the "AO Appraisal" step), but each step will always be shown in the same way (so e.g. Haz. Characterization will always look the same in Expert: General, Expert: Urban Infrastructure and Expert: Transport Infrastructure). In that case it would suffice to create just one set of "Expert-templates" for each EU-GL methodology step.
3) Wrong Study type descriptions
During the attempt to synchronize the Gl-templates I noticed that the Study type description for Adv. Screening: Urban Infrastructure was suddenly mentioning the RESCCUE project and Barcelona.
@ghilbrae is it possible that you changed that description by mistake and that it was actually meant for the "Adv. Screening Transport Infrastructure" Study type? And should that be the general description for the whole Study type or is it only meant for a specific Study, which was created based on this Study type? (I'm asking since the description mentions specifically Barcelona )
4) Updating the Adv. Screening: Urban Infrastructure Study type according to new Test template
@DenoBeno was working on the additional AO steps in a Test Study template (which he renamed to "[TEST] Advanced Urban Screening new"). If this is now complete, I suggest that the actual "Adv. Screening: Urban Infrastructure" Study type is now updated in accordance to this Test Study template, so that it can actually be used by ordinary users.
I will try to get all of this done by the end of the week, so that I can synchronize the remaining content (GL-templates and Study types) from DEV to PROD.
3) Wrong Study type descriptions
During the attempt to synchronize the Gl-templates I noticed that the Study type description for Adv. Screening: Urban Infrastructure was suddenly mentioning the RESCCUE project and Barcelona.
@ghilbrae is it possible that you changed that description by mistake and that it was actually meant for the "Adv. Screening Transport Infrastructure" Study type? And should that be the general description for the whole Study type or is it only meant for a specific Study, which was created based on this Study type? (I'm asking since the description mentions specifically Barcelona )
I don't think so, RESCCUE studies are related to the cities as a whole and not just transport infrastructures. I know that my colleague changed the descriptions of at least one of the RESCCUE studies, so that might be the issue here.
We are still working on these studies will they be correctly migrated to the PROD environment?
3) Wrong Study type descriptions
During the attempt to synchronize the Gl-templates I noticed that the Study type description for Adv. Screening: Urban Infrastructure was suddenly mentioning the RESCCUE project and Barcelona.
@ghilbrae is it possible that you changed that description by mistake and that it was actually meant for the "Adv. Screening Transport Infrastructure" Study type? And should that be the general description for the whole Study type or is it only meant for a specific Study, which was created based on this Study type? (I'm asking since the description mentions specifically Barcelona )I don't think so, RESCCUE studies are related to the cities as a whole and not just transport infrastructures. I know that my colleague changed the descriptions of at least one of the RESCCUE studies, so that might be the issue here.
So you think this is ok and that this Study type should have this description?
We are still working on these studies will they be correctly migrated to the PROD environment?
Study types and the GL-templates that they use will be migrated to PROD. Actual instances of these Studies (Study X with GL-step Y) are not synchronized between DEV and PROD.
@ghilbrae is it possible that you changed that description by mistake and that it was actually meant for the "Adv. Screening Transport Infrastructure" Study type? And should that be the general description for the whole Study type or is it only meant for a specific Study, which was created based on this Study type? (I'm asking since the description mentions specifically Barcelona )
I don't think so, RESCCUE studies are related to the cities as a whole and not just transport infrastructures. I know that my colleague changed the descriptions of at least one of the RESCCUE studies, so that might be the issue here.
So you think this is ok and that this Study type should have this description?
I'm sorry if I did not express myself correctly or if I misunderstood the question. I don't think that the general or default description should change in anyway or add extra content to account for RESCCUE stuff. These changes were meant to be only applied to RESCCUE studies (which are Urban studies) as they are quite specific and have nothing to do with the EU-GL. Also the people from RESCCUE wanted these studies to account for their results and to make their conclusions publicly available, hence the changes on the default descriptions.
I hope this clarifies the issue, but we can discuss this further if there are any lingering doubts.
@ghilbrae is it possible that you changed that description by mistake and that it was actually meant for the "Adv. Screening Transport Infrastructure" Study type? And should that be the general description for the whole Study type or is it only meant for a specific Study, which was created based on this Study type? (I'm asking since the description mentions specifically Barcelona )
I don't think so, RESCCUE studies are related to the cities as a whole and not just transport infrastructures. I know that my colleague changed the descriptions of at least one of the RESCCUE studies, so that might be the issue here.
So you think this is ok and that this Study type should have this description?
I'm sorry if I did not express myself correctly or if I misunderstood the question. I don't think that the general or default description should change in anyway or add extra content to account for RESCCUE stuff. These changes were meant to be only applied to RESCCUE studies (which are Urban studies) as they are quite specific and have nothing to do with the EU-GL. Also the people from RESCCUE wanted these studies to account for their results and to make their conclusions publicly available, hence the changes on the default descriptions.
I hope this clarifies the issue, but we can discuss this further if there are any lingering doubts.
Ok thanks, now it's clear. So please take that description and place it directly in a RESCCUE-specific Study on PROD (since as mentioned before, individual Studies are not migrated from DEV to PROD). I will restore the previous description of that Study type, but I don't want you to lose the texts. Let me know once this is done, so that I can restore the old description.
OK, we'll do that. Just a question. Do we have to recreate the RESCCUE data packages too or can they be migrated? They contain a lot of layers :S
OK, we'll do that. Just a question. Do we have to recreate the RESCCUE data packages too or can they be migrated? They contain a lot of layers :S
Data packages will be migrated, don't worry. They are the main reason why we even created the synchronization between DEV and PROD :)
Thanks!!!
@ghilbrae A bit of topic, but Study scenarios are not used in TM Studies right? I should fully remove this block, since it's not necessary:
That's correct, we are not using Scenarios in the TM studies.
That's correct, we are not using Scenarios in the TM studies.
Thanks, I'll take care of removing that block from TM Studies.
That's correct, we are not using Scenarios in the TM studies.
Thanks, I'll take care of removing that block from TM Studies.
Done in #185.
Is there some issue with templates synchronization or do you simply want the naming fixed in dev instance?
I'll look at the dev instance and make sure that templates are assigned correctly and named correctly. Should be done in 15 min.
Is there some issue with templates synchronization or do you simply want the naming fixed in dev instance?
I'll look at the dev instance and make sure that templates are assigned correctly and named correctly. Should be done in 15 min.
No need to, I'm already working on it.
Is there some issue with templates synchronization or do you simply want the naming fixed in dev instance?
I'll look at the dev instance and make sure that templates are assigned correctly and named correctly. Should be done in 15 min.
And yes, there is an issue with the synchronization. Probably because some of our GL-templates have weird UUIDs (see here). So that's why I'm going to re-create those Nodes.
OK, then I'll let it be.
Just one thing: I am not sure if Pascal has updated the templates or the step files used in https://csis-dev.myclimateservice.eu/study/25/view/intro or both. They should have the same fields, so it's a good idea to cross check this.
https://csis-dev.myclimateservice.eu/study/25/step/5113/view/maps is missing the possibility to choose scenario.
And the difference between "with" and "without" AOs is still really minor. Zero changes for the flood. I'll restart the calculation to see if this improves things.
Is there some issue with templates synchronization or do you simply want the naming fixed in dev instance?
I'll look at the dev instance and make sure that templates are assigned correctly and named correctly. Should be done in 15 min.And yes, there is an issue with the synchronization. Probably because some of our GL-templates have weird UUIDs (see here). So that's why I'm going to re-create those Nodes.
Generating new files based on templates is trivial. I have encountered issues with cloning, but if you make a new study you will have new files. Just a few fields get changed, but that's easily edited back in "edit as template" mode.
One more thing: deleting a study doesn't delete gl steps. Good to know.
@patrickkaleta I've copied the descriptions from DEV to PROD, so they should be safe now.
Should we keep working on DEV or do we switch to PROD? My colleague @LauraMTG is still working on both the DPs and studies for Barcelona and Bristol.
@patrickkaleta I've copied the descriptions from DEV to PROD, so they should be safe now.
Ok thanks!
Should we keep working on DEV or do we switch to PROD? My colleague @LauraMTG is still working on both the DPs and studies for Barcelona and Bristol.
Well, in general the majority of content (DPs, resources, Gl-templates, etc.) will be synchronized from DEV to PROD (this happens around once or twice per month) with only a couple of exceptions:
- actual Study instances (so e.g. Study 55 - Study: A-2 Madrid-Barcelona section Acciona Guadalajara) are not synchronized. Reason: Studies are also stored in external systems (like Emikat or the TM app) and they can't tell apart the DEV-version of a Study from its PROD-version, so syncing Studies would always override the other version of a Study. Due to those unpredictable side-effects, Studies are not synced. IMPORTANT: Study types (so e.g. "Advanced Screening: Urban Infrastructure") are synced between DEV and PROD, since they are used as blueprints for when a new Study is created.
- Showcases are not synced, so they need to be added directly in the PROD system
Everything else is synced, so you need to create it in the DEV system and it will then later be imported onto the PROD system.
Thanks for the explanation it is much clearer now.
1) Wrong UUIDs?
6 of our GL-templates have malformed UUIDs and exactly for those 6 the synchronization to PROD fails (content is looked up based on its UUID and I suppose these wrong looking UUIDs cannot be matched to the appropriate content on PROD)
I have no idea how something like that could even happen. @fgeyer16 believes that it might be a relic of the issue we had with the Business Rules module, which at some point stopped cloning new GL-steps out of the templates and instead started overwriting them.
I re-created those templates that had faulty UUIDs. The wrong onces are still available in the system, but I've unchecked the is_template checkbox, so these elements will no longer be synced onto PROD. Once it's confirmed that the new templates were created correctly, I'll remove the faulty templates for good.
2) Assignments of GL-templates to Study types is incoherent and needs to be fixed.
Each Study type should IMO only contain GL-templates that were build explicitly for that particular Study type. Sharing one and the same GL-template among e.g. Expert and Screening Studies is dangerous can lead to unwanted side-effects should we ever change that template.
Right now only Adv. Scr.: Transport Infrastructure is correct, since it contains only GL-templates that are meant only for this particular Study type. So, changing any of those templates won't affect other Study types and changing any of the other templates won't affect this Study type. For all other Study types this needs to be fixed.
Done. Each Study type with the expection of Expert Studies have their very own set of templates:
IMO the difference for the different types of Expert Studies is the configuration of available EU-GL steps, though I noticed that Expert: UI and Expert: TI share the exact same steps in the exact same order. The only differerence I found was that the description for one of them was sligthly different. Is it supposed to by like that? Do we really even need these two types of Expert Studies?
3) Wrong Study type descriptions
During the attempt to synchronize the Gl-templates I noticed that the Study type description for Adv. Screening: Urban Infrastructure was suddenly mentioning the RESCCUE project and Barcelona.
@ghilbrae is it possible that you changed that description by mistake and that it was actually meant for the "Adv. Screening Transport Infrastructure" Study type? And should that be the general description for the whole Study type or is it only meant for a specific Study, which was created based on this Study type? (I'm asking since the description mentions specifically Barcelona )
This is fixed. The description on DEV was changed by mistake. I've reverted the description back to what is was supposed to be.
4) Updating the Adv. Screening: Urban Infrastructure Study type according to new Test template
@DenoBeno was working on the additional AO steps in a Test Study template (which he renamed to "[TEST] Advanced Urban Screening new"). If this is now complete, I suggest that the actual "Adv. Screening: Urban Infrastructure" Study type is now updated in accordance to this Test Study template, so that it can actually be used by ordinary users.
Done. Though the question remains whether this new configuration is already final or whether it needs to be further improved?
I am not aware of any pending template updates. Could obviously happen in the future. Most likely, we will still improve texts.
I've synced the latest template configuration to our PROD system. The Study types are now up-to-date.
5) Create a dummy Study on DEV for each active Study type to validate template configuration
Done. Here you can view the current template configuration for each Study type (Note: it's DEV, so not all features are working. This is just to demonstrate which GL-steps and Tabs are shown in each Study type and what the default description texts are):