Create fate table in upgrade code
keith-turner opened this issue · 5 comments
The fate table needs to be created in the upgrade process. It should be created exactly the same way as the init code creates the table. Ideally upgrade and init would share code.
There seems to be some existing code that creates the fate table in upgrade. However it does not seem to do everything the init code does like set some initial table properties.
The following is some code that I found that init runs, but could not find upgrade running. Possible I missed something.
Besides setting properties there seems to be a couple more things that would need to be done during upgrade that FileSystemInitializer is handling for a new system. All of it is driven by the FileSystemInitializer initialize() method:
- We need to create the new directories for the new table which is done here and here
- We need to add entries to the metadata table for the new Fate tablet which is done here
@keith-turner - I am not sure if I am missing anything else or if that is it. I am thinking we might need to do some refactoring with the initializer so that the upgrade code can also use it. Because the new Scan ref table is used in both 3.1 and elasticity maybe that should be done first in #4652 and include some refactoring, and then we can update elasticity upgrade to use it for Fate as well. It might be tricky to do both tables together in the same commit with Fate only applying to elasticity.
@cshannon The last piece you need is the table node creation in ZK.
That is handled within the ZooKeeperInitializer here.
If this isn't called then the setTableProperties method will fail because the top-level table entry in ZK doesn't exist yet.
I agree that the init code needs to be refactored so both upgrade and init can use it.