Expaso/hassos-addon-timescaledb

Timescaledb version

Closed this issue · 16 comments

GeorgeBark commented

First of all, thank you very much for the hard work, I am really loving timescaledb vs influxdb for HA.

I am a bit confused though about the versions. In this page, it says TimescaleDB 2.0.1 but then in the installation guide it points to another repo (where I opened an issue before I saw this one) where the version is 1.7.4. The latter is the one that I've got installed but I would like to upgrade to TimescaleDB 2. Can I install this add-on instead of the one in the installation guide?

Thanks!

Kirbo commented

I'd also like to upgrade TimescaleDB. Actually they've recently released version 2.4.2:
https://github.com/timescale/timescaledb/releases/tag/2.4.2

Any chance this addon would also get an update, soon-ish?

Expaso commented

Hi @Kirbo , @GeorgeBark ,

I would really like to do an upgrade on this. It's about time in my opinion also.

Current add-on version is on Postgres 12, while currently Postgres 13 is out.
I've updated the codebase (dev-branch) for this already a while ago, so it's on Timescale 2.0.1 and Postgres 13.2, but this container is not released yet to the HA community. That's why @GeorgeBark sees the 2.0.1 here in the codebase.

And then I got COVID, which took me out for quite a while... 😟

But on topic:

Problem is, how do I do do a seamless in-place upgrade from Postgres 12 to 13 for you guys?
Normally, you'd dump the database, and copy it over to your new postgres version, but people could have really large databases, leading to really really long startup-times (possibly hours?).

So, should I do an upgrade to Postgres 12.8 with timescale 2.4.1? (perhaps en intermediate release?)
Or should I aim for Postgres 13.2 with timescale 2.4.1?

In the meantime, HA also released their own 'long term statistics' functionality...
Any community input would be appreciated here 🙏🏻

Kinds regards,

Hans

guirem commented

Hi @Expaso ,
Maybe a new repo/addon to avoid any conflict to current users would make it easier for you and riskless for users.
If you make it clear in project and addon description why it's depreciated people would understand I guess.

It would also allow current users wanting to migrate data to run both docker instances (aka addon) in parallel to launch a migration script (that migration script would need to be written of course).

Hope you got over the covid thing!

  • 👍1
Expaso commented

Hi @guirem ,

This would indeed be the best / most stable solution. Thank you!

I will upgrade TimeScale addon first in this repo, and create a new add-on, which will be based on PostgeSQL 13.
Problem solved!

Expaso commented

Status update: I'm currently testing the new version for the addon. It will include an auto-update to PostgrteSQL 14 and Timescale 2.x.
Please have a little bit more patience, I'm working as fast as I can!

  • 👍3
  • 🎉1
  • ❤️2
Expaso commented

Still haven't forgotten you guys!

The new add-on is finished, and I'm testing it out on different scenario's.

The hardest part is that Postgres, Timescale and pgAdmin extrentions all have different upgrade / update strategies.
Also, upgrading Postgres from 12 to 14 means that during the upgrade, I need to have both versions running at the same time!
All this (in the latest versions of Alpine) means I have to compile a lot of it by hand. What a journey! 😖.

I chose to bite the apple now, because it makes upgrading to future releases a lot easier.

Any of you like to beta-test?

weltenwort commented

Thanks for all your efforts ❤️ If I can run it concurrently with the previous addon without any data loss I'd be happy to test.

Expaso commented

@weltenwort Thanx!

I think that should be possible. I will build a full version, and then I can supply a local add-on file, which will pull the docker image for you.
The nastiest trick is to get a copy of your database into that add-on space. I can help you with that., but is requires you to have full SSH access to the host, as per: https://developers.home-assistant.io/docs/operating-system/debugging/

Are you familiar with that?

weltenwort commented

Yes, I'm pretty familiar with linux containers. Sounds like you want to test an in-place upgrade? Otherwise I'd simply do a dump/restore. 🤷 Let me know if there are any test scenarios that you are particularly interested in.

weltenwort commented

I've just skimmed the HA dev docs. Looks like I should be able to just install a local working copy of the addon if I change the slug in the config.yaml? Can you think of anything else that I need to change to avoid a collision with the already installed one?

weltenwort commented

I see you've been busy these last few hours committing stuff and building docker images. I'll just wait for you to let me know what you'd consider the best approach for testing.

Expaso commented

Hi @weltenwort ,

Yeah I'm testing a lot of stuff out.
I had to fix and patch a lot of stuff from broken or half broken packages and eco systems to make this work..

But..I'm very close to a release right now.

I need some extra confidence with the following tests:

  1. Does a fresh-install works as expected?
    Should be easy, just install addon, and give her a spin!

  2. Does an in-place upgrade work as expected?
    Could be done by creating a backup of your running timescale addon (stop it first, before taking backup!), and then manipulate the json file and folder name within the tarball to point to the local_timescale addon (see below).
    When restoring that, you trick HA in restoring your original data to the local_timescaledb addon.

Another option is to SSH (with the mentioned root-rights) into HassOS, and copy the data folder over from:

/mnt/data/supervisor/addons/data77b2833f_timescaledb/ to /mnt/data/supervisor/addons/data/local_timescaledb/

Stop original addon first!

image

If you want to try the DEV container (it's on aarch64 only atm, so Raspberry PI), you can simply unzip these files into your addon's folder.
You will then run a separate local addon:

timescaledb.zip

image

image

weltenwort commented

Makes sense, I'll give it a try this weekend 👍

weltenwort commented

Using your installation instructions I've checked the following:

  • fresh install
  • upgrade of existing install
    • timescaledb hypertables still intact
    • timescaledb compression rules still intact
    • triggers still intact
    • data still intact

I found nothing wrong with either scenario. 🎉 My database is pretty simple, though, so I wouldn't view it as a complex test case.

Is there anything else I can help you with?

Expaso commented

@weltenwort Thank you very much! I very much appreciate your test-efforts!

Both plugins currently give errors around packages and repositories not being online and such stuff.,. sigh.. the dark side of CI/CD ;)

But no worries! This release is coming your way ASAP. Thanks for the patience!

  • 🚀2
Expaso commented

Here we go! The new release it out!

Please read the release-notes carefully, and raise an issue should you have any troubles!

  • 🎉1