td-agent 4
Closed this issue ยท 28 comments
Update components:
- ruby 2.7
- latest fluentd v1
- other bundled libraries, jemalloc or etc.
Drop components
- Remove lsb-core dependency from systemd based OS
New support OS:
Debian 10(support in td-agent 3)RHEL / CentOS 8 support (CentOS is not released yet)(support in td-agent3)- Ubuntu 20.04
- Provide arm 64bit version for CentOS8/Amazon Linux 2/Ubuntu 18.04/Ubuntu 20.04/Debian 10
Drop OS:
- CentOS 6 32bit
- Debian 8
- Ubuntu 12.04, 14.04
New plugin:
- No idea for now
Drop plugin:
- fluent-plugin-td-monitoring
td-agent 4 is based on new build system: fluent/fluent-package-builder#24
Update components:
- ruby 2.6
- latest fluentd v1
- other bundled libraries, jemalloc or etc.
Drop components
- Remove lsb-core dependency from systemd based OS
New support OS:
- Debian 10
- RHEL / CentOS 8 support (CentOS is not released yet)
- Provide arm 64bit version
Drop OS:
- CentOS 6 32bit
- Debian 8
- Ubuntu 12.04, 14.04
New plugin:
- No idea for now
Drop plugin:
- fluent-plugin-td-monitoring
First release date:
- September or October
Dear Masahiro-san,
We are working on new openstack kolla release where we are using td-agent package ( https://review.opendev.org/#/c/680447/6/docker/base/sources.list.debian ). Currently we are working on new buster images and we are missing buster version of td-agent . I just want to ask if you can release it or when it will be possible to release.
Thanks,
Michal Arbet
I tested buster and building package is succeeded.
We will provide deb for buster in later this week.
Hello, could you please provide apt URL ?
Dear Masahiro-san,
when the td-agent package for Debian Buster will be available?
My apologies for my impatience.
Viktor Michalek
I just uploaded 3.5.0 to buster. Please check it.
new openstack kolla release
If you need the package for your big project, we recommend to create own package for it like google-fluentd.
Note: ruby 2.6.4 is not good for fluentd. We need to use 2.6.3 or wait 2.6.5.
Thank you Masahiro-san.
Hi๏ผ i am using td-agent3 on ubuntu.
And ruby version2.4 is susceptible to vulnerabilities, which described in https://security.netapp.com/advisory/ntap-20190221-0002.
td-agent4 which has ruby version 2.6.3 is good for me. when is the td-agent4 available for us? i can't wait to get this. thanks a lot
And ruby version2.4 is susceptible to vulnerabilities,
Latest td-agent 3 uses ruby 2.4.6 so above security issue doesn't exist.
Any update? It is December now...
We have a plan to update bundled component to ruby 2.7 and fluentd v1.9.
So the release date will be delayed until the late of Jan.
I have built arm64 package for ubuntu 18.04 and amazon linux 2.
https://s3.amazonaws.com/packages.treasuredata.com/test/td-agent_3.5.1-0_arm64.deb
https://s3.amazonaws.com/packages.treasuredata.com/test/td-agent-3.5.1-0.amazon2.aarch64.rpm
If you are interesting in td-agent for Arm, please test it.
@repeatedly Thanks very much
@repeatedly any update on the release of td-agent 4?
i'm currently blocked due to a limitation of ruby 2.4.x #237
@repeatedly Thanks! I build on Debian Buster Arm64 and tested, it works.
It is March now. I doubt that it ever get released.
Looks like you are unable to manage it. 3.6.0 got Amazon Linux package for aarch64. But not for other distributions which suggests that someone built it by hand. While nowadays such things are done by CI/CD systems.
Treasure-data is part of Arm but you are unable to publish packages for Arm platforms for quite a while (I prefer not to check when you were asked to build them).
Shame.
We are now working on new build system with clearcode team: https://github.com/clear-code/td-agent-builder
This will resolve several omnibus issues: #219
td-agent 4 uses this td-agent-builder. We have a plan to release td-agent 4 RC in the latter of this month.
How work goes? CentOS 8 will not ship fluentd so you have to speed up a bit :D
https://td-agent-package-browser.herokuapp.com/test
We put td-agent 4.0.0 rc1 for Ubuntu/CentOS8. We appreciate any feedback.
We will reduce package size by removing unnecessary files.
Base branch is here: fluent/fluent-package-builder#24
RPM package look non-striped compared to Debian one - 62MB difference in /opt/td-agent/lib/ files.
Add striping as current RPM package enlarges image without any reason.
After fixed nokogiri issue for windows, we have a plan to release td-agent 4
rc1 packages for centos8/ubuntu 18.04/ubuntu 20.04 are uploaded
So when we can expect working repositories?
- centos 8
- debian buster
- ubuntu bionic
- ubuntu focal
Of course aarch64 and x86-64 ones.
where is working repositories for ubuntu/aarch64 platform? or any way to build by myself ? i've checked official repo, only amd64 .deb availablable.
Can someone close this issue as WONTFIX/KTHXBYE/INVALID?
It would summarize authors attempts to get td-agent 4 built and distributed.
OK.
Working on td-agent 4 release: fluent/fluent-package-builder#123
@repeatedly do you know when it'll be available via http://packages.treasuredata.com YUM repo?
(i'm using AMZ linux)
We don't provide amazon linux package. Use amazon linux 2.
@repeatedly i'm using amazon linux 2. i just found what the issue was.
https://toolbelt.treasuredata.com/sh/install-amazon2-td-agent3.sh
as a base url of baseurl=http://packages.treasuredata.com/3/amazon/2/\$releasever/\$basearch
i tried to update the 3 with 4 but it was erroring with 404s
it seems that the problem is the extra /2/
baseurl=http://packages.treasuredata.com/4/amazon/\$releasever/\$basearch worked ๐
[ec2-user@my-host ~]$ sudo yum update td-agent
Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
194 packages excluded due to repository priority protections
Resolving Dependencies
--> Running transaction check
---> Package td-agent.x86_64 0:3.7.1-0.amazon2 will be updated
---> Package td-agent.x86_64 0:4.0.0-1.amzn2 will be an update
--> Finished Dependency Resolution
Dependencies Resolved
============================================================================================================================================================
Package Arch Version Repository Size
============================================================================================================================================================
Updating:
td-agent x86_64 4.0.0-1.amzn2 treasuredata 13 M
Transaction Summary
============================================================================================================================================================
Upgrade 1 Package
Total download size: 13 M
Is this ok [y/d/N]: y
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
td-agent-4.0.0-1.amzn2.x86_64.rpm | 13 MB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Updating : td-agent-4.0.0-1.amzn2.x86_64 1/2
warning: /etc/td-agent/td-agent.conf created as /etc/td-agent/td-agent.conf.rpmnew
Warning: td-agent.service changed on disk. Run 'systemctl daemon-reload' to reload units.
prelink detected. Installing /etc/prelink.conf.d/td-agent-ruby.conf ...
old-prelink detected, Update /etc/prelink.conf ...
Cleanup : td-agent-3.7.1-0.amazon2.x86_64 2/2
Verifying : td-agent-4.0.0-1.amzn2.x86_64 1/2
Verifying : td-agent-3.7.1-0.amazon2.x86_64 2/2
Updated:
td-agent.x86_64 0:4.0.0-1.amzn2
Complete!
[ec2-user@my-host ~]$ sudo yum info td-agent
Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
194 packages excluded due to repository priority protections
Installed Packages
Name : td-agent
Arch : x86_64
Version : 4.0.0
Release : 1.amzn2
Size : 56 M
Repo : installed
From repo : treasuredata
Summary : The stable distribution of Fluentd
URL : https://www.treasuredata.com/
License : ASL 2.0
Description : The stable distribution of Fluentd, called td-agent.
[ec2-user@my-host ~]$