Frequent License Checks Causing Service Disruption in n8n Multi-Main Mode
dominikmukrecki opened this issue · 10 comments
Bug Description
n8n encounters a service disruption due to frequent license renewal requests leading to a "Too many requests" error. This problem is exacerbated in environments utilizing the multi-main instance feature, potentially due to increased check frequency in such configurations. Although the license is valid, the aggressive checking disrupts service operation, demanding an alternative strategy for license verification that does not impair production environments.
https://github.com/8gears/n8n-helm-chart - deployed using this chart with some custom tweaks.
To Reproduce
- Operate n8n in an environment with multi-main instance mode enabled.
- Observe the frequency and handling of license checks during the startup and operation.
- Experience service disruption when the system encounters a "Too many requests" error during license verification.
Expected behavior
The license verification process should be robust enough to handle environments with multiple instances without exceeding API limits, ensuring continuous service operation. The system should implement a throttling mechanism or cache license status to reduce the frequency of checks against the license server.
Operating System
v1.27.11 +k3s1 (Rancher Kubernetes distro)
n8n Version
n8nio/n8n:1.38.0
Node.js Version
default for given image
Database
PostgreSQL
Execution mode
main (default)
Hey @dominikmukrecki,
How many main instances are you running?
Three pods, all operational right now.
Also, check logs:
User
13:20:55.412Z | warn | [license] license renewal failed: Too many requests. Slow down! "{ file: 'LicenseManager.js', function: 'log' }"
2024-04-18T13:20:55.418773134Z 2024-04-18T13:20:55.418Z | debug | Skipping license activation "{ file: 'BaseCommand.js', function: 'initLicense' }"
2024-04-18T13:20:55.423972440Z Error: Your license does not allow for feat:multipleMainInstances. To enable feat:multipleMainInstances, please upgrade to a license that supports this feature.
2024-04-18T13:20:55.424018157Z at Start.initOrchestration (/usr/local/lib/node_modules/n8n/dist/commands/start.js:155:19)
2024-04-18T13:20:55.424024230Z at Start.init (/usr/local/lib/node_modules/n8n/dist/commands/start.js:136:20)
2024-04-18T13:20:55.424030447Z at processTicksAndRejections (node:internal/process/task_queues:95:5)
2024-04-18T13:20:55.424036325Z at Start._run (/usr/local/lib/node_modules/n8n/node_modules/@oclif/core/lib/command.js:303:13)
2024-04-18T13:20:55.424041696Z at Config.runCommand (/usr/local/lib/node_modules/n8n/node_modules/@oclif/core/lib/config/config.js:417:25)
2024-04-18T13:20:55.424046918Z at run (/usr/local/lib/node_modules/n8n/node_modules/@oclif/core/lib/main.js:85:16)
2024-04-18T13:20:55.424052338Z at /usr/local/lib/node_modules/n8n/bin/n8n:49:2
2024-04-18T13:20:55.425755902Z 2024-04-18T13:20:55.425Z | error | Error: Exiting due to an error. "{ file: 'LoggerProxy.js', function: 'exports.error' }"
2024-04-18T13:20:55.426419621Z 2024-04-18T13:20:55.426Z | error | FeatureNotLicensedError: Your license does not allow for feat:multipleMainInstances. To enable feat:multipleMainInstances, please upgrade to a license that supports this feature. "{ file: 'LoggerProxy.js', function: 'exports.error' }"
@Joffcom My idea is - rapid license check during booting all pods at once throws too many requests error.
Proposal - provide some sort of "soft check" enterprise license, set timeout (Portainer for example gives 10 days to upgrade license after exceeding node count).
I also wrote email to my license provider directly.
After few restarts everything works, check this screenshot from Rancher please
Hey @dominikmukrecki,
Looking at that you are running 3 main instances is that correct? When you say you have emailed your license provider is that someone internally here at n8n?
We should be checking the license on startup but I am not sure how often it is actually checked I would have assumed only the once then whenever it is needed so maybe we need to relax the firewall infront of the license check to allow for the connections.
Hey @dominikmukrecki,
Who did you email here so I can check what they are doing already as there may be no need for me to look into this if someone else is already doing it and we can close this issue if it has been directly reported.
Hey @dominikmukrecki,
I have managed to catch up with JC and he has taken the issue to the team to have this looked into. As you are in direct contact about this I am going to mark this as closed for now.
@dominikmukrecki Thank you for your feedback, tracking internally: https://linear.app/n8n/issue/PAY-1517
Fix got released with n8n@1.41.0