Upcoming Transition to Node.js v18: Plugin Compatibility Test Required Overall Issue
ananzh opened this issue · 10 comments
Introduction
Based on the announcement of Node.js , here is a list of end of security support for different node.js package.
- Node.js 14 (LTS): Security Support Ends in 1 month on 30 Apr 2023
- Node.js 16 (LTS): Security Support Ends in 6 months on 11 Sep 2023
- Node.js 18 (LTS): Security Support Ends in 2 years on 30 Apr 2025
- Node.js 19 (latest version but no LTS): Security Support Ends in 3 months on 01 Jun 2023
As we strive to ensure the highest quality and performance for OpenSearch Dashboards, we are planning to upgrade from Node.js v14 to Node.js v18. This major version bump comes with numerous benefits including performance improvements, new features, and enhanced security. However, we understand that such changes may also impact the existing plugins built on the OpenSearch Dashboards platform.
Therefore, we kindly request all plugin owners to test the compatibility of their plugins with the new Node.js version. A branch with Node.js v18 updates has been provided for this purpose:
https://github.com/AMoo-Miki/OpenSearch-Dashboards/tree/node-18-webpack-4
Steps to proceed
- Pull the provided branch that includes the Node.js v18 upgrade and OpenSearch 3.0.0.
- Hook up your plugin(s) with the updated OpenSearch Dashboards. We have opened a list of Plugin Compatibility Test issues. See the next section.
- Run all your existing test suites and perform manual testing as necessary. If there is no problem, close the issue.
- Report any issues or incompatibilities you encounter during your tests. Any questions or issues encountered should be documented in the respective plugin's compatibility test issue and also posted in the comments of this overall issue. The purpose of adding questions and issues here is to enable visibility and transparency among all plugin owners. One plugin's issue might be a common problem affecting other plugins as well. This shared knowledge base will promote collaborative problem-solving and prevent duplication of effort.
Plugin Compatibility Test issues
- opensearch-project/anomaly-detection-dashboards-plugin#485
- opensearch-project/dashboards-observability#471
- opensearch-project/index-management-dashboards-plugin#746
- opensearch-project/dashboards-notifications#45
- opensearch-project/ml-commons-dashboards#183
- opensearch-project/dashboards-reporting#114
- opensearch-project/security-dashboards-plugin#1430
- opensearch-project/alerting-dashboards-plugin#555
- opensearch-project/dashboards-visualizations#185
- opensearch-project/dashboards-query-workbench#71
- opensearch-project/dashboards-i18n#24
- opensearch-project/dashboards-maps#411
- [ ] opensearch-project/opensearch-dashboards-sdk-js#45 - opensearch-project/dashboards-search-relevance#202
- opensearch-project/dashboards-anywhere#178
- opensearch-project/dashboards-desktop#40
- opensearch-project/dashboards-docker-images#13
- https://github.com/opensearch-project/dashboards-catalog/issues/7
Build Team Notification
Please ensure to carry out these tests and report your findings as soon as possible. This will help us identify potential issues early and mitigate them promptly, ensuring a smooth transition for the entire community and release. We understand this requires additional efforts, but your collaboration is crucial to maintain the robustness and reliability of OpenSearch Dashboards. We appreciate your support and commitment to the project.
Currently we have hooked up 5 plugins:
- anomaly-detection-dashboards-plugin
- dashboards-alerting
- dashboards-reporting
- security-dashboards-plugin
- index-management-dashboards-plugin
bootstrap and start 5 plugins
All bootstrap and start success. Plugin pages can load successfully.
ubuntu@ip-172-31-55-237:~/OpenSearch-Dashboards$ nvm use 18.15.0
Now using node v18.15.0 (npm v9.5.0)
ubuntu@ip-172-31-55-237:~/OpenSearch-Dashboards$ yarn osd bootstrap
yarn run v1.22.19
$ scripts/use_node scripts/osd bootstrap
info [opensearch-dashboards] running yarn
$ scripts/use_node ./preinstall_check
[1/4] Resolving packages...
warning Resolution field "typescript@4.0.2" is incompatible with requested version "typescript@~4.5.2"
success Already up-to-date.
info [opensearch-alerting-dashboards] running yarn
[1/4] Resolving packages...
warning Resolution field "loader-utils@1.4.2" is incompatible with requested version "loader-utils@^2.0.4"
success Already up-to-date.
info [anomaly-detection-dashboards] running yarn
[1/4] Resolving packages...
success Already up-to-date.
info [reports-dashboards] running yarn
[1/4] Resolving packages...
warning Resolution field "ansi-regex@5.0.1" is incompatible with requested version "ansi-regex@^4.1.0"
warning Resolution field "ansi-regex@5.0.1" is incompatible with requested version "ansi-regex@^2.0.0"
warning Resolution field "ansi-regex@5.0.1" is incompatible with requested version "ansi-regex@^2.0.0"
warning Resolution field "json-schema@0.4.0" is incompatible with requested version "json-schema@0.2.3"
warning Resolution field "ansi-regex@5.0.1" is incompatible with requested version "ansi-regex@^3.0.0"
success Already up-to-date.
$ node ./scripts/postinstall.js
Modified files for html2canvas Safari support:
Downloaded eng.traineddata.gz for tesseract.js
info [opensearch_index_management_dashboards] running yarn
[1/4] Resolving packages...
success Already up-to-date.
info [opensearch-security-dashboards] running yarn
[1/4] Resolving packages...
success Already up-to-date.
succ yarn.lock analysis completed without any issues
warn yarn.lock file is out of date, please run `yarn osd bootstrap` to re-enable caching
warn yarn.lock file is out of date, please run `yarn osd bootstrap` to re-enable caching
warn yarn.lock file is out of date, please run `yarn osd bootstrap` to re-enable caching
warn yarn.lock file is out of date, please run `yarn osd bootstrap` to re-enable caching
warn yarn.lock file is out of date, please run `yarn osd bootstrap` to re-enable caching
succ 24 bootstrap builds are cached
Done in 4.91s.
ubuntu@ip-172-31-55-237:~/OpenSearch-Dashboards$ yarn start
yarn run v1.22.19
$ scripts/use_node scripts/opensearch_dashboards --dev
log [20:28:08.067] [info][plugins-service] Plugin "dataSourceManagement" has been disabled since the following direct or transitive dependencies are missing or disabled: [dataSource]
log [20:28:08.072] [info][plugins-service] Plugin "dataSource" is disabled.
log [20:28:08.073] [info][plugins-service] Plugin "visTypeXy" is disabled.
log [20:28:08.155] [info][plugins-service] Plugin initialization disabled.
log [20:28:08.165] [warning][savedobjects-service] Skipping Saved Object migrations on startup. Note: Individual documents will still be migrated when read or written.
np bld log [20:28:09.336] [info][@osd/optimizer] initialized, 53 bundles cached
np bld log [20:28:09.336] [warning][@osd/optimizer] only building [v7dark,v7light] themes, customize with the OSD_OPTIMIZER_THEMES environment variable
np bld log [20:28:09.336] [success][@osd/optimizer] all bundles cached, success after 0.6 sec
watching for changes (2364 files)
log [20:28:09.707] [info][server] basepath proxy server running at http://localhost:5601/pic
server log [20:28:12.449] [info][plugins-service] Plugin "dataSourceManagement" has been disabled since the following direct or transitive dependencies are missing or disabled: [dataSource]
server log [20:28:12.452] [info][plugins-service] Plugin "dataSource" is disabled.
server log [20:28:12.453] [info][plugins-service] Plugin "visTypeXy" is disabled.
server log [20:28:12.565] [info][plugins-system] Setting up [42] plugins: [alertingDashboards,usageCollection,opensearchDashboardsUsageCollection,opensearchDashboardsLegacy,mapsLegacy,share,opensearchUiShared,legacyExport,embeddable,expressions,data,home,console,apmOss,management,indexPatternManagement,advancedSettings,savedObjects,indexManagementDashboards,reportsDashboards,anomalyDetectionDashboards,dashboard,visualizations,visTypeVega,visTypeTimeline,timeline,visTypeTable,visTypeMarkdown,visBuilder,tileMap,regionMap,inputControlVis,visualize,charts,visTypeVislib,visTypeTimeseries,visTypeTagcloud,visTypeMetric,discover,savedObjectsManagement,securityDashboards,bfetch]
server log [20:28:12.780] [info][savedobjects-service] Waiting until all OpenSearch nodes are compatible with OpenSearch Dashboards before starting saved objects migrations...
server log [20:28:12.851] [info][savedobjects-service] Starting saved objects migrations
server log [20:28:12.952] [info][plugins-system] Starting [42] plugins: [alertingDashboards,usageCollection,opensearchDashboardsUsageCollection,opensearchDashboardsLegacy,mapsLegacy,share,opensearchUiShared,legacyExport,embeddable,expressions,data,home,console,apmOss,management,indexPatternManagement,advancedSettings,savedObjects,indexManagementDashboards,reportsDashboards,anomalyDetectionDashboards,dashboard,visualizations,visTypeVega,visTypeTimeline,timeline,visTypeTable,visTypeMarkdown,visBuilder,tileMap,regionMap,inputControlVis,visualize,charts,visTypeVislib,visTypeTimeseries,visTypeTagcloud,visTypeMetric,discover,savedObjectsManagement,securityDashboards,bfetch]
all 5 plugins are installed in OpenSearch Dashboards
dashboards-reporting page
index-management-dashboards-plugin
security-dashboards-plugin
dashboards-alerting
anomaly-detection-dashboards-plugin
Is it expected that we can't run yarn start on macos with node 18.16.0?
throw new Error('No native build was found for ' + target + '\n loaded from: ' + dir + '\n')
^
Error: No native build was found for platform=darwin arch=x64 runtime=node abi=108 uv=1 libc=glibc node=18.16.0
loaded from: /Users/amgalitz/Documents/ad-workspace-macos/frontend-regular/OpenSearch-Dashboards/node_modules/lmdb-store
at Function.load.path (/Users/amgalitz/Documents/ad-workspace-macos/frontend-regular/OpenSearch-Dashboards/node_modules/node-gyp-build/index.js:59:9)
at load (/Users/amgalitz/Documents/ad-workspace-macos/frontend-regular/OpenSearch-Dashboards/node_modules/node-gyp-build/index.js:21:30)
at Object.<anonymous> (/Users/amgalitz/Documents/ad-workspace-macos/frontend-regular/OpenSearch-Dashboards/node_modules/lmdb-store/index.js:6:49)
at Module._compile (node:internal/modules/cjs/loader:1254:14)
at Object.Module._extensions..js (node:internal/modules/cjs/loader:1308:10)
at Module.load (node:internal/modules/cjs/loader:1117:32)
at Function.Module._load (node:internal/modules/cjs/loader:958:12)
at Module.require (node:internal/modules/cjs/loader:1141:19)
at Module.Hook._require.Module.require (/Users/amgalitz/Documents/ad-workspace-macos/frontend-regular/OpenSearch-Dashboards/node_modules/require-in-the-middle/index.js:80:39)
at Module.Hook._require.Module.require (/Users/amgalitz/Documents/ad-workspace-macos/frontend-regular/OpenSearch-Dashboards/node_modules/require-in-the-middle/index.js:80:39)
```
@amitgalitz The error message indicates that the lmdb-store module is looking for a prebuilt binary compatible with your current environment but is unable to find one. The environment is identified by a combination of platform (darwin), architecture (x64), Node.js runtime version (node=18.16.0), and the libc version (glibc). However we are not using lmdb-store any more
ubuntu@ip-172-31-55-237:~/OpenSearch-Dashboards$ yarn why lmdb-store
yarn why v1.22.19
[1/4] Why do we have the module "lmdb-store"...?
[2/4] Initialising dependency graph...
warning Resolution field "typescript@4.0.2" is incompatible with requested version "typescript@~4.5.2"
[3/4] Finding dependency...
error We couldn't find a match!
Done in 0.85s.
We updated to lmdb. Seems your node_modules is not cleaned and it fetched old modules. I have verified using Mac
Could you pls run yarn osd clean first? Make sure use 18.16.0 and checkout Miki's PR. After bootstrap, double check in node_modules folder that lmdb is there, not lmdb-store.
the portal to save settings in advanced settings doesn't seem to work, had to scroll to bottom to see it. not sure if it's intentional
@joshuali925 could you be more specific about this issue?
I think you could either search experimental visualizations
Or you will need to scroll down. Any differences from usage perspective?
Not the actual setting, i'm talking about the black banner appended to the bottom of the screen by react portal. It shows 1 unsaved setting, Cancel changes, Save changes. Now user have to scroll down to see the banner.
i tried again using miki's branch https://github.com/AMoo-Miki/OpenSearch-Dashboards/tree/node-18-webpack-4 and saw the same issue. I do have observability OSD plugin main branch installed, didn't check if that affects it.
@joshuali925 The issue is caused by observability plugin modifying global style classes. This is not related to Node.js 18.
This opensearch-dashboards-body is used to wrap the entire body of application. Opened an issue: opensearch-project/dashboards-observability#485
Hi, dashboards-notification repo encounters an issue that with Node 18, the workflow failed for 5601 port do not respond: opensearch-project/dashboards-notifications#49, while with normal main branch, the workflow passes: opensearch-project/dashboards-notifications#50
This issued is caused Node.js 18, DNS puts IPv6 addresses before IPv4, as IPv6 gets widespread usage.
➜ index-management-dashboards-plugin git:(main) ✗ netstat -an | grep 5601
tcp6 0 0 ::1.5601 *.* LISTEN
Cypress doesn't support ipv6. Therefore, one work-around is to bind OpenSearch Dasbhoards to 0.0.0.0. So OSD will listen on all IPv4 addresses. In opensearch_dashboards.yml file, the setting is:
server.host: "0.0.0.0"
When you run workflow or cypress using node 18, should config opensearch_dashboards.yml and update the server.host.
Node 18 has been released in 2.8
close this issue