gaia compile
anborg opened this issue · 21 comments
I couldn't compile frontend. Where can I find steps to compile ?
At root
go build
go: downloading github.com/hashicorp/go-hclog v0.0.0-20180709165350-ff2cf002a8dd
go: downloading github.com/dgrijalva/jwt-go v3.2.0+incompatible
go: downloading github.com/robfig/cron v0.0.0-20180505203441-b41be1df6967
cd frontend
yarn install
yarn serve
frontend % yarn serve
yarn run v1.22.4
$ vue-cli-service serve
INFO Starting development server...
98% after emitting CopyPlugin
ERROR Failed to compile with 1 errors 5:59:35 PM
error in ./src/store/modules/menu/lazyLoading.js
Module build failed (from ./node_modules/@vue/cli-plugin-eslint/node_modules/eslint-loader/index.js):
TypeError: Cannot read property 'range' of null
Occurred while linting /Volumes/data01/projects/projects_go/gaia/frontend/src/store/modules/menu/lazyLoading.js:3
at SourceCode.getTokenBefore (/Volumes/data01/projects/projects_go/gaia/frontend/node_modules/eslint/lib/token-store/index.js:303:18)
at checkSpacingBefore (/Volumes/data01/projects/projects_go/gaia/frontend/node_modules/eslint/lib/rules/template-curly-spacing.js:60:42)
at TemplateElement (/Volumes/data01/projects/projects_go/gaia/frontend/node_modules/eslint/lib/rules/template-curly-spacing.js:119:17)
at /Volumes/data01/projects/projects_go/gaia/frontend/node_modules/eslint/lib/util/safe-emitter.js:45:58
at Array.forEach (<anonymous>)
at Object.emit (/Volumes/data01/projects/projects_go/gaia/frontend/node_modules/eslint/lib/util/safe-emitter.js:45:38)
at NodeEventGenerator.applySelector (/Volumes/data01/projects/projects_go/gaia/frontend/node_modules/eslint/lib/util/node-event-generator.js:251:26)
at NodeEventGenerator.applySelectors (/Volumes/data01/projects/projects_go/gaia/frontend/node_modules/eslint/lib/util/node-event-generator.js:280:22)
at NodeEventGenerator.enterNode (/Volumes/data01/projects/projects_go/gaia/frontend/node_modules/eslint/lib/util/node-event-generator.js:294:14)
at CodePathAnalyzer.enterNode (/Volumes/data01/projects/projects_go/gaia/frontend/node_modules/eslint/lib/code-path-analysis/code-path-analyzer.js:632:23)
at /Volumes/data01/projects/projects_go/gaia/frontend/node_modules/eslint/lib/linter.js:752:32
at Array.forEach (<anonymous>)
at runRules (/Volumes/data01/projects/projects_go/gaia/frontend/node_modules/eslint/lib/linter.js:747:15)
at Linter._verifyWithoutProcessors (/Volumes/data01/projects/projects_go/gaia/frontend/node_modules/eslint/lib/linter.js:899:31)
at /Volumes/data01/projects/projects_go/gaia/frontend/node_modules/eslint/lib/linter.js:955:35
at Array.map (<anonymous>)
@ ./src/router/index.js 5:0-60 22:15-26 26:15-26 30:15-26
@ ./src/main.js
@ multi (webpack)-dev-server/client?http://192.168.0.199:8081/sockjs-node (webpack)/hot/dev-server.js ./src/main.js
Hi @anborg
That looks like some kind of version thing... Frigging node. :) I'll take a look.
What are you trying to do? I usually run npm run serve or if you would like to create a production install then run make compile_frontend from the root which does this:
compile_frontend:
cd ./frontend && \
rm -rf dist && \
npm install && \
npm run build
Meaning we usually don't use yarn.
Hi @Skarlso,
Sorry I almost forgot what I did...
The frontend is compiled and started
cd ./frontend && \
rm -rf dist && \
npm install && \
npm run serveThe backend seem to have problem starting.
/Volumes/data01/projects/project-experts/gaia/cmd/gaia
go build
ls
gaia main.go
./gaia
2020-12-13T00:51:25.654-0500 [DEBUG] Gaia: executeable path found: path=/Volumes/data01/projects/project-experts/gaia/cmd/gaia
2020-12-13T00:51:26.177-0500 [WARN ] Gaia: using auto-generated key to sign jwt tokens, do not use in production
2020-12-13T00:51:26.177-0500 [INFO ] Gaia: vault file doesn't exist. creating...
2020-12-13T00:51:26.178-0500 [INFO ] Gaia: the vault is empty
2020-12-13T00:51:26.178-0500 [INFO ] Gaia: global worker registration secret has not been generated yet. Will generate it now...
2020-12-13T00:51:26.178-0500 [INFO ] Gaia: rbac enabled: false
2020-12-13T00:51:26.178-0500 [INFO ] Gaia: rbac disabled
2020-12-13T00:51:26.179-0500 [ERROR] Gaia: Cannot find assets in production mode.
2020-12-13T00:51:26.179-0500 [ERROR] Gaia: cannot initialize handlers: error="could not locate box "../frontend/dist""
Perhaps I'm starting from wrong directory?
Would be great if you could provide list of commands to run from root gaia directory
These frontend errors are likely due to backend errors
App running at:
- Local: http://localhost:8081/
- Network: http://192.168.0.137:8081/
Note that the development build is not optimized.
To create a production build, run npm run build.
Proxy error: Could not proxy request /api/v1/login from localhost:8081 to http://localhost:8080.
See https://nodejs.org/api/errors.html#errors_common_system_errors for more information (ECONNREFUSED).
Proxy error: Could not proxy request /api/v1/login from localhost:8081 to http://localhost:8080.
See https://nodejs.org/api/errors.html#errors_common_system_errors for more information (ECONNREFUSED).
Proxy error: Could not proxy request /api/v1/login from localhost:8081 to http://localhost:8080.
See https://nodejs.org/api/errors.html#errors_common_system_errors for more information (ECONNREFUSED).
Proxy error: Could not proxy request /api/v1/login from localhost:8081 to http://localhost:8080.
See https://nodejs.org/api/errors.html#errors_common_system_errors for more information (ECONNREFUSED).
Proxy error: Could not proxy request /api/v1/login from localhost:8081 to http://localhost:8080.
See https://nodejs.org/api/errors.html#errors_common_system_errors for more information (ECONNREFUSED).
Indeed. You need to build from root. There is a Makefile so if you have make then simply run make to run in dev mode. Otherwise there is another command to run in prod I have to look up. I'm on the phone.
So, if you want a prod release, you run the following commands:
make release
This will create the single binary which will have all the components in it, and that you can run now from anywhere you like.
This only creates a linux binary. Does it crate exe in windows, and mac executable?
cd ./frontend && \
rm -rf dist && \
npm install && \
npm run build
up to date, audited 2725 packages in 18s
58 vulnerabilities (37 low, 12 moderate, 8 high, 1 critical)
To address issues that do not require attention, run:
npm audit fix
To address all issues (including breaking changes), run:
npm audit fix --force
Run `npm audit` for details.
> gaia@0.2.5 build
> vue-cli-service build
⠋ Building for production...Browserslist: caniuse-lite is outdated. Please run next command `npm update`
Browserslist: caniuse-lite is outdated. Please run next command `npm update`
Browserslist: caniuse-lite is outdated. Please run next command `npm update`
Browserslist: caniuse-lite is outdated. Please run next command `npm update`
Browserslist: caniuse-lite is outdated. Please run next command `npm update`
Browserslist: caniuse-lite is outdated. Please run next command `npm update`
⠹ Building for production...
WARNING Compiled with 2 warnings 2:04:06 PM
warning
asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).
This can impact web performance.
Assets:
fonts/Lobster-Regular.9b458667.ttf (397 KiB)
img/fontawesome-webfont.912ec66d.svg (434 KiB)
img/logo.02b61d4e.png (785 KiB)
css/app.b3662530.css (249 KiB)
js/chunk-259a07e9.3f9ca052.js (693 KiB)
js/chunk-vendors.32472a9b.js (479 KiB)
warning
entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance.
Entrypoints:
app (758 KiB)
css/chunk-vendors.95203435.css
js/chunk-vendors.32472a9b.js
css/app.b3662530.css
js/app.d68bb284.js
File Size Gzipped
dist/js/chunk-259a07e9.3f9ca052.js 693.01 KiB 175.34 KiB
dist/js/chunk-vendors.32472a9b.js 478.71 KiB 161.68 KiB
dist/js/chunk-3254d052.89ce089b.js 169.24 KiB 48.72 KiB
dist/js/chunk-db1f81b8.006298be.js 63.18 KiB 16.39 KiB
dist/js/chunk-2e317017.e234ace6.js 47.53 KiB 10.35 KiB
dist/js/app.d68bb284.js 28.94 KiB 9.11 KiB
dist/js/chunk-4fd86b95.cd5a4fc3.js 15.56 KiB 5.45 KiB
dist/js/chunk-249bc364.0252e2c6.js 14.59 KiB 4.98 KiB
dist/js/chunk-3cb231ea.4d011e8f.js 13.58 KiB 3.54 KiB
dist/js/chunk-42555882.2675cc3f.js 13.57 KiB 5.05 KiB
dist/js/chunk-3da1355d.cfa66de9.js 12.33 KiB 4.44 KiB
dist/js/chunk-22e7666e.13dc113d.js 11.18 KiB 4.26 KiB
dist/js/chunk-debf950a.1cbdbf2c.js 9.52 KiB 2.46 KiB
dist/js/chunk-3046779f.ac0f0007.js 4.16 KiB 1.71 KiB
dist/js/chunk-54d4695e.0c849a1b.js 3.06 KiB 1.26 KiB
dist/css/app.b3662530.css 249.43 KiB 36.72 KiB
dist/css/chunk-3254d052.e4b6a03c.css 13.80 KiB 2.44 KiB
dist/css/chunk-2e317017.e9e0f2c5.css 3.69 KiB 0.87 KiB
dist/css/chunk-249bc364.a8dc939c.css 2.02 KiB 0.66 KiB
dist/css/chunk-db1f81b8.b56bbd50.css 1.90 KiB 0.77 KiB
dist/css/chunk-vendors.95203435.css 1.15 KiB 0.51 KiB
dist/css/chunk-3cb231ea.9edd5170.css 0.83 KiB 0.40 KiB
dist/css/chunk-3046779f.fc7c8a39.css 0.74 KiB 0.39 KiB
dist/css/chunk-259a07e9.a3ddfca4.css 0.40 KiB 0.22 KiB
dist/css/chunk-42555882.d2f81421.css 0.39 KiB 0.22 KiB
dist/css/chunk-22e7666e.4659164d.css 0.37 KiB 0.18 KiB
dist/css/chunk-4fd86b95.5613c8db.css 0.36 KiB 0.19 KiB
dist/css/chunk-debf950a.816e7747.css 0.31 KiB 0.20 KiB
dist/css/chunk-3da1355d.1766c1fd.css 0.31 KiB 0.17 KiB
dist/css/chunk-54d4695e.67e32fc5.css 0.06 KiB 0.08 KiB
Images and other types of assets omitted.
Images and other types of assets omitted.
DONE Build complete. The dist directory is ready to be deployed.
INFO Check out deployment instructions at https://cli.vuejs.org/guide/deployment.html
go get github.com/GeertJohan/go.rice && \
go get github.com/GeertJohan/go.rice/rice && \
cd ./handlers && \
rm -f rice-box.go && \
rice embed-go && \
cd ../helper/assethelper && \
rm -f rice-box.go && \
rice embed-go
go: github.com/GeertJohan/go.rice upgrade => v1.0.2
go: downloading github.com/GeertJohan/go.rice v1.0.2
go: found github.com/GeertJohan/go.rice/rice in github.com/GeertJohan/go.rice v1.0.2
go: downloading github.com/GeertJohan/go.incremental v1.0.0
go: downloading github.com/akavel/rsrc v0.8.0
go: downloading github.com/nkovacs/streamquote v1.0.0
go: downloading github.com/jessevdk/go-flags v1.4.0
env GOOS=linux GOARCH=amd64 go build -ldflags "-s -w -extldflags -static" -o gaia-linux-amd64 ./cmd/gaia/main.go
-rwxr-xr-x 1 mer staff 42893312 Dec 27 14:05 gaia-linux-amd64
-rw-r--r-- 1 mer staff 11626 Dec 13 00:44 gaia.go
-rw-r--r-- 1 mer staff 3391 Dec 27 14:04 go.mod
-rw-r--r-- 1 mer staff 39316 Dec 27 14:04 go.sum
drwxr-xr-x 15 mer staff 510 Dec 27 14:04 handlers
drwxr-xr-x 7 mer staff 238 Dec 13 00:44 helm
drwxr-xr-x 7 mer staff 238 Dec 13 00:44 helper
drwxr-xr-x 5 mer staff 170 Dec 13 00:44 plugin
drwxr-xr-x 7 mer staff 238 Dec 13 00:44 providers
drwxr-xr-x 9 mer staff 306 Dec 13 00:44 screenshots
drwxr-xr-x 12 mer staff 408 Dec 13 00:44 security
drwxr-xr-x 3 mer staff 102 Dec 13 00:44 server
drwxr-xr-x 4 mer staff 136 Dec 13 00:44 services
drwxr-xr-x 5 mer staff 170 Dec 13 00:44 static
drwxr-xr-x 15 mer staff 510 Dec 13 00:44 store
drwxr-xr-x 8 mer staff 272 Dec 13 00:44 workers
mer@PKML01 gaia % ./gaia-linux-amd64
zsh: exec format error: ./gaia-linux-amd64
Eek, sorry, you're right. That only creates a linux binary. We definitely should use gox to build for multiple platforms. :/
Luckily, now that you have everything else built, you just need to run this command:
go build $(GO_LDFLAGS_STATIC) -o gaia.exe ./cmd/gaia/main.go
which replaces the compile_backend part of the release process.
Since GO is multi platform, perhaps a separate Makefile config, that creates gaia.exe or gaia(mac). Automatically detecting the OS - I believe GO supports such building, not sure how complex it is to add to current setup.
End user usecase:
Build an os-native binary of whichever operating system where the user builds gaia. Possible reasons could be (1) User do not want to depend on docker (2) user can not use docker (3) User do not want to have overhead of docker (4) user prefers simple native binary.
@anborg hi. If you use the attached pr I added gox to the mix. you have to install it, and after that if you run
make binaries it will create all the binaries. :)
So the steps would be:
# Install Gox
make compile_frontend
make static_assets
make binaries
This should now include all the things in all the binaries. Executed from the root of the project.
@Skarlso , thanks..
Is there a way to get it working on win?
C:\data\projects\projects_go\src\github.com\gaia-pipeline\gaia>make dev
go run ./cmd/gaia/main.go -home-path=/tmp -dev=true
2020-12-29T10:33:26.836-0500 [WARN ] Gaia: using auto-generated key to sign jwt tokens, do not use in production
2020-12-29T10:33:26.839-0500 [INFO ] Gaia: rbac enabled: false
2020-12-29T10:33:26.839-0500 [INFO ] Gaia: rbac disabled
2020-12-29T10:33:26.844-0500 [INFO ] Gaia: worker gRPC server about to start on port: 8989
⇨ http server started on [::]:8080
Though message says some grpc server running :8989, not sure if it is running in windows.
What are you trying to do? Gaia provides a rest api. Not a GRPC one. The GRPC api is the plugin's api. That's not intended to be used directly. If you are trying to use it without the frontend, I suggest to take a look at the swagger docs, which you can view if you started up the server by navigating to : http://localhost:8080/api/v1/swagger/index.html
You can then see what you can do.
Looking at the make command though, you are running under dev, which means you aren't running the frontend by default.
Navigate to the frontend folder, and run npm run server then navigate to http://localhost:8081 where the frontend lives.
Great thanks...
Backend:
make devhttp://localhost:8080/api/v1/swagger/index.html
Frontend:
cd frontend
npm run servehttp://localhost:8081/#/overview
I can live happily without docker, in windows (though I wish to use linux).
Will try different options
- saving state of server in a folder
- changing port for backendapi & frontend
- using Vault for storing pwd, db setting etc and
- check if Consul can be used for storing config values, and use those in worker/action nodes in a pipeline?!
- check if I can use letsencrypt to serve as secure
- figure out if "pipeline" topology can be saved / versioned as a file?!
- find if there is a way to see the overall status of pipeline workflow jobs in a dasboard?!
How do I create a single binary that serves both api, and javascript-frontend in single port e.g 8080, i.e no need to run two different commands?
Yes docker combines both BE&FE. Yes it is good to keep BE&FE separate. But, if I could just create a simple gaia-app.go which serves a fixed version js-frontend (no npm), and the swagger, I can just drag drop that exe/binary anywhere and try different things ...
I'm new to go. Feels like there must be a way to just copy a fixed version of js/css (output of npm?) , build a uber binary, and serve under 8080? Any suggestions?
You don't even have to provide that. Just run the compiled binary of your chose and navigate to http://localhost:8080.
This will now contain the backend AND the frontend.
The steps are, without using make release:
make compile_frontend
make static_assets
make binaries
In this case, the resulting binary (under bin/${os}/${arch}/gaia) will contain both front and the backend.
Thank you
It took a bit of time to have 4 binaries. Additional time may not matter for my needs..
Fair point. I added an override. You can now run this:
make BINARIES=windows/amd64 binaries
This will only build windows/amd64. You can define whatever you want in there, with a coma separated list. Look for an example in the Makefile.