Our build is not reproducible by the Mozilla Review team
feinstaub opened this issue · 11 comments
...which means the latest addon version will be removed and replaced with 0.99.57 soon.
Andreea Neamtiu wrote:
I was able to build the final .xpi and I found that there are differences between the generated xpi and the one you uploaded. The differences appear in this files: options.js and app.js
Sorry but there must be no difference. Please fix the issue.
I asked to provide the affected files.
Andreea Neamtiu wrote:
I uploaded the generated files here: https://we.tl/CJbASom8NI
If you compare them with the ones you uploaded for this version, you will notice that the differences appear at:
options.js line 11767, 11995
app.js line 11678, 11823
The files will be deleted from the upload service within 7 days.
OK, I've compared these files and it's related to css scoping of Vue single file components.
I'm not sure why it's generating different ids the following line shows one of these differences:
"data-v-ecd858fe" vs. "data-v-3e533e26" in line 11767 from options.js
Todos:
- Check which Webpack plugin is creating the ids and why it can be generated differently --> maybe I'll check this later.
- Remove scoping from CSS and re-test styling. Maybe remove global styles.
A few days ago, the affected add-on versions were disabled on AMO. Now, the latest available version is 0.99.47 (Last updated, a year ago (Mar 12, 2017))
@feinstaub can you please build the extension and upload it to Mozilla store? The build should be consistent now.
@AWolf81 New version is now online. I close this ticket for now. We can reopen if there is still an issue with the build.
I received email from AMO today. There still seems to be a problem
I tried the build on the Ubuntu virtual machine, using the source code and instructions that you provided this time. Unfortunately, there are still differences in the following files:
app.js line 11670, 11676, 11820, 11824
options.js line 11759, 11765, 29296, 29300
Please fix the issue or use a different method to obfuscate the code.
...
If we do not hear from you within seven (7) days of this notification, this listing may be removed from addons.mozilla.org.
Oh, that's bad and probably not that easy to fix. I need to check if I can find the build step that is creating the different output.
At my build I got one line difference at each file (seems like stlye-loader is creating a different id - it's the style-loader because line 29293 mentions it in options.js):
options.js 0.99.61 (Ubuntu VM 16.04.3 64bit) line 29300
// add the styles to the DOM
var update = __webpack_require__(9)("1e580681", content, true);
options.js 0.99.61-an+fx (Open Suse) line 29300
// add the styles to the DOM
var update = __webpack_require__(9)("28f235c4", content, true);
--------------------------
app.js 0.99.61 (Ubuntu VM 16.04.3 64bit) line 11824
// add the styles to the DOM
var update = __webpack_require__(9)("26164642", content, true);
app.js 0.99.61-an+fx (Open Suse) line line 11824
// add the styles to the DOM
var update = __webpack_require__(9)("45037496", content, true);
@feinstaub Can you please request more information?
- Ubuntu version (I couldn't get the mentioned differences with virtual machine running Ubuntu 16.04.3 64bit - only line 11824 in app.js & 29300 in option.js are different at my build)
- Node and npm version with
$ npm -vand$ node -v. - Also their generated XPI file would help to compare the build.
We already got the answer, the Mozilla Review team is really forthcoming.
I hope it does help.
This is my setup:
- Ubuntu 17.10 64bit
- npm -v : 5.6.0
- node -v: v9.4.0
The generated XPI:
https://we.tl/Yk6HjsecUG
Also, the build process:
local-filesystem-links-0.99.61-src.tar/local-filesystem-links-0.99.61-src$ npm install
dtrace-provider@0.8.5 install /local-filesystem-links-0.99.61-src.tar/local-filesystem-links-0.99.61-src/node_modules/dtrace-provider
node scripts/install.js
node-sass@4.7.2 install /local-filesystem-links-0.99.61-src.tar/local-filesystem-links-0.99.61-src/node_modules/node-sass
node scripts/install.js
Downloading binary from https://github.com/sass/node-sass/releases/download/v4.7.2/linux-x64-59_binding.node
Download complete ] - :
Binary saved to /local-filesystem-links-0.99.61-src.tar/local-filesystem-links-0.99.61-src/node_modules/node-sass/vendor/linux-x64-59/binding.node
Caching binary to /.npm/node-sass/4.7.2/linux-x64-59_binding.node
spawn-sync@1.0.15 postinstall /local-filesystem-links-0.99.61-src.tar/local-filesystem-links-0.99.61-src/node_modules/spawn-sync
node postinstall
node-sass@4.7.2 postinstall /local-filesystem-links-0.99.61-src.tar/local-filesystem-links-0.99.61-src/node_modules/node-sass
node scripts/build.js
Binary found at /local-filesystem-links-0.99.61-src.tar/local-filesystem-links-0.99.61-src/node_modules/node-sass/vendor/linux-x64-59/binding.node
Testing binary
Binary is fine
npm WARN alien-local-filesystem-links@0.99.61 No repository field.
npm WARN alien-local-filesystem-links@0.99.61 license should be a valid SPDX license expression
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.0 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.0: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
added 1292 packages in 46.905s
/local-filesystem-links-0.99.61-src.tar/local-filesystem-links-0.99.61-src$ npm run build
alien-local-filesystem-links@0.99.61 build /local-filesystem-links-0.99.61-src.tar/local-filesystem-links-0.99.61-src
npm run clean && cross-env NODE_ENV=production webpack --progress --hide-modules
alien-local-filesystem-links@0.99.61 clean /local-filesystem-links-0.99.61-src.tar/local-filesystem-links-0.99.61-src
rimraf dist
[11:54:52] [write-file-webpack-plugin] options { exitOnErrors: true,
force: false,
log: true,
test: null,
useHashIndex: true }
10% building modules 0/1 modules 1 active ...esystem-links-0.99.61-src/src/main 10% building modules 0/2 modules 2 active ...-0.99.61-src/src/extension/options 10% building modules 0/3 modules 3 active ...-0.99.61-src/src/extension/content 10% building modules 0/4 modules 4 active ...src/src/extension/background/index 10% building modules 1/4 modules 3 active ...src/src/extension/background/index 10% building modules 2/4 modules 2 active ...src/src/extension/background/index 10% building modules 3/4 modules 1 active ...src/src/extension/background/index 10% building modules 4/5 modules 1 active ....99.61-src/src/extension/constants 10% building modules 4/6 modules 2 active ...nsion/background/checkInstallation 10% building modules 4/7 modules 3 active .../extension/background/addonbarIcon 10% building modules 4/8 modules 4 active ...extension/background/EventHandlers 10% building modules 4/9 modules 5 active ...c/node_modules/vue/dist/vue.common 10% building modules 5/9 modules 4 ac
tive ...c/node_modules/vue/dist/vue.common 10% building modules 6/9 modules 3 active ...c/node_modules/vue/dist/vue.common 10% building modules 7/9 modules 2 active ...c/node_modules/vue/dist/vue.common 10% building modules 8/9 modules 1 active ...c/node_modules/vue/dist/vue.common 11% building modules 9/10 modules 1 active ...ode_modules/webpack/buildin/globa 11% building modules 10/11 modules 1 active ...system-links-0.99.61-src/package 11% building modules 11/12 modules 1 active ...rc/src/extension/background/noti 11% building modules 11/13 modules 2 active ...61-src/src/extension/utils/os-ut 11% building modules 11/14 modules 3 active ...nsion/background/helpers/whiteli 11% building modules 11/15 modules 4 active ...esystem-links-0.99.61-src/src/Ap 11% building modules 11/16 modules 5 active ...src/extension/components/Setting 11% building modules 12/16 modules 4 active ...src/extension/components/Setting 11% building modules 12/17 modules 5 active ...ode_modules/timers-browseri
fy/ma 11% building modules 13/17 modules 4 active ...ode_modules/timers-browserify/ma 11% building modules 14/17 modules 3 active ...ode_modules/timers-browserify/ma 11% building modules 15/17 modules 2 active ...ode_modules/timers-browserify/main.js(node:2454) DeprecationWarning: loaderUtils.parseQuery() received a non-string value which can be problematic, see webpack/loader-utils#56
parseQuery() will be replaced with getOptions() in the next major version of loader-utils.
11% building modules 16/17 modules 1 active ...ode_modules/timers-browserify/ma 12% building modules 17/18 modules 1 active ...esystem-links-0.99.61-src/src/Ap 12% building modules 17/19 modules 2 active ...src/extension/components/Setting 12% building modules 18/19 modules 1 active ...src/extension/components/Setting 12% building modules 19/20 modules 1 active ...ue-loader/lib/component-normaliz 12% building modules 19/21 modules 2 active ...esystem-links-0.99.61-src/src/Ap 12% building modules 19/22 modules 3 active ...src/extension/components/Setting 12% building modules 20/22 modules 2 active ...src/extension/components/Setting 12% building modules 21/22 modules 1 active ...ue-loader/lib/component-normaliz 12% building modules 22/23 modules 1 active ....61-src/node_modules/lodash/loda 12% building modules 23/24 modules 1 active ...ode_modules/webpack/buildin/modu 12% building modules 24/25 modules 1 active ...esystem-links-0.99.61-src/src/Ap 13% building modules 25/26 modules 1
active ...src/extension/components/Setting 13% building modules 26/27 modules 1 active ...esystem-links-0.99.61-src/src/Ap 13% building modules 26/28 modules 2 active ...src/extension/components/Setting 13% building modules 26/29 modules 3 active ..._modules/setimmediate/setImmedia 13% building modules 27/29 modules 2 active ..._modules/setimmediate/setImmedia 13% building modules 28/29 modules 1 active ..._modules/setimmediate/setImmedia 13% building modules 28/30 modules 2 active ...e-style-loader/lib/addStylesClie 13% building modules 29/30 modules 1 active ...e-style-loader/lib/addStylesClie 13% building modules 29/31 modules 2 active ...de_modules/css-loader/lib/css-ba 13% building modules 30/31 modules 1 active ...de_modules/css-loader/lib/css-ba 13% building modules 31/32 modules 1 active ...1-src/node_modules/process/brows 13% building modules 31/33 modules 2 active .../vue-style-loader/lib/listToStyl 13% building modules 32/33 modules 1 active .../vue-style-loader/lib/listT
oStyl 95% emitting[11:54:57] [write-file-webpack-plugin] compiler.outputFileSystem is "NodeOutputFileSystem".
DONE Compiled successfully in 5261ms 11:54:57
Hash: 5c5ba4a53dda7e364405
Version: webpack 2.7.0
Time: 5261ms
Asset Size Chunks Chunk Names
img/Webextension_Setup_warning_message_Windows.gif 1.02 MB [emitted] [big]
options.js 866 kB 0 [emitted] [big] options
background.js 26.7 kB 2 [emitted] background
content.js 14.1 kB 3 [emitted] content
manifest.json 777 bytes [emitted]
installed.css 1.16 kB [emitted]
installed.html 5.12 kB [emitted]
installed.js 507 bytes [emitted]
_locales/de/messages.json 4.27 kB [emitted]
_locales/en/messages.json 4.05 kB [emitted]
_locales/ru/messages.json 4.82 kB [emitted]
css/self-hosted-materialize.css 162 kB [emitted]
css/style.css 724 bytes [emitted]
app.js 314 kB 1 [emitted] [big] app
js/jquery-3.3.1.min.js 86.9 kB [emitted]
js/jquery-observe.js 9.28 kB [emitted]
img/active_icon_16.png 983 bytes [emitted]
img/active_icon_32.png 2.5 kB [emitted]
img/active_icon_64.png 4.94 kB [emitted]
img/addon_icon_48.png 3.11 kB [emitted]
img/addon_icon_96.png 8.1 kB [emitted]
img/inactive_icon_16.png 768 bytes [emitted]
img/inactive_icon_32.png 1.85 kB [emitted]
img/inactive_icon_64.png 3.46 kB [emitted]
index.html 309 bytes [emitted]
background.html 245 bytes [emitted]
options.html 312 bytes [emitted]
/local-filesystem-links-0.99.61-src.tar/local-filesystem-links-0.99.61-src$ npm run bundle
alien-local-filesystem-links@0.99.61 bundle /local-filesystem-links-0.99.61-src.tar/local-filesystem-links-0.99.61-src
web-ext build -s dist/ --overwrite-dest
Building web extension from /local-filesystem-links-0.99.61-src.tar/local-filesystem-links-0.99.61-src/dist
Your web extension is ready: /local-filesystem-links-0.99.61-src.tar/local-filesystem-links-0.99.61-src/web-ext-artifacts/local_filesystem_links-0.99.61.zip
I couldn't find an image to setup a VM ware. I just see the latest version 18.x of Ubuntu. So I can not run a build with Ubuntu 17.10, at the moment.
So I compared the files from AMO and the published version and detected the following differences:
App.js diff (left amo, right published)
- reference numbers of the includes are different
- style loader creates a different id (similar to the issue from Vue scoping)
Options.js diff
- same issues as at app.js
At the moment, I don't know what's wrong and I would have to do more investigations:
- Why are the import numbers different? Is this a known issue of style loader? Or is there a different cause?
- Why is the id at the style-loader different? Is the dependency style-loader a different version? NPM isn't locking dependencies which could cause problems - could be improved with Yarn.
@feinstaub Can you please ask Mozilla team if it's possible to accept the version as it is now and we'll change the build system with the next version? The code is not identically which is bad but the functionality is not affected by these differences. Just different comments and one different identifier.
I think that task can not be solved with-in 7 days. I'm thinking of the following possibilities:
- Remove build system and load the scripts manually
- Use a different build system (e.g. Rollup) but this could cause same issues
@feinstaub Can you please publish version 0.99.62?
I changed the build system and compared the created bundle from three systems Windows, Ubuntu and OpenSuse. So I'm pretty sure that the new version should work and is consistent.
I'm still not sure why the Vue files are creating inconsistent builds but with Vue-template-loader it is working and I'm not seeing any inconsistencies.
Basically I've splitted the *.vue files into three separate parts HTML, CSS and JS. (Removed SCSS because I think CSS is OK here.) So the build is bundling three files into the final js.
As side-effect to the build change I've updated all dependencies that are required to build the extension. So we're now up-to-date with the dependencies.
I've also tested the extension and everything is working like before the change.
Thanks for that cleanup! I submitted 0.99.62 just now.
I think we can close this issue now.
If we're getting a feedback from Mozilla about a similar issue we can re-open it.