Node's nested node_modules approach is basically incompatible with Windows
jonrimmer opened this issue · 175 comments
Node needs an alternative approach to endless, recursively nested node_modules folders on Windows. Most Windows tools, utilities and shells cannot handle file and folder paths longer than 260 characters at most. This limit is easily exceeded, and once it is, install scripts start breaking and node_modules folders can no longer be deleted using conventional methods.
This will only get worse as Node packages get more complicated, with bigger and deeper dependency hierarchies. There should be a way to install and run packages that does not use file system recursion to represent the dependency hierarchy.
Suggestion: to mitigate the above mentioned problem, make Node.js version 1.0 (and the corresponding npm) use n_m (3 characters) instead of node_modules (12 characters) by default. This change is expected to double the possible hierarchy depth for modules with names containing nine letters or less.
If a backwards compatibility is necessary, node_modules could still be supported (though not as default). For example,
require('modulename')could look innode_modulesif the module is not found inn_m,npm update modulenamecould killnode_modules/modulename(if it exists) and installn_m/modulenameand dependencies.
That doesn't solve the problem, just gives you slightly more breathing room.
The only real solution is going to have all modules at a single level, requiring each other, instead of all having private copies of their dependencies. This would either have to be versioned, and/or module maintainers need to be more careful about breaking changes.
Most package managers work this way, and seem to be getting along nicely.
As an idea: node_modules/module@version
The module@version solution eliminates the problem much better than n_m, but not without introducing a bunch of other problems.
Example: if somemodulename@1.2.3 and somemodulename@4.5.6 are both present, what should npm update somemodulename do?
In the same example, what should npm uninstall somemodulename do?
@Mithgol it should probably install somemodule@la.te.st, alongside the others. But npm purge or npm clean should probably be added.
@Mithgol if the latest version is 4.5.6 do nothing, otherwise add the latest version. If you can be clever and determine that a previous version is no longer needed then remove it, otherwise don't worry about it.
@randunel somemodule@latest would be far more sensible if there's an actual need for it, though for backwards-compatibility, I'd guess somemodule would symlink to the latest anyway.
👍, devs are running into this with individual lodash packages too.
See lodash/lodash#501, gulpjs/gulp-util#23, & twitter/theporchrat.
There looks to be the start of addressing the issue here, but it may need to be used in other methods.
Related to #6891.
Some docs for the code @jdalton linked to: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx#maxpath
I guess Node's developers would have to rewrite both require() and npm to use \\?\… paths. If it's possible, 256-character limitation becomes void — even without resorting to n_m or @version.
(Maybe not exactly require() and npm, but rather the entire underlying fs module, also process.cwd(), etc.)
Even if all of Node and NPM are rewritten to use long paths, so long as NPM supports scripts there is a danger of them randomly blowing up when run on Windows due to finding themselves inside a very deep hierarchy then trying to run utilities that don't support long paths.
@jonrimmer Many third-party scripts for NPM (such as Grunt, or Mocha, or node-pre-gyp) are Node.js scripts and thus they are to become automagically fixed when Node.js core modules (such as fs and path) and path-related methods (such as process.cwd()) all start using \\?\… unilaterally.
The other utilities would indeed break and thus the community would have to replace them or demand upgrades. (A similar ruckus once happened when Node.js started supporting Windows and many UN*X-only tools and CLI scripts experienced a lot of incompatibility-related problems. There was much suffering; however, the community eventually fixed the problems or developed some workarounds, sometimes as simple as using path.join instead of a former + '/' +. Or using os.EOL instead of a former \n.)
Hello everyone! I was unpleasantly surprised when I found out our module was hitting an unexpected wall in Windows host environments. If anyone needs a quick remedy:
- You can delete the Windows paths by slicing out a branch of the Node dependency tree and moving it to a shorter path in the host environment.
- If you are developing on a Windows host, but using a Unix environment in production, you can use Vagrant to create a virtual machine, and run you application in the guest environment.
If you would like to keep your sources and Node modules synced with the Windows host, you can install lodash in the guest environment and usenpm linkto resolve it as a module dependency for your application.
FYI Node (and thus npm) always uses UNC paths internally, before actually calling into the filesystem. It is only third-party tools that have a problem. (Unfortunately one of those third-party tools is Windows Explorer, but, that's Microsoft's bug...)
FYI Node (and thus npm) always uses UNC paths internally
I do see the _makeLong() helper used a lot in fs though is there a chance something was missed?
Is it handled in require?
@domenic Microsoft have said many times that MAX_PATH limitations are not considered a bug, just a feature of Windows. The file system supports long paths, the OS does not, there is a difference.
If Node's intention is to support NTFS, then long paths are fine. If the intention is to support Windows, then there needs to be an alternative to long paths.
@jonrimmer The OS does support long paths, or at least all API methods that libuv uses support long paths. So the distinction you are trying to make does not apply.
@jdalton require delegates to fs.
@domenic Cool.
I'm experimenting with this on my Windows 8.1 machine (I haven't hit this issue on my own yet):
- I created several nested long name directories and navigated via the command line as far as I could before receiving
The filename or extension is too long.errors. - I then did
npm install lodash.template. While I couldn't create further folders in Explorer or navigate via the command line thenpm installsucceeded and the folders were created. - Once they were created by
npmI could navigate to them with Explorer
(still not with the command line though). - In Node, from the directory I performed
npm installin, I didrequire('lodash.template')and it loaded the module without problems. - I even did
require('lodash.template/node_modules/lodash.defaults/node_modules/lodash._objecttypes')and it loaded the module w/o issue. - I noticed in Explorer when I try to copy the path it gives me a truncated form:
C:\Users\jdalton\Desktop\A12345~1\A12345~1\A12345~1\A12345~1\NODE_M~1\LODASH~1.TEM\NODE_M~1\LODASH~1.DEF\node_modules\lodash._objecttypes. - Also if I
cd <that truncated path>I can navigate to the path in the command line as well. So now I'm in the directory in the command line and in Explorer (I can also create more folders/files) and the module is loading in Node.
@domenic Microsoft: "In the Windows API (with some exceptions discussed in the following paragraphs), the maximum length for a path is MAX_PATH, which is defined as 260 characters." [1]
Some (but not all) Windows APIs support, as an alternative, unicode paths up to 32,767 chars, but this is an alternative - not the default. It is not the option used in many Windows utilities including core parts of the OS. Claiming therefore, that Windows supports long paths, is like claiming old NT was Unix compatible because there is a POSIX subsystem, or claiming Linux is compatible with the win32 APIs because you can install Wine. It is a entirely disingenuous attempt to misrepresent the actual reality. Windows as an OS does not support long paths. That it is possible to write Windows compatible software that supports long paths does not change that fact.
[1] http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx
I don't really care about whatever word games you want to play around the word "support." Sure, if you wish, all I'm saying is that Node and npm are Windows compatible software that support long paths.
You are the one playing games - calling core parts of Windows like Explorer "3rd party tools", and suggesting that not supporting long paths is a bug. Microsoft have made it clear repeatedly that non-support for long paths is not a bug, and not something that will change.
A package manager creating paths that do not work with the majority of the software written for an OS, then claiming compatibility with the OS, is playing games, at your users expense.
I guess we'll have to stick with just being compatible with node and npm for Windows then.
To argue that the node module system should change due to deficiencies in parts of Windows is a dead end.
Well, I guess that writes off my ability to use node (or at least some node projects) under a Windows environment, then? I've been attempting to merge a repository elsewhere on Github that uses node, and the merge command fails due to the length of the pathnames.
Maybe there's a workaround I can use (seriously, I'm open to suggestions) - or maybe you can argue that the problem lies with how the Git command line deals with directory paths - but the end result is the same: I can't work with this node project under Windows.
Nested dependenices shouldn't be in git repositories. Each dependency has its own repository, and npm assembles them.
You could always open a ticket with git to add support for long paths, just like node does.
Or you can check out the git repo directly under C: ?
@OrangeDog - I'm looking into ways that I can work with this repository specifically - I'm not sure how it came to be this way, but it is unworkable currently. My completely-off-the-cuff speculation would be trying to .gitignore the node_modules directory, trusting the node system to build the deep directories correctly.
@dougwilson - that was the first thing I tried, but no dice - even putting the repo at c:/g/ the paths are still too long:
c:/g/node_modules/grunt-contrib-imagemin/node_modules/image-min/node_modules/gifsicle/node_modules/bin-wrapper/node_modules/download/node_modules/request/node_modules/form-data/node_modules/combined-stream/node_modules/delayed-stream/test/integration
Just a comment. This appears to be problems with tools that do not support long paths in Windows; is this truly a Node issue or just a common problem with Windows applications?
So, Node and npm both work with long paths. That's good. But… what if a module contains a C/C++ addon? Can Python and Visual Studio actually build it if a path is long?
The most annoying part of this bug to me is that I can't move or delete folders with long path names inside of them. To do so, I either have to crawl the directory tree manually, slicing out portions and pasting them into a temp folder, or resort to a cygwin shell, which will actually move/delete even very long path names.
One of the things that struck me was that people seem to rely strongly on all of these miniscule lodash mini-packages. Is lodash truly so large, and your requirements truly so strict, that you can't just require all of lodash? If you require a certain lodash mini-package, that package will require other lodash-mini packages, which in turn require their own lodash mini-packages, to the point where you've basically included half of lodash, and your startup times are going to be far higher than they would have been with the full lodash package just from having to load so many individual modules.
The gulp-jshint module is a good example of this. Install it and then trace down the node_modules tree, and see how ridiculously far down the rabbit hole goes.
One of the things that struck me was that people seem to rely strongly on all of these miniscule lodash mini-packages. Is lodash truly so large, and your requirements truly so strict, that you can't just require all of lodash?
Modules are great and large dep graphs can and do happen. Trying to poo-poo a dev for using small reusable modules is wonky. Things should "just work".
If you require a certain lodash mini-package, that package will require other lodash-mini packages, which in turn require their own lodash mini-packages, to the point where you've basically included half of lodash, and your startup times are going to be far higher than they would have been with the full lodash package just from having to load so many individual modules.
There are pros & cons to monolithic vs individual module vs bundles of modules.
The
gulp-jshintmodule is a good example of this. Install it and then trace down the node_modules tree, and see how ridiculously far down the rabbit hole goes.
To avoid this issue in the next release Lo-Dash core is inlining ~60 functions into various modules which greatly reduces the dep graphs (many to 0 deps) at the cost of duplication. It's a bummer of a band-aid but it's the state of things at the moment.
To avoid this issue in the next release Lo-Dash core is inlining ~60 functions into various modules which greatly reduces the dep graphs (many to 0 deps) at the cost of duplication. It's a bummer of a band-aid but it's the state of things at the moment.
From a maintainability and quality perspective this kind of sucks, but from a performance perspective this is a win, because those functions will be defined in the same V8 context and can thus be inlined when the various pieces of lodash get hot, which can't be done when helpers are defined in separate modules.
From a maintainability and quality perspective this kind of sucks,
Generally yes, but for Lo-Dash maintainability and quality aren't a problem because it's all generated from the same monolithic reference source and all modules are run through the same unit tests. So for me it was just adding a list of always-inlined modules and adding a heuristic to inline functions with only one dependent. Even though functions are inlined the individual modules for each are still created so devs can still use every bit and bob.
but from a performance perspective this is a win, because those functions will be defined in the same V8 context
Good to know.
Moving the thread back on track. About two weeks ago I did some experiments to narrow down the issue. I couldn't come up with a case where Node/npm failed however I did find gotchas in the Windows UI/command-line.
For devs running into these issues is there a recommended workaround or best practice?
If so, is it documented?
A bigger question would be is there another dependency structure that Node could use that would avoid the nested issue?
Haven't read all of this thread, but couldn't this be solved by NPM doing a dedupe on postinstall or at least allowing the npm dedupe command to have a --save option to save the deduped tree?
Aside from being a substantial piece of work, are there any real no-can-do issues with changing the structure to be flat, like the previously suggested node_modules/module@version?
The issues mentioned aren't things that will fundamentally break node, just things that need some thinking to come up with a decent solution. Keeping on with the nested folders approach however is causing real problems, regardless of what one might think of windows explorer or other applications that don't play nice with long paths.
@mstade what require('module-name') is supposed to do if there is node_modules/module-name@version1 and node_modules/module-name@version2?
@vkurchatkin node would need a new look-up algorithm of course, presumably making use of package.json in order to figure out version information and such.
I doubt that it's impossible to come up with a new algorithm that works with a flat folder structure. The current algorithm is simple and intuitive, but the assumption that deep folder nesting is A-OK doesn't hold up on a major OS, so it might be worth spending a few brain cells figuring out a more fitting solution.
I don't think this is rocket surgery; but I also don't doubt it'd be a fair bit of work to implement, particularly in making sure existing projects don't get shafted.
@mstade There are several issues at hand with a naive flatten of all modules with the same version to a single folder. The following is not a complete list, but are real world problems that would need to be argued against as it could/would damage experiences on other OS.
-
Shared state expectations in modules is broken. See the following as a simple example:
var uid = 0; module.exports = function next() { return ++uid; }
If this is originally 2 submodules and is then converted into a single one (name@version); the next() function can no longer be guaranteed to be incrementing by 1 since it may (very dangerous word) be consumed by other modules.
-
This does not fit with the current require's algorithm, which is very stable and changing it would have dramatic consequences. In particular, you will need to be sure to tag to a specific version of submodules rather than having possible collisions.
-
This does not work with how some plugin/nested modules work, where they
require('../../..')to interact with a parent module since the parent module has been flattened. -
Collisions become much more of an issue since you can have multiple possible matches in a node_modules directory with different versions. Imagine
require('project_assets/api.js'). Having this resolve to multiple things can introduce much complexity to a simple system.
-
will increment by one regardless, but you couldn't make the assumption that
nextwon't be called from another module that shares the same dependency. Arguably, this code is not particularly good, and the current algorithm enables it, but I realize that's not a particularly good argument for flattening the dependencies – breaking existing code never helped anyone. -
is again bad code making too many assumptions, just as 1). Of course, the current algorithm enables it, so that's unfortunate. Again, I realize pointing fingers at bad code and suggesting they suit themselves isn't a viable option, just stating some obvious things.
I'll assume 2) is a continuation of 1) and you're absolute right – if this is a widespread pattern then it would have pretty dramatic consequences. I didn't know (and still don't, really) this was a common pattern. I don't use and haven't seen either 1) or 2) before; please forgive my ignorance.
I think 4) has a straight forward solution in using the package.json metadata.
The examples you bring up with code making too many assumptions (enabled by the current algorithm) are a good argument against flattening, since it would indeed introduce some serious complexity. My gut tells me this is a problem that can be overcome, but off hand I have no reasonable suggestions.
Thanks for the insights @bmeck, I learned something.
As an idea: node_modules/module@version
I like this idea. It would be nice, if such a large change were ever to occur, to also rename node_modules to packages, it would nicely compliment package.json and remove the word node which makes certain developers believe npm is a tool for node development and nothing else.
I might be stating the obvious, but isn't this a discussion to be had on https://github.com/npm/npm ?
Implementing the above discussed changes would require some Node.js changes outside of npm as well; for example, require. (Might also affect node-gyp, I dunno. My comment #6960 (comment) is still unanswered.)
Example: if somemodulename@1.2.3 and somemodulename@4.5.6 are both present, what should npm update somemodulename do?
Nothing. My exdi module as example:
npm install exdi -> install latest exdi in "exdi" directory, npm update will install "exdi"
npm install exdi@1.1 -> install exdi 1.1 in "exdi@1.1" directory and leave it alone
require is a problem but we could follow exact same convention so if You require "exdi" it will look for "exdi" and then for "exdi@*" and if we require "exdi@1.1" it will look for "exdi@1.1" directory
If there is a package.json with "exdi: 1.1" then require('exdi') will always require version 1.1.
In the same example, what should npm uninstall somemodulename do?
Again, same thing as above. This way we can have multiple versions of one package without need to have nested structure. When parent directory is node_modules, require could look into current directory for deps and not into subdirectory "node_modules".
I'm new to node development on Windows, and this is basically a showstopper. The lodash packages are particularly problematic, and cause some installs to hang silently. The problem is exacerbated when trying to clean up the failed installation manually, which doesn't work because of the long path problem.
This is, IMO, a very bad experience for people starting to develop with node on Windows, which most likely will drive them away. If the node/npm teams think this is a problem in Windows itself, and they're not willing to make it work, fine, but please don't provide Windows installers and don't claim Windows support.
This all-or-nothing approach is no good. Of course it is essential to continue providing Windows installers for Node.js.
@khaledh node/npm itself should work fine, what part of the install hangs and can you provide and logs npm-debug or other around this. The only thing which should fail is other windows programs which do not support long paths, such as cmd.exe
I develop on Windows, and this is a huge problem for me right now.
mstade said:
I doubt that it's impossible to come up with a new algorithm that works with a flat folder structure. The current algorithm is simple and intuitive, but the assumption that deep folder nesting is A-OK doesn't hold up on a major OS, so it might be worth spending a few brain cells figuring out a more fitting solution.
I don't think this is rocket surgery; but I also don't doubt it'd be a fair bit of work to implement, particularly in making sure existing projects don't get shafted.
I couldn't agree more. I can understand concerns about taking the time to design & implement a solution, but from both an architecture and a usability perspective, this seems like it should be a top priority. For the end-developer, maintaining endlessly-nested levels of "node_modules" directories is vastly more difficult to support and maintain, on Windows or otherwise, than a simple flat list of "module@version" entries -- but the blatant brokenness of the situation on Windows is beyond frustrating.
Btw -- I know there are several issues to address, but specifically regarding the question about require() and which module version it should actually be requiring -- isn't that easily determined from the version referenced in each module's own packages.json manifest?
@hyrmedia
The only field in package.json that node looks at is main. node's require algorithm is entirely unaware of module versions. it's also a frozen part of node core.
As far as I can tell, nobody is particularly enthusiastic about changing the behaviour of npm or node entirely for Windows alone — a platform few of us use, even fewer of us like, and only a couple of us care about. node has done the correct thing by handling long paths correctly, and at this point it feels like we're being asked to compensate for all the software around us on Windows being crap.
Could you please not use phrases like "a platform few of us use" ? The last two jobs I've had have been exclusively windows shops, and both of them have used node extensively. Don't make generalizations based only on your personal experience. I do like windows, and care a lot about node compatibility on windows, and I know many others aside from the multiple people represented in this thread that do too. I'm not sure exactly what you mean by "node has done the correct thing by handling long paths correctly," other than the fact that node supports long paths, which is not so much a node feature as it is a file system feature of the OS you're on. Since windows continues to hold roughly 90% of the OS market, it seems like increasing node's compatibility with the platform can only benefit node in the long run. In addition, I feel like the module@version approach would have at least two major benefits on any OS compared to the current approach - eliminating duplication and reducing complexity.
The "Software around us on Windows" that rightly or not that are referred to as crap are core utilities. Developers on Windows would have to do a lot of extra work to avoid getting jammed up on this.
Yes, in the ideal world we wouldn't have to deal with this. But developers work in an environment that is dirty and messy, and they just need to get stuff done. Refusing to recognize reality is not a viable position. We are being asked to compensate for the good of the project, and we should.
I don't use windows, but I know a lot of developers who do, and who would like to use node. Please, let's just fix this (or compensate/whatever). Unless we really just want to cut out what is still a majority of the potential market for node.
I'm still missing why this discussion is happening here and not on npm's
issue tracker.
I think it's a valid discussion for either location. Node has to understand
how to locate dependencies, while npm needs to understand how & where to
install them.
On Fri, May 30, 2014 at 4:14 PM, Trevor Norris notifications@github.com
wrote:
I'm still missing why this discussion is happening here and not on npm's
issue tracker.—
Reply to this email directly or view it on GitHub
#6960 (comment).
Another aside on why this may be more pertinent on npm is that flat file structures can emulate nested structures using symlinks (available in Windows Vista and up). Basic concept can be shown by running: https://github.com/bmeck/flatter to see the output (not finished but concept is sound). I will fix it when I get back from JSConf.
Basically:
root/node_modules/a
root/node_modules/a/node_modules/b
root/node_modules/a/node_modules/c
root/node_modules/a/node_modules/c/node_modules/d
becomes:
root/node_modules/f~0 # previously a
root/node_modules/f~1 # previously b
root/node_modules/f~2 # previously c
root/node_modules/f~3 # previously d
# root modules need to link to their flattened counterpart
root/node_modules/a -> root/node_modules/f~0
# flattened modules need to provide their module scope via symlink
# a's scope
root/node_modules/f~0/node_modules/a -> root/node_modules/f~0
root/node_modules/f~0/node_modules/b -> root/node_modules/f~1
root/node_modules/f~0/node_modules/c -> root/node_modules/f~2
# d's scope
root/node_modules/f~3/node_modules/a -> root/node_modules/f~0
root/node_modules/f~3/node_modules/b -> root/node_modules/f~1
root/node_modules/f~3/node_modules/c -> root/node_modules/f~2
root/node_modules/f~3/node_modules/d -> root/node_modules/f~3
npm supporting an install structure such as this can be done without breaking node.
caveats are limited to modules that are not self contained:
- accessing files outside of your module will cause problems (ie. require(path-to-current-modules+'/../../'))
- relying on realpath-ing to have the nested directory structure will break.
I am emphatically against this.
If we're going to modify anything here, it needs to be a clean break from current design. Symlinking to a flat structure is just avoiding addressing the problem, and is introducing complexity we don't need.
A clean break to a directory structure that looks like this is what I have in mind:
node_modules/foo@1.0.0
node_modules/bar@0.0.9
node_modules/foo@1.0.1
node_modules/foo@latest #symlink to 1.0.1
node_modules/bar@latest #symlink to 0.0.9
npm update foo would install the latest version of foo to foo@version, and update the symlink foo@latest.
Tell me I'm crazy.
No offense, but that seems like way too much unnecessary effort, and like it would just lend itself to confusing users.
On May 31, 2014, at 9:25 AM, Bradley Meck notifications@github.com wrote:
Another aside on why this may be more pertinent on npm is that flat file structures can emulate nested structures using symlinks (available in Windows Vista and up). Basic concept can be shown by running: https://github.com/bmeck/flatter to see the output (not finished but concept is sound). I will fix it when I get back from JSConf.
Basically:
root/node_modules/a
root/node_modules/a/node_modules/b
root/node_modules/a/node_modules/c
root/node_modules/a/node_modules/c/node_modules/d
becomes:root/node_modules/f
0 # previously a1 # previously b
root/node_modules/f
root/node_modules/f2 # previously c3 # previously c
root/node_modules/froot modules need to link to their flattened counterpart
root/node_modules/a -> root/node_modules/f~0
flattened modules need to provide their module scope via symlink
a's scope
root/node_modules/f
0/node_modules/a -> root/node_modules/f0
root/node_modules/f0/node_modules/b -> root/node_modules/f1
root/node_modules/f0/node_modules/c -> root/node_modules/f2d's scope
root/node_modules/f
3/node_modules/a -> root/node_modules/f0
root/node_modules/f3/node_modules/b -> root/node_modules/f1
root/node_modules/f3/node_modules/c -> root/node_modules/f2
root/node_modules/f3/node_modules/d -> root/node_modules/f3
npm supporting an install structure such as this can be done without breaking node.caveats are limited to modules that are not self contained:
accessing files outside of your module will cause problems (ie. require(path-to-current-modules+'/../../'))
relying on realpath-ing to have the nested directory structure will break.
—
Reply to this email directly or view it on GitHub.
I'm fairly new to Node, so forgive me if this is not at all possible...
Could all modules just reside alongside each other in node_modules/ ? This would do away with infinite levels of directories. You could then have a node_modules.json file which could show which modules depend on others
This would also mean that if multiple modules used the same module, there would only be one instance of it, rather than an instance in each module that requires it.
You can use npm dedupe if you don't want multiple versions of the same package.
I think the important question to be answered in this thread is are the node and npm team willing to consider a change to how npm installs its packages.
If the answer is a firm "never" then this issue will not be fixed and will never go anywhere.
We could always go for:
node_modules/my_module/1.0
node_modules/my_module/1.1
node_modules/my_module/current (reference to latest)
We just look into package.json for package version. If not set, current is used. Packages could be either always global or local but at application level. This should not break anything.
Or we could just ignore Windows users and go one with our lives without need to change.
I've just encountered this 'bug' also. I'm not developing on Windows, but are looking at setting up a development environment for javascript (amongst others) for a colleague of mine. I installed Windows into Virtualbox and encountered this as soon as I tried to build the generated code from yeoman/generator-angular - I'm glad I'm not using that operative system outside of Virtualbox.
I'm no node expert by any means, but in my point of view the flatten module@version structure with linked nested tree sounds like a possible solution. Won't we hit that 260 char path limit on linked structures?
Reminder: Windows (the OS and file system) supports long file names. Most applications included with Windows do not. The fix for said applications is to move to the unicode version or currently recommended function of the Windows APIs ( http://msdn.microsoft.com/en-us/library/windows/desktop/aa364232(v=vs.85).aspx & see "?" stuff in http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx and link).
I will try to reiterate that moving to a naive module@version can cause issues when modules expect shared state but are present multiple times at the same version in a dependency tree (uncommon but possible and present in various logging / config modules). You can see a couple of comments from various people above about this.
As for the problem with symlinks vs 260 char limit, this may cause issues depending on how the application consuming the file is set up (once again this is all application bugs not Windows the OS). If a realpath occurs prior to access, it will not be a problem; if access is merely absolute it can cause issues. This does let people do things like cd node_modules/really; cd node_modules/deep; cd node_modules/nesting which would have not worked before, and a number of broken apps should start working (not all, once again they should be using long name file access convention).
However, most (all to my knowledge) of these problems revolve around applications using old or limited APIs on purpose (or in ignorance) on Windows. There are several solutions, but I feel that full flat structure being the norm would require a very convincing reason to alter:
- The very stable module system that has build tools and infrastructure around it
- Shared state problems akin to dedupe problems are always present
- Multiple version pruning would require walking all dependencies
- We are working with an OS that supports long paths already... (not talking about applications)
- Seriously, this will never be fixed in Windows bundled applications as everyone decides to harm their structures to appease the 260 char bug in other Windows applications. Technical debt is our bff?
I do understand the issue, but a divide is clear where people using applications supporting long paths vs those without (not OS/FS, cause all modern OS/FS support long enough paths). Some agreement needs to be made on both sides; as flat mapping files causes some issues for long path based workflows (disallows various shared state modules), and long path based workflows do not work with outdated/bundled Windows applications.
I would suggest that this come around to npm if we can use flat structures to emulate nested modules as this would not require changing node. If emulating nested structures does not work sufficiently (basic dev tools need to stop complaining [this would most likely not tailor to various specific programs]), people may want to support a secondary module loading convention that would require changes to both npm and node. However, having 2 different ways to load modules is harmful (multiple possibilities == infinite complexity).
Just a side note, this npm issue says that
Node (and thus npm) always uses the UNC paths internally for fs operations
It seems (as noticed here already by others) that the problem described here is problem of some Windows tools rather than Node's/npm's.
Why not add a field to each package.json file at install time that specifies a relative path to a node_modules directory and defaults to ./node_modules? Where necessary, you could generate a short uid to move the package back up to the top level.
Given an expected path like this;
app/node_modules/a/node_modules/b/node_modules/c/node_modules/d
On *nix, you can do;
app/package.json (packages_path = './node_modules')
app/node_modules/a/package.json (packages_path = './node_modules')
app/node_modules/a/node_modules/b/package.json (packages_path = './node_modules')
app/node_modules/a/node_modules/b/node_modules/c/package.json (packages_path = './node_modules')
app/node_modules/a/node_modules/b/node_modules/c/node_modules/d/package.json
And on Windows, you can do;
app/package.json (packages_path = './node_modules/{packages}')
app/node_modules/{packages}/a/package.json (packages_path = '../../{a_packages}')
app/node_modules/{a_packages}/b/package.json (packages_path = '../../{b_packages}')
app/node_modules/{b_packages}/c/package.json (packages_path = '../../{c_packages}')
app/node_modules/{c_packages}/d/package.json
Also, I agree with the idea of changing the node_modules folder name to packages. It feels more generic that way, and more unified with the naming convention of package.json. This design would make that easy too, or even customizable by any user.
I have asked about renaming node_modules to packages and I think that's also a no go. I think looking at JSPM for a browser package manager might be helpful as it does have flat directory structures. Along with inter-op with npm and resource loading support.
I was developing on Windows and am suddenly not able to push on github anymore, because I installed gulp with npm. It is kind of frustrating, because I have to work around and recreate my developing area on ubuntu, which just consumes time. I wish that windows users should be warned before the installation, because Node really isn't supporting Windows right now. And it doesn't matter if this is the fault of the OS, an API or any other application. All that matters is that it isn't working on Windows right now and there doesn't seem to be a reasonable solution to this problem. So get your shit together and make some decisions, because it is very user unfriendl for Windowsuser.
This is ridiculous. This is hindering developer's ability to use node. They
are core windows utilites. Please stop referring to them like they're not -
ignoring reality doesn't change it.
If we don't want to fix it, fine. But let's stop lying to ourselves.
Daniel
On Jun 9, 2014 8:16 PM, "Bradley Meck" notifications@github.com wrote:
To be clear Node/NPM fully support windows, various Windows applications
do not. We cannot say it does not support Windows long paths, only that
other applications that are useful do not fully support Windows long paths.—
Reply to this email directly or view it on GitHub
#6960 (comment).
See, here's what I don't get:
We're talking about the node_modules directory here. It's machine-created (via the npm client), and for all practical purposes should never be needed to be traversed. I've developed tons of modules and test them on Windows and have quite honestly never needed to dive into that directory because npm provides commands for everything you should ever need to do to that directory, and it supports long paths just fine.
So I clearly must be missing some part of your guys' app development flow because as a module author it's never been a problem for me.
I think an example was just provided of that. Just because it doesn't
disrupt your flow, or break any of your cases doesn't excuse core utilities
being broken. There needs to be at minimum a workaround devised for this.
It's ridiculous to not want to correct this. Unless we really just don't
want to support windows.
On Jun 9, 2014 8:33 PM, "Nathan Rajlich" notifications@github.com wrote:
See, here's what I don't get:
We're talking about the node_modules directory here. It's machine-created
(via the npm client), and for all practical purposes should never be
needed to be traversed. I've developed tons of modules and test them on
Windows and have quite honestly never needed to dive into that directory
because npm provides commands for everything you should ever need to do to
that directory, and it supports long paths just fine.So I clearly must be missing some part of your guys' app development flow
because as a module author it's never been a problem for me.—
Reply to this email directly or view it on GitHub
#6960 (comment).
None of us here (I hope) would deliver a system to a paying customer that
exhibited this behavior and claim that it supported Windows.
On Jun 9, 2014 8:40 PM, "Daniel Taylor" dantaylor08@gmail.com wrote:
I think an example was just provided of that. Just because it doesn't
disrupt your flow, or break any of your cases doesn't excuse core utilities
being broken. There needs to be at minimum a workaround devised for this.
It's ridiculous to not want to correct this. Unless we really just don't
want to support windows.
On Jun 9, 2014 8:33 PM, "Nathan Rajlich" notifications@github.com wrote:See, here's what I don't get:
We're talking about the node_modules directory here. It's
machine-created (via the npm client), and for all practical purposes
should never be needed to be traversed. I've developed tons of modules
and test them on Windows and have quite honestly never needed to dive into
that directory because npm provides commands for everything you should ever
need to do to that directory, and it supports long paths just fine.So I clearly must be missing some part of your guys' app development flow
because as a module author it's never been a problem for me.—
Reply to this email directly or view it on GitHub
#6960 (comment).
@TooTallNate Try npm install mocha --dev on Windows. Its weird behaviour was once a reason behind my pull request TryGhost/node-sqlite3#171, and I have always thought (ever since) that JavaScript modules are fine, but only unless they contain C/C++ addons that are harder to build in a deep directory.
@TooTallNate it isn't necessarily down to traversing the directories, I found this issue when working with NPM and Git - (from memory) I commited node_modules into Git on my Ubuntu server, but when I pulled it down to my Windows dev environment, it couldn't create a directory due to how long the path was.
The Grunt plugin in question was grunt-contrib-imagemin, which had created the following path;
/node_modules/grunt-contrib-imagemin/node_modules/imagemin/node_modules/imagemin-gifsicle/node_modules/gifsicle/node_modules/bin-wrapper/node_modules/download/node_modules/decompress/node_modules/adm-zip/test/assets/attributes_test
I dislike that the problem seems to exist since January and that there isn't yet any solution to this, but only tricks that circumvent or postpone the problem.
It would be nice, if at least someone took a poll to find out, if the community is going to address this problem and if yes, to what priority.
Windows isn't goint to change their max char for filepaths. And the community has to decide, if they want to have Windwos Users or not.
If yes, npm has to change the way it creates directories. If not at least don't say that node is Windows compatibel, because it is not. Of course some will say that it "works", but what is it worth, if I can't push my work to my favourite collaboration plattform, because Windows doesn't let me touch those files?
Please consider changing the way npm works, so that everybody can benefit from Node and npm, and not only Non-Windows Users. (And yes, I dislke Windows a lot for this, but changing the development environment for a plugin, that is installed via npm, seems like a total overkill and it makes me to not want to implement this technology in my current projects.)
@olimortimer you should be sure to enable core.longpaths in your git config to tell it to use long paths:
you should be sure to enable core.longpaths in your git config to tell it to use long paths
note you need git 1.9+
Got hit by this when gyp failed to compile one of the native modules due to long paths.
Even if you don't care about windows, the endless abyss of node modules nesting always baffled me. It's just so inefficient, wasteful and detrimental to load speeds...
Here is my 2¢ on how I think npm install should work:
-
Install every package to root
node_modulesby default -
When 2 semver incompatible versions of the same module are required, install the latest one to the root, and the lower one to the topmost
node_moduleswithout a conflict.You can look at it as playing Tetris with node modules where the tree grows up, and the highest versions should fall to the bottom.
This is simple, greatly increases the efficiency, eliminates 99% of nesting hell, and you don't have to touch require() at all. It'd also make npm dedupe* unneeded and obsolete.
If there would still be a module with big nesting problem (quite unlikely, but possible), it could be solved by module's maintainer updating its dependencies.
* Doing npm_dedupe would not work in my case, as there is nothing to dedupe because native module compilation failed and took the whole module with it. I'd have to find out which module caused it, and install it manually to the root, hoping the dependency abyss isn't too deep and it'll be enough. What fun.
I have created an npm issue if you want to comment on this npm/npm#5458
I would like to throw out a suggestion that hasn't been stated yet. Use the module_name@semver pattern. On top of this, implement a virtual file system in node that allows virtual paths to be mapped to physical paths. Then, just as node already internally redirects file system calls to use UNC paths, file system calls from anywhere to a path within a virtual file system (e.g. a very long file path under node_modules/) would in turn be redirected to the corresponding physical directory, which can now be in a flat structure.
The VFS could also map multiple virtual paths to a single physical path, so that for example multiple copies of the same package in the current deep node_modules tree could be mapped to a single directory in the flat node_modules/module_name@semver physical directory. To make things simpler for npm, the VFS api could allow npm to map virtual files to physical files on the fly. So, npm would create a VFS for all paths under 'node_modules/' and then register file system overrides (to map virtual paths to physical paths and other operations like listing files/directories in a virtual directory). Node would also need to provide the capability to explicitly opt-out of VFS mapping, so that NPM could still manipulate the physical directories under node_modules.
This would solve the path-too-long problem on windows (paths can now be as large as a JS string can contain), eliminate duplication of packages for everyone (always only one copy of a library per version), but would not break existing packages or change any existing external node or npm behavior (e.g. npm update would work the same as before).
Additionally, by implementing the change with a virtual file system, you would limit the changes that would need to be made to fix this windows path issue to just 1) implementation of the VFS functionality, and 2) changes to npm to construct and maintain a VFS for paths under node_modules/. Anything else that uses the file system primitives through node would not even need to be aware of the VFS functionality.
Fyi, if you want to remove very long paths in Windows, you can npm i -g rimraf and then use the rimraf command line bin it creates. Works just like rm -rf on Unix.
@darsain Basically, we need to dedupe up-front at install time. Working on it. It's not trivial.
@isaacs I think @darsain's proposal was dedupe up-front as well as installing dependencies of dependencies "de-duped" (installed at the top level, so long as semver of the dependencies allows for it).
It would lose the portability of being able to move around a node module directory and being confident that its dependencies were moved along with it, but it sounds like it would make Windows dev life easier.
@isaacs does that include using a tmp dir with small pathname for running the install scripts per module? so gyp does not fail in long paths even when the nesting is a problem.
@nathan7 said:
As far as I can tell, nobody is particularly enthusiastic about changing the behaviour of npm or node entirely for Windows alone — a platform few of us use, even fewer of us like, and only a couple of us care about. node has done the correct thing by handling long paths correctly, and at this point it feels like we're being asked to compensate for all the software around us on Windows being crap.
Who are these people that aren't "particularly enthusiastic about changing the behavior of npm or node"? I were a bit chocked when I read your reply, and was wondering if this is solely your own personal opinion, or if its an opinion shared amongst the rest of the npm-team. If it's the latter I must say that my otherwise strong enthusiasm towards the npm as a project has taken somewhat of a hit.
@nathan7
I can only support what teltploek said. In my 15 years developing websites, I've only worked in companies that developed on Windows platforms, and I don't think that is such a rare occurrence.
Speaking from experience, the major web development and communication agencies in my country, use Windows machines to work on.
Be careful of making sweeping generalizations based, it seems, solely on your own preferences and workplace experiences.
That aside, since it seems that there is a real problem with too long path names, wouldn't it be more productive looking at a solution to the problem rather than just saying that it works as intended.
The way NPM handles dependencies is brilliant and a hallmark feature of npm.
If you move all modules to top level node_modules it's too hard to examine the node_modules used by a particular module. Making it more difficult to read/edit/experiment with submodules which is common for developers to do. I would be much less likely to read/edit source for submodules if they were all in one giant directory. It's very common to look in node_modules and see what is in there. This kind of tinkering is critical to the npm ecosystem.
@teltploek @janniknilsson As a developer at npm, Inc., I want npm to work well for you regardless of the platform you're developing on. Windows is a supported platform for Node and npm, and I believe that Node has been a success story on and for Windows. I want to see this continue, and if there are bugs and pain points in npm for Windows, I want to see them ironed out.
At the same time, the way npm and node themselves interact with Windows paths is correct, and was originally designed with input from MS developers. The problem is not with node or npm but with the tooling around them. The problem is painful enough that it's worth having this discussion, but the basic way that node's module system works is not a cowpath so much as an 18-lane paved freeway. Maybe more aggressive flattening a la npmd or npm dedupe will do the job, but moving entirely to a flat directory structure (or doing anything that changes how node's module loader looks up dependencies) is painful enough to be a nonstarter.
And, to a certain extent @nathan7 is right – I think @domenic is the only npm contributor who uses Windows as his development environment, and with the significant exception of people using Azure Web Server, the footprint of Windows as a production deployment environment for Node is minimal. I say this not to diminish or attempt to invalidate your frustrations, but to point out that any changes made to npm or Node to deal with environmental issues on Windows could cause pain for a much larger number of people than those currently affected.
@othiym23 said:
the footprint of Windows as a production deployment environment for Node is minimal
Depends on the definition of the "production deployment environment". Node is not only used as a web app platform: there are command line tools, nodewebkit applications, etc. My guess is that there are significantly more installations of grunt, gulp, less and other utilities written in node.js than production deployed websites written in node.js. Probably even the number of installations of grunt.js alone on windows is higher than the total number of deployed node.js web apps. And for a web developer using grunt for his front-end work, node.js is just a part of the grunt "product" and the fact that the product (or one of its plugins) doesn't work with his established workflow on windows is sad to say at least.
Thank you for understanding and acknowledging our pain. I support your
suggestion to aggressively flatten the directory structure a la npmd or npm
depude into the native npm installment, so that we could at least postpone
the problem until later.
If other people think, that this makes sense for the beginning, then let's
implement it to the next update.
Horyzon - Online Marketing
+49 (0)178 714 2007
georg@horyzon.de
http://s.wisestamp.com/links?url=mailto%3Ageorg%40horyzon.de
www.horyzon.de
http://s.wisestamp.com/links?url=http%3A%2F%2Fwww.horyzon.de
[image: Google Plus Page]
http://s.wisestamp.com/links?url=https%3A%2F%2Fplus.google.com%2Fu%2F0%2Fb%2F113367440655979214836%2F113367440655979214836%2Fposts
[image:
Facebook]
http://s.wisestamp.com/links?url=https%3A%2F%2Fwww.facebook.com%2Fgeorg.geladaris
[image:
YouTube]
http://s.wisestamp.com/links?url=http%3A%2F%2Fwww.youtube.com%2Fchannel%2FUCO_PPRfmqqWsBqAiELkkRNQ
[image:
Twitter]
http://s.wisestamp.com/links?url=https%3A%2F%2Ftwitter.com%2FGeorgGeladaris
2014-06-13 4:13 GMT+02:00 Forrest L Norvell notifications@github.com:
@teltploek https://github.com/teltploek @janniknilsson
https://github.com/janniknilsson As a developer at npm, Inc., I want
npm to work well for you regardless of the platform you're developing on.
Windows is a supported platform for Node and npm, and I believe that
Node has been a success story on and for Windows. I want to see this
continue, and if there are bugs and pain points in npm for Windows, I want
to see them ironed out.At the same time, the way npm and node themselves interact with Windows
paths is correct, and was originally designed with input from MS
developers. The problem is not with node or npm but with the tooling
around them. The problem is painful enough that it's worth having this
discussion, but the basic way that node's module system works is not a
cowpath so much as an 18-lane paved freeway. Maybe more aggressive
flattening a la npmd or npm dedupe will do the job, but moving entirely
to a flat directory structure (or doing anything that changes how node's
module loader looks up dependencies) is painful enough to be a nonstarter.And, to a certain extent @nathan7 https://github.com/nathan7 is right –
I think @domenic https://github.com/domenic is the only npm contributor
who uses Windows as his development environment, and with the significant
exception of people using Azure Web Server, the footprint of Windows as a
production deployment environment for Node is minimal. I say this not to
diminish or attempt to invalidate your frustrations, but to point out that
any changes made to npm or Node to deal with environmental issues on
Windows could cause pain for a much larger number of people than those
currently affected.—
Reply to this email directly or view it on GitHub
#6960 (comment).
@ArtemGovorov said:
Depends on the definition of the "production deployment environment". Node is not only used as a web app platform: there are command line tools, nodewebkit applications, etc. My guess is that there are significantly more installations of grunt, gulp, less and other utilities written in node.js than production deployed websites written in node.js. Probably even the number of installations of grunt.js alone on windows is higher than the total number of deployed node.js web apps. And for a web developer using grunt for his front-end work, node.js is just a part of the grunt "product" and the fact that the product (or one of its plugins) doesn't work with his established workflow on windows is sad to say at least.
I couldn't agree more. Although I was happy to read a more nuanced reply from @othiym23, I still feel that you're, somewhere in there, neglecting the fact that node and the utilities at hand are used in lot of different setups. We, for instance, have a Windows continuous integration server that, among other things, handles and processes our frontend assets using gulp etc., in what has become a somewhat ambitious setup. The long path problem is very much an issue in this setup, although I understand that it isn't directly a problem with node or npm.
@othiym23 as mentioned above, node is not just for web servers. I've personally written more node based desktop apps than I've written servers. And they need to run on windows. The cross-platform nature and the ease of development on node is the reason I chose it for this task in the first place.
And even if it would be only for servers, as I said above, if you don't care about windows, than look at the flattening by default approach described in my post as: Making your node apps more efficient by removing the dependency abyss and countless module duplicates. That in turn results into a smaller memory footprint, and way less fs ios which drastically speeds up loading times.
The inefficient way how node handles dependencies always bother me more than occasional nw-gyp errors anyway. And if fixing that also helps with long paths? Than one more fly swatted by a single stroke, and everyone benefits :)
Off topic
The other issue I have with npm is that publish only needed files is not enforced, just suggested by the existence of .npmignore and package.json:files property, neither of which is really used properly in the wild.
Building apps for desktop than means going through dependencies and removing bloat, which is in many cases 80% of the node_modules directory, if not more. And inlining the whole readme file in package.json by npm also doesn't help.
Just imagine npm install downloading 10MB of 100% needed code instead of 100MB of mostly bloat. Oh the wishes of the lieges.
Wanted to write a module for cleaning all that bloat out, but never got, nor looking forward to it :/
The way how dependencies work right now is great when compared to weird or none at all solutions of other environments, but I'd definitely not call it brilliant and a hallmark feature of npm (quoting @llambda above). At least not yet :)
It's a bit telling that, of easily 50k+ node developers, less than 10 are actively complaining about this. The other 99% must either not be effected or not care.
That's not to say it's not worth thinking of a fix. It absolutely is. Just that the vocal minority should acknowledge that they are in fact the minority here, and will need to respect that any fix made would need to have no negative impact on the experience of the majority users.
If you think it's so important, submit a pull request, or suggest alternatives that don't require breaking changes. Just understand there is more to it than simply adding such functionality. It also needs to be maintained, and both node and npm are sorely lacking in maintainers that use Windows.
Don't be offended that all these suggestions are getting turned down or start making accusations that the devs don't care about Windows. There's only a handful of maintainers and they have limited bandwidth.
It's a bit telling that, of easily 50k+ node developers, less than 10 are actively complaining about this. The other 99% must either not be effected or not care
Or not paying attention to this thread, or complaining elsewhere, or are shy, or affected slightly but not enough to complain, or don't have time, or are watching, or are hoping someone else can fix it, or...
My experience with my own consumer products, is that only an extremely small minority complain about issues that affect them. The rest either don't (for the reasons above), or stop using the product. My assumption is that when a single user is complaining about an issue affecting them, there are likely 100 others that are also affected but not complaining.
The fact that there are so many folks adding on to this enormously long thread is likely an indicator that there are at least thousands of devs affected by this.
For what it's worth, I would love to contribute my time to helping fix this problem on Windows. From what I've read on this thread, there doesn't seem to be a consensus (from the wild or core Node developers) on the solution.
If this is a Windows-dev manpower thing, I'd be happy to donate my time, as I'm sure many others on the thread would be.
It's coming to an agreeable, long-term solution first that appears to be the challenge.
The base architecture of node_modules is ill-conceived to the point of rendering node unsustainable and useless without a serious architectural change. Downloading 30 lo_dash modules mostly with the same version level is not a sustainable direction even if it seems to be the only way to ensure the exact version in any npm bundle is used.
The core flaw in the default architecture — while it can be worked around — is enough to almost ensure that the current version of node will not receive wide-spread enterprise-level use for large applications for some time. Walmart and the gang have done amazing things working around this, but I get the impression it is because they've developed their own plugin architecture hapi instead of using the default node_modules and npm architectures.
I observed architectures come and go for the last 20 years (as Nike's Intranet webmaster and with IBM). I too want to help with this, but in our small private school I've gone back to emphasizing Python, Java, and C# for students focused on large applications as much as I love Node and want to see it succeed. There is no doubt it is the future, just not in it's current form.
While I enjoy hearing everyone's perspectives about the best way to handle package dependencies in abstract terms, I find myself trying to piece together a repeatable problem report around long paths in Windows. Does anyone have such a set of steps?
Without such a replicable situation, it seems that we are unable to move past conflicting apocryphal claims of functionality/brokenness made from personal experience - which I fear will doom this issue to an endless back-and-forth.
After an hour or two this morning, this is the closest thing I can come up with:
-
Starting from a Node folder in my Documents
The base path is c:\Users\username\Documents\Node -
install grunt-contrib-imagemin
This then calls in nine levels of dependencies, which install correctly:
grunt-contrib-imagemin
1- imagemin
1-2- imagemin-gifsicle
1-2-3- gifsicle
1-2-3-4-bin-wrapper
1-2-3-4-5- download
1-2-3-4-5-6- decompress
1-2-3-4-5-6-7- tar
1-2-3-4-5-6-7-8- fstream
1-2-3-4-5-6-7-8-9- graceful-fs
- I'm able to browse to the bottom of this directory via the Windows Explorer UI (Windows 7)
Toward the bottom the path switches to DOS-style names that are limited to eight characters. The full path is:
C:\Users\mjbernha\DOCUME1\Node\NODE_M1\GRUNT-3\NODE_M1\imagemin\NODE_M1\IMAGEM4\NODE_M1\gifsicle\NODE_M1\BIN-WR1\NODE_M1\download\NODE_M1\DECOMP1\NODE_M1\tar\NODE_M1\fstream\node_modules\graceful-fs
- However, when I try to traverse down to the bottom using a command prompt, I am unable to get past fstream - the path at that point is 252 characters long. Switching to DOS-style directory names can get me to the bottom, however.
Caveat:
I haven't tried to actually use anything in this setup yet - that's next, but I need to set this aside for a bit. My hope is that one of the other contributors to this thread can list out a chain of steps to break something under Windows more significantly than I outline above. I'm very much a beginner with Node, trying to learn as much as I can - but I'm happy to contribute what I can.
Since we all agree that this would take some serious dev work and issue tackling, a decision must be taken by the project's maintainers regarding the changes, if any.
Once a decision is taken, I'm sure that the affected non-maintainers would be happy to contribute. Starting to work on this without a prior decision would be under a rejection risk, for bad architecture/design/implementation.
Just make up your mind, make a decision and mark it as low priority. When it's done, all you need to do is review the code.
Has there been any discussion of a node_modules local (and I hate the word) registry? Perhaps just something like packages.json? npm and company could use it to keep the n_m level to 3 tiers.
I think it is important to emphasis that this is not just a Windows problem. It addresses issues that the npm itself has already shown with the hundreds of redundant downloads it has to make.
I find myself trying to piece together a repeatable problem report around long paths in Windows. Does anyone have such a set of steps?
One step seems to be enough:
npm install mocha --dev
Or just try deleting the root node_modules directory via DOS or windows explorer. Or moving it. Or trying to create a git repo with it.
On Jun 13, 2014, at 11:28 AM, Mithgol notifications@github.com wrote:
I find myself trying to piece together a repeatable problem report around long paths in Windows. Does anyone have such a set of steps?
Actually one step is enough:
npm install mocha --dev
—
Reply to this email directly or view it on GitHub.
@robmuh If you read above, @isaacs is working on default deduping on install, which should help the Windows depth issue.
@robmuh If you read above, @isaacs is working on default deduping on install, which should help the Windows depth issue.
As long as you have dependencies that can be deduped, otherwise the issue remains.
Default deduping is great but from my understanding of the issue it's not a permanent fix.
It's not a perfect fix for all use-cases for Windows users, no. But it is a start that would help a lot of Windows users, if I understand the issue correctly.