reimplement process using native
Closed this issue · 28 comments
[@gavinking] We have done system, operatingSystem, and now runtime. I need a volunteer to do process.
[Migrated from ceylon/ceylon.language#754]
[@quintesse] Can we at the same time add {String*} propertyNames() and {String*} environmentVariableNames()?
Although perhaps I'd even prefer: {String->String*} properties() and {String->String*} environmentVariables() because that makes quickly printing out all the values very convenient.
[@gavinking] Well I guess, if you know how to implement those sensibly for JavaScript. But please don't make them methods.
[@gavinking] Assigning to you, @quintesse, because I think you already started work on it.
[@quintesse] No way we can do this now. And who knows if ever, because it would entail removing all of the model loaders from the backend compilers and put them somewhere central (ceylon-model?) so each compiler can load the "binaries" of each of the other backends just so it can check which packages it contains.
[@quintesse] Completely finished code for process can be found here: https://github.com/ceylon/ceylon.language/tree/nativeprocess in case we can ever solve this.
[@gavinking] @quintesse see on gitter where I just proposed an extremely simple solution to the problem.
[@FroMage] Well no, gitter is not the right place to document what should be done in issues. So I'll sum it up here:
Why doesn't the exact same problem happen for native("js") module imports when compiling on the JVM? I'm pretty sure we have the same problem reversed.
We "just" have to have a minimal "module/model loader" that knows how to extract 1/ module imports (we already have that in the CMR) and 2/ packages from modules for other backends then by scanning module imports of other backends (native("jvm") from the js compiler for example), we can see if the package belongs there for the JVM that's pretty trivial to scan the jar I bet it's easy for the JS archives too and yes we can cache that, I expect the problem to only come up in rare cases, and only pay there.
So for instance if we find that we have a missing package, check if we have native module imports for other backends, and then load them to check if the package doesn't come from there. That's the real issue we're trying to fix, right?
The CMR already knows how to load module imports for any backend: JS/JVM/POM/OSGi/JBossModules. All it needs is to be augmented to also load the package lists.
I'm pretty sure we have the same problem reversed.
No, because there is no thing as a "legacy JS module", if you want to use existing JS code you use dynamic and you don't have to import anything.
If on the other hand you import a native("js") module it's packages is always have a name that starts with the name of the module itself so they get matched.
Edit: but that is obviously a hack, so having the possibility to get the real list would definitely improve the situation!
All it needs is to be augmented to also load the package lists.
And it would need to be factored out of the CMR and be generally available, otherwise this wouldn't work when dealing with a flat classpath (AFAIK at least).
[@FroMage]
If on the other hand you import a native("js") module it's packages is always have a name that starts with the name of the module itself so they get matched.
OK that's true. Although, can't we also import node.js modules?
And it would need to be factored out of the CMR and be generally available, otherwise this wouldn't work when dealing with a flat classpath
No, even the flat classpath stuff uses the CMR for module info loading.
[@FroMage] As I said on IRC, this would be a lot more expensive than annotating the package import, and would not scale well to other backends as loading all module imports for native("!js") may require loading more than just the JVM. It would also require that the compiler at hand knows how to load native modules, and we can expect a JS compiler written in Ceylon running on nodejs to not be able to load the JDK modules just to verify what packages they have.
Now, @gavinking said that we'd need package imports to be given a type in the metamodel since that's how annotations are constrained, but I think that's fine since that can just be an empty interface with no runtime instance, as module imports are not reified at runtime.
Although, can't we also import node.js modules?
I don't know, can we? That would be cool but I've never seen any code doing that.
[@thradec] I think we can, at least I'm doing it in testjs, see
https://github.com/ceylon/ceylon-sdk/blob/master/source/com/redhat/ceylon/testjs/tool.ceylon#L68
2015-10-07 16:28 GMT+02:00 Tako Schotanus notifications@github.com:
Although, can't we also import node.js modules?
I don't know, can we? That would be cool but I've never seen any code
doing that.—
Reply to this email directly or view it on GitHub
<#5632#issuecomment-146210806>
.
[@quintesse]
@thradec well yes, but that's runtime, the compiler doesn't know anything about that :)
Edit: meaning you don't import it in the module descriptor, nor do you add any import statements in the code. But it's nice to see how easy it is to load Node.js modules!
[@gavinking] We can interop with node or common JS modules, but that is always using dynamic.
Now that we have scoped imports I resurrected this issue in the hope of finally implementing it fully, but I've encountered another problem I just can't figure out a nice solution for.
So the process toplevel object has a property called arguments which holds the arguments that were passed on the command line at startup. But to make that work in Java we have the main() methods pass their arguments to the process object using an @Ignored method called setupArguments().
The problem is that if we want to rewrite process using Ceylon-only code then we can't do this anymore because there's no such thing as a "hidden shared" method. And even if there were such a thing we have the problem that it gets passed a j.l.String[] which is not a type that the JS backend knows about.
We could just make the main() method pass the arguments to some private class in the language module's private com.redhat.ceylon.compiler.java package and make process get its arguments from that but that would break binary backward compatibility.
So the only crappy solution I can come up with is that the main() method would use introspection to find out on what Ceylon version it's running and either call the current/old/hidden process.setupArguments() or this new someHelperClass.setupArguments().
Are there any other ideas you people can come up with? @gavinking @FroMage @tombentley @lucaswerkmeister @jvasileff
@quintesse is the issue that the native("jvm") can't define a shared method? Could you make it private and use setAccessible()?
I don't think using reflection to find out if there is a setupArguments() method is bad, but... is the question backwards? A 1.3.1 compiled program has:
public static void main(.java.lang.String[] args) {
.ceylon.language.process_.get_().setupArguments(args);
.simple.run_.run();
}As long as we want to support that program, the shipping version of process will need to have setupArguments().
@gavinking we could yes, but unfortunately @jvasileff is right: it would still break BB compatibility. So unless someone can come up with a good idea I'm afraid this still has to be put in the freezer until such a time we break BB compat anyway. :/
Ah, right, sure.
For what it’s worth – I currently use a function to re-set process.arguments in ceylond, so I’d prefer it if that was still possible in some way.
I think @gavinking and I discussed at some point allowing additional non-native shared member declarations in native classes. (Meaning you could add shared members that only exist in that particular backend implementation. We only allow that for non-shared members right now).
Perhaps combining that with an ignore annotation could work? (I never tried to use ignore in Ceylon as we do with @Ignore for Java. It should work, right?)
I think @gavinking and I discussed at some point allowing additional non-native shared member declarations in native classes.
Well the problem here is that I think we actually do want to hide the setupArguments() function from clients written in Ceylon, don't we?
Well the problem here is that I think we actually do want to hide the setupArguments() function from clients written in Ceylon, don't we?
Yeah, that's why I hoped putting an ignore annotation on it would still work, but that might be wishful thinking ;)
OK, finally done, after all this time, and seems to be working fine, though I don't have a good way to test it in a browser.
This change has resulted in the Jenkins build failing for the last 4 days.
This change has resulted in the Jenkins build failing for the last 4 days.
Grrr. Call me crazy, but I actually don't think that ceylon doc should just give up and stop doing work just because it encounters one error while typechecking the code.
@quintesse it should be fixed now, just waiting for the Jenkins build to run overnight.
Thanks.
Apparently you didn't fix the problem hard enough :)