J2CL Now & Next

Question

J2CL Now & Next

gkdn opened this issue 4 years ago · 23 comments

Answer 1 · 2020-09-15T11:39:25.000Z

Are you planning gradle support ? Bazel is not known to any IDE.

Answer 2 · 2020-09-15T14:26:03.000Z

https://docs.bazel.build/versions/master/ide.html ?
(my main problem with the IntelliJ IDEA plugin though is that it's currently not compatible with latest IDEA 2020.2: bazelbuild/intellij#1998)

Answer 3 · 2020-12-10T05:38:58.000Z

i would love to see some gRPC support on this list :)

Answer 4 · 2020-12-23T13:30:00.000Z

Are there any workarounds how to run tests, before j2cl_test ll be ported ? It's a critical feature

Answer 5 · 2021-10-15T08:36:35.000Z

any updates here?

Answer 6 · 2021-10-15T19:50:27.000Z

I update it every now and then but comment date doesn't reflect that (you need to click edited to see the change dates).

Answer 7 · 2021-12-02T20:56:03.000Z

Any update on Kotlin support? That would be amazing and java projects are increasingly hybridized with Kotlin

Answer 8 · 2021-12-14T02:26:12.000Z

We just started Kotlin work. Our goal is to have something usable by end of the year.

Answer 9 · 2021-12-14T20:28:55.000Z

@gkdn that is amazing to hear but Kotlin JS is officially developed and has an excellent and mature support for transpiling Kotlin to JS (and generating typescript types)
However while kotlin multiplarform/js is great, and while j2cl allow to transpile Java to js, I feel like the main area for improvement would be for J2CL to leverage existing Kotlin js transpiling ability and to be interoperable with the generated js from Java.
If you could work on making it easier to have hybrid Kotlin-Java projects that target js that would be amazing, maybe that the biggest issue is about IDE supports at the boundaries (Kotlin js will not allow calling java code even if this java code transpile to J2CL). Improving this IDE interop between Java and Kotlin JS should be possible in theory, either by contributing directly to the Kotlin js compiler or by making a Kotlin compiler plugin (note that the replacement API for the new Kotlin compiler (K2) is in active development https://youtrack.jetbrains.com/issue/KT-49508
)
Also you might consider using arrow-meta which is a library that helps building compiler plugins.
https://github.com/arrow-kt/arrow-meta
BTW since you work at Google, maybe you could contact one of the official Google kotlin devs, as you can see here: https://kotlinlang.org/docs/kotlin-foundation.html#current-personnel
for some guidance.
The best would be to contact Roman Elizarov or to open a youtrack ticket for a collaboration :)
I really hope there is the possibility of making a synergy with JetBrains!
Anyway just my two cents on how to achieve the best of both worlds.

BTW: I concur that Gradle support would be a step in the right direction for adoption.

Answer 10 · 2021-12-16T01:39:05.000Z

Our high demanding customers relies on very Closure style centric Google JavaScript stack (which J2CL is optimized for) and they count every single byte for performance. Unfortunately Kotlin/JS output doesn't match our needs and sometimes in a way that might be contradicting with the Kotlin open-source users needs who rely on different JavaScript stacks.

So for now our main focus we will be enabling Kotlin to those customers. Being said that I already met with Roman on this and we are in contact with JetBreains. We will be still looking into bringing the two worlds together in the long run.

Answer 11 · 2021-12-16T12:15:11.000Z

@gkdn I suppose you could apply a post processing step to the generated kotlin-js by feeding it to Closure. You could also work on improving the efficiency of the IR backend.
But sure feel free to implement your own incompatible transpiler.

Answer 12 · 2022-10-22T20:51:44.000Z

@gkdn just in case, do you have any updates about testing support for Open Source ?

Answer 13 · 2022-10-25T20:54:14.000Z

We have an intern working on it at the moment but we will see how it will turn out.

Answer 14 · 2022-10-25T21:32:37.000Z

We have an intern working on it at the moment but we will see how it will turn out.

ok, and i suppose you have a deadline right ?

Answer 15 · 2022-10-25T22:00:56.000Z

Should be within 2 months; if the project is successful.

Answer 16 · 2023-02-02T06:49:02.000Z

Should be within 2 months; if the project is successful.

It looks like it failed ... So pity :(

Answer 17 · 2023-02-04T00:03:24.000Z

Actually basic unit testing support is released, sample app and getting started code is updated accordingly.

I haven't closed the task since it still doesn't support test suites but I will update the notes.

Edit: updated

Answer 18 · 2024-01-31T07:58:37.000Z

@gkdn Do you plan to add support for the following methods from Java 9-11?
Stream: takeWhile, dropWhile, ofNullable, and iterate
String: isBlank, lines, repeat, strip, stripLeading, and stripTrailing
List/Set/Map: copyOf
Collectors: toUnmodifiable*

And if not, can we make a PR?

Answer 19 · 2024-01-31T20:16:07.000Z

We do accept contributions for missing jre methods.

Note that the implementations need to :

be supportable in the web (e.g. java.net.*, reflective apis, etc, are no supported, ).
not impose onerous codesize increases (e.g. most functionality of Locale is not supported due to this).
be created independently of existing implementations in other sdks.

Isn't String.repeat is already in the JRE?

Answer 20 · 2024-02-01T02:06:31.000Z

String.repeat is indeed already implemented - but can you direct us how to run its tests? (I assume tests are required for such submissions, even though it wasn't on your list ;). )

I see that jre/javatests/com/google/j2cl/jre/java/lang/StringTest.java has a testRepeat method, but there don't seem to be any bazel targets to run individual test classes. #93 (comment) says (and seems to still be correct) that suites cannot run, and only jre/javatests/com/google/j2cl/jre/LangSuite.java references StringTest.

Here's the other things I tried, just in case I missed the obvious:
I noticed that LangSuite is referenced by the Lang j2cl_multi_test in jre/javatests/BUILD`, but running that results in

$ bazel test //jre/javatests:Lang                                          
ERROR: Skipping '//jre/javatests:Lang': error loading package 'jre/javatests': Label '//javascript/tools/jscompiler/builddefs:flags.bzl' is invalid because 'javascript/tools/jscompiler/builddefs' is not a package; perhaps you meant to put the colon here: '//:javascript/tools/jscompiler/builddefs/flags.bzl'?
ERROR: error loading package 'jre/javatests': Label '//javascript/tools/jscompiler/builddefs:flags.bzl' is invalid because 'javascript/tools/jscompiler/builddefs' is not a package; perhaps you meant to put the colon here: '//:javascript/tools/jscompiler/builddefs/flags.bzl'?
INFO: Elapsed time: 0.207s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
FAILED: Build did NOT complete successfully (0 packages loaded)
    currently loading: jre/javatests

Typically when we see //javascript labels, or references at all to jscompiler, it means that this isn't available to be run outside of Google. Removing that import line

load("//javascript/tools/jscompiler/builddefs:flags.bzl", "VERBOSE_WARNING_FLAGS_STRICT", "WHITESPACE_ONLY_FLAGS")

and all of the conformance test wiring that depends on it at least advances us to a new error:

$ bazel test //jre/javatests:Lang
ERROR: Skipping '//jre/javatests:Lang': error loading package 'jre/javatests': Label '//third_party/bazel_rules/rules_closure/closure:defs.bzl' is invalid because 'third_party/bazel_rules/rules_closure/closure' is not a package; perhaps you meant to put the colon here: '//third_party:bazel_rules/rules_closure/closure/defs.bzl'?
ERROR: error loading package 'jre/javatests': Label '//third_party/bazel_rules/rules_closure/closure:defs.bzl' is invalid because 'third_party/bazel_rules/rules_closure/closure' is not a package; perhaps you meant to put the colon here: '//third_party:bazel_rules/rules_closure/closure/defs.bzl'?
INFO: Elapsed time: 0.068s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
FAILED: Build did NOT complete successfully (0 packages loaded)
    currently loading: jre/javatests

Fortunately this is also not terrible to fix: there are two separate load() calls for closure_js_binary - removing the second one still leaves the first (though to just run Lang, I've commented out both of them).

Next, I was getting this error:

$ bazel test //jre/javatests:Lang
ERROR: Traceback (most recent call last):
        File "/home/colin/workspace/j2cl/jre/javatests/BUILD", line 85, column 16, in <toplevel>
                j2cl_multi_test(
        File "/home/colin/workspace/j2cl/jre/javatests/j2cl_multi_test.bzl", line 50, column 25, in j2cl_multi_test
                j2kt_native_test(
        File "/home/colin/workspace/j2cl/build_defs/internal_do_not_use/j2kt_test.bzl", line 16, column 21, in j2kt_native_test
                j2cl_test_common(
        File "/home/colin/workspace/j2cl/build_defs/internal_do_not_use/j2cl_test_common.bzl", line 274, column 13, in j2cl_test_common
                fail("Unknown platform: " + platform)
Error in fail: Unknown platform: J2KT-NATIVE

It looks like j2cl_multi_test itself isn't able to run in open source due to J2KT-NATIVE platform not being populated? The code only has branches for CLOSURE and WASM. Next I took the drastic step of removing all j2cl_multi_test (getting past the point of what I could comfortably suggest might even make sense to commit...), and instead wrote a custom test target:

j2cl_test(
    name = "Lang",
    test_class = "com.google.j2cl.jre.LangSuite",
    runtime_deps = [":emul_tests_lib"],
)

This hit compile errors, oddly - it seems that emul_tests_lib itself isn't set up properly? How can this run inside of Google at all? Or do you have entirely separate BUILD files (not just BUILD vs BUILD.bazel), and the checked in BUILD is never used?
To fix, I changed my approach, not trying to run tests at all any more, but just get them to build, via bazel build //jre/javatests:emul_tests_lib.
First I had to add //third_party:jspecify_annotations-j2cl to the deps list. Next, it seems that the list of disabled errorprone checks in emul_tests_lib is inconsistent with currently checked-in code - TreeMapTest.ConflictingKey either needs a SuppressWarnings on it, or a flag passed to errorprone - I opted for -Xep:ComparableType:OFF, as that seems to be the convention.

At this point, emul_tests_lib can succeed in building.

Next, I tried to build (not test) the modified Lang rule above - this failed with a confusing message:

ERROR: /home/colin/workspace/j2cl/jre/javatests/BUILD:129:10: Executing genrule //jre/javatests:genLang_test.js failed: (Exit 1): bash failed: error executing command /bin/bash -c ... (remaining 1 argument skipped)

        FAIL: j2cl_test currently supports testing with a single testsuite only.
        IF YOU HAVE MULTIPLE TESTSUITES, WE DO NOT KNOW IF ALL OF YOUR TESTS PASS OR NOT!

Is it accurate to say that suites don't work, but a single suite would? Regardless, I changed my custom rule to only run StringTest:

j2cl_test(
    name = "StringTest",
    test_class = "com.google.j2cl.jre.java.lang.StringTest",
    runtime_deps = [":emul_tests_lib"],
)

This passed a bazel build finally, but tests fail (specifically testIndexOfNull).

-> 02:03:46.792 : Running test: testIndexOfNull
-> $module$exports$junit$framework$Assert$impl$m_fail__java_lang_String__void$$@http://localhost:50362/filez/com_google_j2cl/jre/javatests/StringTest_bin.js:1832:421
-> $testIndexOfNull$@http://localhost:50362/filez/com_google_j2cl/jre/javatests/StringTest_bin.js:4879:179
[native code]
$invokeFunction_$@http://localhost:50362/filez/com_google_j2cl/jre/javatests/StringTest_bin.js:3869:42
$safeRunTest_$@http://localhost:50362/filez/com_google_j2cl/jre/javatests/StringTest_bin.js:3855:32
[native code]
$goog$testing$TestCase$Continuation_$run$$@http://localhost:50362/filez/com_google_j2cl/jre/javatests/StringTest_bin.js:4226:99
http://localhost:50362/filez/com_google_j2cl/jre/javatests/StringTest_bin.js:3780:167
[native code]
$goog$testing$TestCase$Continuation_$run$$@http://localhost:50362/filez/com_google_j2cl/jre/javatests/StringTest_bin.js:4226:99
http://localhost:50362/filez/com_google_j2cl/jre/javatests/StringTest_bin.js:3778:47
$goog$Promise$$@http://localhost:50362/filez/com_google_j2cl/jre/javatests/StringTest_bin.js:2631:32
$JSCompiler_StaticMethods_runTestsReturningPromise$$@http://localhost:50362/filez/com_google_j2cl/jre/javatests/StringTest_bin.js:3777:22
$JSCompiler_StaticMethods_JSC$2497_execute$$@http://localhost:50362/filez/com_google_j2cl/jre/javatests/StringTest_bin.js:5582:159
http://localhost:50362/filez/com_google_j2cl/jre/javatests/StringTest_bin.js:5714:53
-> 02:03:46.793 : testIndexOfNull : FAILED

Is there an easier way to go about running tests, so that we can migrate already-working java9-11 stream/optional/collection code into j2cl, complete with tests?

Would any/all of this be suitable for a contribution, so that a CI script could exist that runs these emulation tests, in anticipation of emulation patches?

Answer 21 · 2024-02-01T05:30:14.000Z

The JRE unit tests are not ported to open-source so they would not work out of the box.

We are currently swamped with couple of a big projects but if you like to commit to the JRE emulation, the best I can offer at the moment is:

Patch gkdn@9c7e780
Update StringTest in jre/javatests/BUILD with the class that you are testing.
bazel test jre/javatests:StringTest to run the test
bazel run jre/javatests:StringTest_debug and follow the link to run/debug on your browser.

If you have further questions, let's start a discussion in the discussions section.

Answer 22 · 2024-02-01T09:23:01.000Z

@gkdn Thanks a million; it works like a charm.

Answer 23 · 2024-02-01T18:39:22.000Z

Great, thank you @gkdn - different changes than I had made, but clearly solving the same issues. I appreciate the pointer to the _debug test target. @treblereel has submitted #222 with changes from GWT, verified in j2cl by running tests as you described.

Note: