LinkageError when initializing a non-mc library

We have been debugging this issue for few weeks now where the modloader fails to load a non-mc lib because of a loader constraint violation
We are using this with Architectury to have a multiloader, we use the kotlin for fabric library and we have no issues
We are using reflections to load objects at runtime

Code

        Reflections(
            ConfigurationBuilder()
                .addUrls(ClasspathHelper.forJavaClassPath())
                .addUrls(ClasspathHelper.forClassLoader())
                .addScanners(Scanners.SubTypes)
        ).getSubTypesOf(Module::class.java).forEach { moduleClass ->
            moduleClass.declaredFields.find {
                it.name == "INSTANCE"
            }?.apply {
                isAccessible = true
                (get(null) as? Module)?.let { module ->
                    modules.add(module)
                }
            }
        }

Minified build script

val includeLib: Configuration by configurations.creating
val includeMod: Configuration by configurations.creating

fun DependencyHandlerScope.setupConfigurations() {
    includeLib.dependencies.forEach {
        forgeRuntimeLibrary(it)
        include(it)
    }

    includeMod.dependencies.forEach {
        forgeRuntimeLibrary(it)
        include(it)
    }
}

dependencies {
    // Forge API
    forge("net.minecraftforge:forge:$forgeVersion")

    // Add dependencies on the required Kotlin modules.
    includeLib("org.reflections:reflections:0.10.2")
    includeLib("org.javassist:javassist:3.28.0-GA")
    includeLib("com.github.caoimhebyrne:KDiscordIPC:$discordIPCVersion") { isTransitive = false }

    // Add mods to the mod jar
    includeMod("thedarkcolour:kotlinforforge:$kotlinForgeVersion")

    // Fix KFF
    compileOnly(kotlin("stdlib"))
    
    // Finish the configuration
    setupConfigurations()
}

Error

java.lang.LinkageError: loader constraint violation: loader 'SECURE-BOOTSTRAP' @52563b54 wants to load interface kotlin.jvm.functions.Function0. A different interface with the same name was previously loaded by 'LAYER PLUGIN' @13596c4e. (kotlin.jvm.functions.Function0 is in module thedarkcolour.kotlinforforge@4.10.0-all of loader 'LAYER PLUGIN' @13596c4e, parent loader 'SECURE-BOOTSTRAP' @52563b54)
	at java.lang.ClassLoader.defineClass1(Native Method) ~[?:?] {}
	at java.lang.ClassLoader.defineClass(ClassLoader.java:1012) ~[?:?] {}
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150) ~[?:?] {}
	at net.minecraftforge.securemodules.SecureModuleClassLoader.readerToClass(SecureModuleClassLoader.java:482) ~[securemodules-2.2.10.jar:2.2.10] {}
	at net.minecraftforge.securemodules.SecureModuleClassLoader.findClass(SecureModuleClassLoader.java:399) ~[securemodules-2.2.10.jar:2.2.10] {}
	at net.minecraftforge.securemodules.SecureModuleClassLoader.loadClass(SecureModuleClassLoader.java:415) ~[securemodules-2.2.10.jar:2.2.10] {}
	at java.lang.ClassLoader.loadClass(ClassLoader.java:520) ~[?:?] {}
	at dev.cbyrne.kdiscordipc.KDiscordIPC.<init>(KDiscordIPC.kt:46) ~[KDiscordIPC-0.2.2.jar:?] {}
	at *.DiscordRPC.<clinit>(DiscordRPC.java:66) ~[main/:?] {re:classloading}
	at jdk.internal.misc.Unsafe.ensureClassInitialized0(Native Method) ~[?:?] {}
	at jdk.internal.misc.Unsafe.ensureClassInitialized(Unsafe.java:1155) ~[?:?] {}
	at jdk.internal.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:42) ~[?:?] {}
	at jdk.internal.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:185) ~[?:?] {}
	at java.lang.reflect.Field.acquireFieldAccessor(Field.java:1132) ~[?:?] {}
	at java.lang.reflect.Field.getFieldAccessor(Field.java:1113) ~[?:?] {}
	at java.lang.reflect.Field.get(Field.java:425) ~[?:?] {}
	at *.ModuleRegistry.load(ModuleRegistry.java:42) ~[main/:?] {re:classloading}
	at *.core.Loader.initialize(Loader.java:33) ~[main/:?] {re:classloading}
	at *.initialize(Mod.java:38) ~[main/:?] {re:mixin,re:classloading}
	at ForgeLoader.<clinit>(ForgeLoader.kt:13) ~[%230!/:?] {re:classloading}

Do you have a GitHub repo to test with?

Do you have a GitHub repo to test with?

We want to keep our project private until the initial release, I can try to reproduce the error in a minimal environment if you wish

That works for me

I was able to recreate the error, make sure to use the intellij thing when launching

https://github.com/Edouard127/ClassLoaderError

Is there a followup to this ?
The error happens when it constructs a mod, so I guess it fails to load the library jar

KotlinForForge/forge/kfflang/src/main/kotlin/thedarkcolour/kotlinforforge/KotlinModContainer.kt

Lines 61 to 69 in 97a99c4

    
           private fun constructMod() { 
        
               try { 
        
                   LOGGER.trace(Logging.LOADING, "Loading mod instance ${getModId()} of type ${modClass.name}") 
        
                   modInstance = modClass.kotlin.objectInstance ?: modClass.getDeclaredConstructor().newInstance() 
        
                   LOGGER.trace(Logging.LOADING, "Loaded mod instance ${getModId()} of type ${modClass.name}") 
        
               } catch (throwable: Throwable) { 
        
                   LOGGER.error(Logging.LOADING, "Failed to create mod instance. ModID: ${getModId()}, class ${modClass.name}", throwable) 
        
                   throw ModLoadingException(modInfo, ModLoadingStage.CONSTRUCT, "fml.modloading.failedtoloadmod", throwable, modClass) 
        
               }

I haven't tested this yet but a similar issue has been happening for a while. When a library that uses kotlin libs, like okhttp, is loaded, it tries to load classes from the same module layer instead of all module layers. Fixing one issue of the duplicate module exports means that kotlin libs are in the SERVICE layer, when OkHttp is trying to load from GAME layer.

That would make sense, kdiscordipc uses kotlin
Is there a way to tell the loader to always use the service layer when it's a non-mc library ?

Not currently, because nonmclibs are supposed to be in the game layer. The reason Kotlin has them in the service layer is because it needs Kotlin libs to load Kotlin mod classes with reflection, and the IModLanguageProvider service runs in the Service layer. To force the Kotlin libs to be in the service layer, KFF shades them rather than JarJar. I wonder why OkHttp doesn't just pull from the service layer, since it should be accessible to the game layer?

But shading a non-mc lib can cause compatibility issues in the future, is this even fixable at the moment ?

In arch loom, minecraftLibraries forces a dependency to load in the GAME layer
When using it I get a class not found error instead of the layer error

I have tried this

        forgeRuntimeLibrary(it)
        minecraftLibraries(it)
        include(it)

To include and load in the game layer but get this error
Caused by: java.lang.module.ResolutionException: Modules thedarkcolour.kotlinforforge and kotlin.stdlib export package kotlin.io to module thedarkcolour.kotlinforforge.lang

This snippet will allow the game to load with OkHttp, you might be able to adapt it to your needs:

dependencies {
    minecraft "com.mojang:minecraft:${project.minecraft_version}"
    mappings loom.officialMojangMappings()

    forge "net.minecraftforge:forge:${project.forge_version}"

    implementation "thedarkcolour:kotlinforforge:${project.kff_version}"

    implementation('com.squareup.okhttp3:okhttp:4.12.0') {
        exclude group: 'org.jetbrains.kotlin'
    }
    include('com.squareup.okhttp3:okhttp:4.12.0') {
        exclude group: 'org.jetbrains.kotlin'
    }
    forgeRuntimeLibrary ('com.squareup.okhttp3:okhttp:4.12.0') {
        exclude group: 'org.jetbrains.kotlin'
    }
}

It prevents the mod from loading
I checked the library and the kotlin module isn't there

Exception in thread "main" java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at net.minecraftforge.bootstrap.Bootstrap.start(Bootstrap.java:48)
	at net.minecraftforge.bootstrap.ForgeBootstrap.main(ForgeBootstrap.java:18)
	at juuxel.unionrelauncher.UnionRelauncher.main(UnionRelauncher.java:92)
	at net.fabricmc.devlaunchinjector.Main.main(Main.java:86)
	at dev.architectury.transformer.TransformerRuntime.main(TransformerRuntime.java:219)
Caused by: java.lang.module.FindException: Module kotlin.stdlib not found, required by kotlinx.serialization.core
	at java.base/java.lang.module.Resolver.findFail(Resolver.java:893)
	at java.base/java.lang.module.Resolver.resolve(Resolver.java:192)
	at java.base/java.lang.module.Resolver.resolve(Resolver.java:141)
	at java.base/java.lang.module.Configuration.resolveAndBind(Configuration.java:492)
	at java.base/java.lang.module.Configuration.resolveAndBind(Configuration.java:298)
	at net.minecraftforge.bootstrap@2.0.0/net.minecraftforge.bootstrap.Bootstrap.moduleMain(Bootstrap.java:77)
	... 9 more

Had to exclude kotlinx and gradle clean
And the end result is... linkage error

Add another exclusion for kotlinx

Did you find additional information about the kotlin layer issue ?

I couldn't figure out the issue so far, but I am very busy this week so I can try again a little more in depth this weekend.

That's all fine

After some investigating in Forge's discord server and github issues I have stumbled upon your Forge issue MinecraftForge/MinecraftForge#8878

Because we can't fix the issue until forge does it, I will leave this thread open and come back once fixed

Code reference for PRODUCTION only

fun DependencyHandlerScope.setupConfigurations() {
    shadowBundle.dependencies.forEach {
        shadowCommon(it)
        shadow(it)
    }
    ....
}

dependencies {
    shadowBundle("com.github.caoimhebyrne:KDiscordIPC:$discordIPCVersion") {
        exclude(group = "org.jetbrains.kotlin")
        exclude(group = "org.jetbrains.kotlinx")
        exclude(group = "org.slf4j")
    }
    ....
}

The cause of this issue is the same as in #86, so I'll just close this as duplicate and link to it

	private fun constructMod() {
	try {
	LOGGER.trace(Logging.LOADING, "Loading mod instance ${getModId()} of type ${modClass.name}")
	modInstance = modClass.kotlin.objectInstance ?: modClass.getDeclaredConstructor().newInstance()
	LOGGER.trace(Logging.LOADING, "Loaded mod instance ${getModId()} of type ${modClass.name}")
	} catch (throwable: Throwable) {
	LOGGER.error(Logging.LOADING, "Failed to create mod instance. ModID: ${getModId()}, class ${modClass.name}", throwable)
	throw ModLoadingException(modInfo, ModLoadingStage.CONSTRUCT, "fml.modloading.failedtoloadmod", throwable, modClass)
	}