[KSP] dagger-android hanging indefinitely on a large multi module project
wbonnefond opened this issue · 2 comments
wbonnefond commented
I've attempted to switch our project using dagger-android from KAPT to KSP, but have run into issues. After manually updating all internal provides/binds/contributes methods to be public to address this issue: #4193, the project will hang indefinitely on the kspDebugKotlin task in the app/root module. After updating to Kotlin 1.9.22 and KSP 1.9.22-1.0.17 which addresses the internal method issue, the same behavior occurs. It appears that the dagger KSP processor may be getting into an infinite loop somewhere. For reference, this app consists of ~300 modules.
I built dagger from source to try to add some additional logging, but it looks like logs don't reach gradle output until the task ends (I manually kill it) so it makes debugging very difficult. I'm looking for some insight into where the best place to add additional debug logs would be, or if there are any hunches on where the problem might lie.
This is reproducible with both Dagger 2.49 and 2.50
eboudrant commented
We can reproduce the issue, I was able to get the potential deadlock stacks (from YourKit), see here.
We use Dagger 2.49 (and made the switch to ksp recently).
The issue don't happen to everyone which is strange.
We also have ~300 modules.
kspDebugKotlin is getting stuck in random modules using dagger.
RMI TCP Connection(558)-127.0.0.1 Runnable CPU usage on sample: 1s 2ms
java.io.BufferedInputStream.read1(BufferedInputStream.java:282)
java.io.BufferedInputStream.read(BufferedInputStream.java:343)
java.io.DataInputStream.readFully(DataInputStream.java:201)
java.io.DataInputStream.readFully(DataInputStream.java:172)
org.jetbrains.kotlin.com.intellij.util.io.IOUtil.readString(IOUtil.java:41)
com.google.devtools.ksp.FileKeyDescriptor.read(PersistentMap.kt:39)
com.google.devtools.ksp.FileKeyDescriptor.read(PersistentMap.kt:37)
org.jetbrains.kotlin.com.intellij.util.io.keyStorage.AppendableStorageBackedByResizableMappedFile.read(AppendableStorageBackedByResizableMappedFile.java:94)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.findValueFor(PersistentEnumeratorBase.java:438)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.lambda$valueOf$2(PersistentEnumeratorBase.java:430)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase$$Lambda$617.0x000000080192b290.compute()
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.catchCorruption(PersistentEnumeratorBase.java:608)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.valueOf(PersistentEnumeratorBase.java:429)
org.jetbrains.kotlin.com.intellij.util.io.PersistentBTreeEnumerator.valueOf(PersistentBTreeEnumerator.java:667)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.isKeyAtIndex(PersistentEnumeratorBase.java:373)
org.jetbrains.kotlin.com.intellij.util.io.PersistentBTreeEnumerator.enumerateImpl(PersistentBTreeEnumerator.java:528)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.lambda$doEnumerate$0(PersistentEnumeratorBase.java:267)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase$$Lambda$572.0x00000008018a3008.compute()
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.catchCorruption(PersistentEnumeratorBase.java:608)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.doEnumerate(PersistentEnumeratorBase.java:266)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.enumerate(PersistentEnumeratorBase.java:279)
org.jetbrains.kotlin.com.intellij.util.io.PersistentMapImpl.enumerate(PersistentMapImpl.java:425)
org.jetbrains.kotlin.com.intellij.util.io.PersistentMapImpl.doPut(PersistentMapImpl.java:398)
org.jetbrains.kotlin.com.intellij.util.io.PersistentMapImpl.put(PersistentMapImpl.java:360)
org.jetbrains.kotlin.com.intellij.util.io.PersistentHashMap.put(PersistentHashMap.java:102)
com.google.devtools.ksp.PersistentMap.set(PersistentMap.kt:26)
com.google.devtools.ksp.IncrementalContextBase.updateCaches$update(IncrementalContextBase.kt:371)
com.google.devtools.ksp.IncrementalContextBase.updateCaches(IncrementalContextBase.kt:390)
com.google.devtools.ksp.IncrementalContextBase.updateCachesAndOutputs(IncrementalContextBase.kt:473)
com.google.devtools.ksp.AbstractKotlinSymbolProcessingExtension.doAnalysis(KotlinSymbolProcessingExtension.kt:358)
org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration(TopDownAnalyzerFacadeForJVM.kt:112)
org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration$default(TopDownAnalyzerFacadeForJVM.kt:77)
org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:256)
org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:247)
org.jetbrains.kotlin.cli.common.messages.AnalyzerWithCompilerReport.analyzeAndReport(AnalyzerWithCompilerReport.kt:115)
org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.analyze(KotlinToJVMBytecodeCompiler.kt:247)
org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.repeatAnalysisIfNeeded(KotlinToJVMBytecodeCompiler.kt:181) <2 recursive calls>
org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli(KotlinToJVMBytecodeCompiler.kt:87)
org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli$default(KotlinToJVMBytecodeCompiler.kt:43)
org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:165)
org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:50)
org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:104)
org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:48)
org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:101)
org.jetbrains.kotlin.daemon.CompileServiceImpl.compile(CompileServiceImpl.kt:1523)
jdk.internal.reflect.GeneratedMethodAccessor96.invoke()
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:568)
sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:360)
sun.rmi.transport.Transport$1.run(Transport.java:200)
sun.rmi.transport.Transport$1.run(Transport.java:197)
java.security.AccessController.executePrivileged(AccessController.java:807)
java.security.AccessController.doPrivileged(AccessController.java:712)
sun.rmi.transport.Transport.serviceCall(Transport.java:196)
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:587)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:705)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$90.0x0000000801085528.run()
java.security.AccessController.executePrivileged(AccessController.java:776)
java.security.AccessController.doPrivileged(AccessController.java:399)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:704)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
java.lang.Thread.run(Thread.java:833)
RMI TCP Connection(565)-127.0.0.1 Parked CPU usage on sample: 0ms
jdk.internal.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:211)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:715)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:938)
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:959)
org.jetbrains.kotlin.com.intellij.util.io.StorageLockContext.lockWrite(StorageLockContext.java:70)
org.jetbrains.kotlin.com.intellij.util.io.PagedFileStorage.lockWrite(PagedFileStorage.java:94)
org.jetbrains.kotlin.com.intellij.util.io.ResizeableMappedFile.lockWrite(ResizeableMappedFile.java:279)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.lockStorageWrite(PersistentEnumeratorBase.java:229)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumeratorBase.<init>(PersistentEnumeratorBase.java:137)
org.jetbrains.kotlin.com.intellij.util.io.PersistentBTreeEnumerator.<init>(PersistentBTreeEnumerator.java:93)
org.jetbrains.kotlin.com.intellij.util.io.PersistentEnumerator.createDefaultEnumerator(PersistentEnumerator.java:66)
org.jetbrains.kotlin.com.intellij.util.io.PersistentMapImpl.<init>(PersistentMapImpl.java:136)
org.jetbrains.kotlin.com.intellij.util.io.PersistentMapBuilder.buildImplementation(PersistentMapBuilder.java:62)
org.jetbrains.kotlin.com.intellij.util.io.PersistentMapBuilder.build(PersistentMapBuilder.java:44)
org.jetbrains.kotlin.com.intellij.util.io.PersistentHashMap.<init>(PersistentHashMap.java:39)
org.jetbrains.kotlin.com.intellij.util.io.PersistentHashMap.<init>(PersistentHashMap.java:54)
org.jetbrains.kotlin.incremental.storage.CachingLazyStorage.createMap(CachingLazyStorage.kt:128)
org.jetbrains.kotlin.incremental.storage.CachingLazyStorage.getStorageIfExists(CachingLazyStorage.kt:47)
org.jetbrains.kotlin.incremental.storage.CachingLazyStorage.getKeys(CachingLazyStorage.kt:65)
org.jetbrains.kotlin.incremental.storage.LookupMap.getKeys(LookupMap.kt:56)
org.jetbrains.kotlin.incremental.TrackedLookupMap.getKeys(LookupStorage.kt:306)
org.jetbrains.kotlin.incremental.LookupStorage.getLookupSymbols(LookupStorage.kt:89)
org.jetbrains.kotlin.incremental.classpathDiff.ClasspathSnapshotShrinker$shrinkClasspath$lookupSymbols$1.invoke(ClasspathSnapshotShrinker.kt:37)
org.jetbrains.kotlin.incremental.classpathDiff.ClasspathSnapshotShrinker$shrinkClasspath$lookupSymbols$1.invoke(ClasspathSnapshotShrinker.kt:36)
org.jetbrains.kotlin.incremental.classpathDiff.ClasspathSnapshotShrinker$MetricsReporter.getLookupSymbols(ClasspathSnapshotShrinker.kt:330)
org.jetbrains.kotlin.incremental.classpathDiff.ClasspathSnapshotShrinker.shrinkClasspath(ClasspathSnapshotShrinker.kt:36)
org.jetbrains.kotlin.incremental.classpathDiff.ClasspathChangesComputer.computeClasspathChanges(ClasspathChangesComputer.kt:50)
org.jetbrains.kotlin.incremental.IncrementalJvmCompilerRunner.calculateSourcesToCompileImpl(IncrementalJvmCompilerRunner.kt:197)
org.jetbrains.kotlin.incremental.IncrementalJvmCompilerRunner.calculateSourcesToCompile(IncrementalJvmCompilerRunner.kt:135)
org.jetbrains.kotlin.incremental.IncrementalJvmCompilerRunner.calculateSourcesToCompile(IncrementalJvmCompilerRunner.kt:62)
org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.tryCompileIncrementally$lambda$6$compile(IncrementalCompilerRunner.kt:207)
org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.tryCompileIncrementally(IncrementalCompilerRunner.kt:247)
org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.compile(IncrementalCompilerRunner.kt:116)
org.jetbrains.kotlin.daemon.CompileServiceImplBase.execIncrementalCompiler(CompileServiceImpl.kt:657)
org.jetbrains.kotlin.daemon.CompileServiceImplBase.access$execIncrementalCompiler(CompileServiceImpl.kt:105)
org.jetbrains.kotlin.daemon.CompileServiceImpl.compile(CompileServiceImpl.kt:1620)
jdk.internal.reflect.GeneratedMethodAccessor96.invoke()
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:568)
sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:360)
sun.rmi.transport.Transport$1.run(Transport.java:200)
sun.rmi.transport.Transport$1.run(Transport.java:197)
java.security.AccessController.executePrivileged(AccessController.java:807)
java.security.AccessController.doPrivileged(AccessController.java:712)
sun.rmi.transport.Transport.serviceCall(Transport.java:196)
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:587)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:705)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$90.0x0000000801085528.run()
java.security.AccessController.executePrivileged(AccessController.java:776)
java.security.AccessController.doPrivileged(AccessController.java:399)
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:704)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
java.lang.Thread.run(Thread.java:833)
bcorso commented
@eboudrant thanks for getting the stack traces. It looks like this is stuck on the KSP side. Do you mind filing a bug with KSP team and giving them these stack traces?