Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Android Studio with KSP: After changing kotlin code, the first assembleDebug task fails with compilation error, 2nd attempt always works fine but takes too long to build #1678

Open
MaratTursynbek opened this issue Jan 10, 2024 · 12 comments

Comments

@MaratTursynbek
Copy link

After upgrading to Android Studio Hedgehog | 2023.1, AGP to 8.2.0, kotlin to 1.9.21 and ksp to 1.9.21-1.0.16 I started to experience following behaviour of IDE:

  1. Change kotlin code
  2. Run the app
  3. Compilation fails
  4. Click run again (no code changes)
  5. Compiles and runs the app but takes a long time to build the project (feels like clean rebuild although only assembleDebug command was given.

As a result, what would take a couple of seconds to compile and run has become apprx 2 minutes between each code changes.

Changing XML code works fine. Incremental build is fast and no problems there. After AS Patch 1 was released with AGP 8.2.1 and kotlin 1.9.22 with corresponding ksp version, the problem is still there.

I would appreciate any help in resolving this problem. Developer experience is now very low and it is getting hard to be productive at work. Is there any settings I'm missing or should I reinstall the macOS and AS all together?

Compilation error stack trace shows different things but this one below is the most frequent among them.

My system info:

Android Studio Hedgehog | 2023.1.1 Patch 1
Build #AI-231.9392.1.2311.11255304, built on December 26, 2023
Runtime version: 17.0.7+0-17.0.7b1000.6-10550314 aarch64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
macOS 14.2
GC: G1 Young Generation, G1 Old Generation
Memory: 4096M
Cores: 12
Metal Rendering is ON
Registry:
    external.system.auto.import.disabled=true
    debugger.new.tool.window.layout=true
    ide.text.editor.with.preview.show.floating.toolbar=false
    ide.instant.shutdown=false
    ide.experimental.ui=true
    id("com.android.application") version "8.2.1" apply false
    id("com.android.library") version "8.2.1" apply false
    id("com.android.test") version "8.2.1" apply false
    id("org.jetbrains.kotlin.android") version "1.9.22" apply false
    id("com.google.devtools.ksp") version "1.9.22-1.0.16" apply false
[ksp] java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 0
	at org.jetbrains.kotlin.com.intellij.util.io.CompressedAppendableFile.loadChunk(CompressedAppendableFile.java:185)
	at org.jetbrains.kotlin.com.intellij.util.io.CompressedAppendableFile.access$100(CompressedAppendableFile.java:30)
	at org.jetbrains.kotlin.com.intellij.util.io.CompressedAppendableFile$FileChunkReadCache.get(CompressedAppendableFile.java:464)
	at org.jetbrains.kotlin.com.intellij.util.io.CompressedAppendableFile$SegmentedChunkInputStream.read(CompressedAppendableFile.java:519)
	at java.base/java.io.DataInputStream.readFully(Unknown Source)
	at org.jetbrains.kotlin.com.intellij.util.io.PersistentHashMapValueStorage$ReaderOverCompressedFile.get(PersistentHashMapValueStorage.java:772)
	at org.jetbrains.kotlin.com.intellij.util.io.PersistentHashMapValueStorage.readBytes(PersistentHashMapValueStorage.java:542)
	at org.jetbrains.kotlin.com.intellij.util.io.PersistentMapImpl.doGet(PersistentMapImpl.java:604)
	at org.jetbrains.kotlin.com.intellij.util.io.PersistentMapImpl.get(PersistentMapImpl.java:545)
	at org.jetbrains.kotlin.com.intellij.util.io.PersistentHashMap.get(PersistentHashMap.java:178)
	at org.jetbrains.kotlin.incremental.storage.CachingLazyStorage.get(CachingLazyStorage.kt:74)
	at com.google.devtools.ksp.FileToFilesMap.get(Incremental.kt:134)
	at com.google.devtools.ksp.IncrementalContext.calcDirtyFiles(Incremental.kt:378)
	at com.google.devtools.ksp.AbstractKotlinSymbolProcessingExtension$doAnalysis$2.invoke(KotlinSymbolProcessingExtension.kt:193)
	at com.google.devtools.ksp.AbstractKotlinSymbolProcessingExtension$doAnalysis$2.invoke(KotlinSymbolProcessingExtension.kt:186)
	at com.google.devtools.ksp.AbstractKotlinSymbolProcessingExtension.handleException(KotlinSymbolProcessingExtension.kt:410)
	at com.google.devtools.ksp.AbstractKotlinSymbolProcessingExtension.doAnalysis(KotlinSymbolProcessingExtension.kt:186)
	at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration(TopDownAnalyzerFacadeForJVM.kt:112)
	at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration$default(TopDownAnalyzerFacadeForJVM.kt:77)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:256)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:247)
	at org.jetbrains.kotlin.cli.common.messages.AnalyzerWithCompilerReport.analyzeAndReport(AnalyzerWithCompilerReport.kt:115)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.analyze(KotlinToJVMBytecodeCompiler.kt:247)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli(KotlinToJVMBytecodeCompiler.kt:87)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli$default(KotlinToJVMBytecodeCompiler.kt:43)
	at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:165)
	at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:50)
	at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:104)
	at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:48)
	at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:101)
	at org.jetbrains.kotlin.daemon.CompileServiceImpl.compile(CompileServiceImpl.kt:1523)
	at jdk.internal.reflect.GeneratedMethodAccessor96.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at java.rmi/sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
	at java.rmi/sun.rmi.transport.Transport$1.run(Unknown Source)
	at java.rmi/sun.rmi.transport.Transport$1.run(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Unknown Source)
	at java.rmi/sun.rmi.transport.Transport.serviceCall(Unknown Source)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Unknown Source)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 0
	at org.jetbrains.kotlin.net.jpountz.util.SafeUtils.checkRange(SafeUtils.java:24)
	at org.jetbrains.kotlin.net.jpountz.util.UnsafeUtils.checkRange(UnsafeUtils.java:56)
	at org.jetbrains.kotlin.net.jpountz.lz4.LZ4JavaUnsafeFastDecompressor.decompress(LZ4JavaUnsafeFastDecompressor.java:23)
	at org.jetbrains.kotlin.net.jpountz.lz4.LZ4FastDecompressor.decompress(LZ4FastDecompressor.java:107)
	at org.jetbrains.kotlin.com.intellij.util.CompressionUtil.readCompressedWithoutOriginalBufferLength(CompressionUtil.java:93)
	at org.jetbrains.kotlin.com.intellij.util.io.CompressedAppendableFile.decompress(CompressedAppendableFile.java:353)
	at org.jetbrains.kotlin.com.intellij.util.io.CompressedAppendableFile.loadChunk(CompressedAppendableFile.java:173)
	... 46 more

@MaratTursynbek MaratTursynbek changed the title Android Studio with KSP: After changing kotlin code, the first assembleDebug task fail with compilation error, 2nd attempt always works fine Android Studio with KSP: After changing kotlin code, the first assembleDebug task fails with compilation error, 2nd attempt always works fine but takes too long to build Jan 10, 2024
@palpapp04
Copy link

I am having a similar issue after migrating dagger to use ksp instead of kapt. My usual failing stack trace is the following:

java.nio.file.FileAlreadyExistsException: C:\....\build\generated\ksp\....debug\java\com\....\MyClass_Factory.java
	at java.base/sun.nio.fs.WindowsFileCopy.copy(WindowsFileCopy.java:124)
	at java.base/sun.nio.fs.WindowsFileSystemProvider.copy(WindowsFileSystemProvider.java:284)
	at java.base/java.nio.file.Files.copy(Files.java:1305)
	at com.google.devtools.ksp.common.IncrementalUtilKt.copyWithTimestamp(IncrementalUtil.kt:80)
	at com.google.devtools.ksp.AbstractKotlinSymbolProcessingExtension.updateFromShadow(KotlinSymbolProcessingExtension.kt:477)
	at com.google.devtools.ksp.AbstractKotlinSymbolProcessingExtension.doAnalysis(KotlinSymbolProcessingExtension.kt:375)
	at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration(TopDownAnalyzerFacadeForJVM.kt:112)
	at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration$default(TopDownAnalyzerFacadeForJVM.kt:77)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:256)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:247)
	at org.jetbrains.kotlin.cli.common.messages.AnalyzerWithCompilerReport.analyzeAndReport(AnalyzerWithCompilerReport.kt:115)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.analyze(KotlinToJVMBytecodeCompiler.kt:247)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.repeatAnalysisIfNeeded(KotlinToJVMBytecodeCompiler.kt:181)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.repeatAnalysisIfNeeded(KotlinToJVMBytecodeCompiler.kt:181)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli(KotlinToJVMBytecodeCompiler.kt:87)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli$default(KotlinToJVMBytecodeCompiler.kt:43)
	at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:165)
	at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:50)
	at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:104)
	at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:48)
	at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:101)
	at org.jetbrains.kotlin.daemon.CompileServiceImpl.compile(CompileServiceImpl.kt:1523)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.rmi/sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:360)
	at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:200)
	at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:197)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
	at java.rmi/sun.rmi.transport.Transport.serviceCall(Transport.java:196)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:587)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:705)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:704)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)

I had your stack trace after migrating dagger to ksp but after the latest dagger release that is gone and this is the new error I am getting. The dagger team told me that since this is a ksp only stack trace I should file a bug with the ksp team. If you are using I suggest trying to upgrade dagger to the latest version but maybe you are getting your error without dagger. In any case I am also commenting to raise awareness of these issues

@soygabimoreno
Copy link

I am experiencing something that seems related.

I am using ksp.incremental=true.

And I have many compilation failures (not always) that fail the first time (and works after building again):

java.nio.file.FileAlreadyExistsException: /<project-path-name>build/app/build/generated/ksp/debug/<package-name>/BuildConfigUtils_Factory.java

I am using:

Gradle: 8.6
Android Gradle Plugin: 8.3.2
Kotlin: 1.9.22
KSP: 1.9.22-1.0.16

This is my environment:

Android Studio Iguana | 2023.2.1
Build #AI-232.10227.8.2321.11479570, built on February 21, 2024
Runtime version: 17.0.9+0-17.0.9b1087.7-11185874 aarch64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
macOS 14.4.1
GC: G1 Young Generation, G1 Old Generation
Memory: 8192M
Cores: 10

@pAlkesz
Copy link

pAlkesz commented May 20, 2024

I am experiencing this issue issue as well. I looked into the source code briefly and to me it seems that the cause of the issue this code snippet:

val dst = File(options.javaOutputDir, File(it.path).toRelativeString(roundDir)) if (dst.isFile || !dst.exists()) { copyWithTimestamp(it, dst, overwrite = false) }

Because ksp calls this copyWithTimestamp function with the false overwrite parameter the copying of the file fails in case there is already an existing file there. As you can see if the dst (destination) is a file and it exists then the if statement will get called and the copying will fail. This causes the build to fail but after one failing build probably gradle empties the cache or some of the build folders so the next one will be successful because the file got deleted. In my opinion the overwrite parameter of the copyWithTimestamp function should be true in this case to resolve this. Although I am not yet a contributor to the ksp project and I am hoping someone who is more expert in the inner workings of ksp will take a quick look at this bug. I might open up a small pr about this if I have the time but I am not 100% sure that what I am proposing is the best solution to the issue

@pAlkesz
Copy link

pAlkesz commented Jun 29, 2024

Just commenting again to mention that the issue is the same with KSP2 and also to raise awareness to this issue which is really annoying and most likely easily fixed. However when I tried to open a pr about it I did not have the necessary permissions and the contribution guide is not clear about how to actually become a contributor so I could not open a pr (but maybe I am missing something).

@eddiecrawford
Copy link

Seeing the same FileAlreadyExistsException stack traces as above:

ksp = 2.0.0-1.0.22
kotlin = 2.0.0
androidGradlePlugin = 8.5.0

@ivtoto
Copy link

ivtoto commented Aug 6, 2024

I'm on Kotlin 2.0.0, KSP 2.0.0-1.0.24 (the same is reproducible with 2.0.20-RC and 2.0.20-RC-1.0.24), and Dagger 2.52, and I'm now having the same/similar issue:

  • the first build passes just fine
  • the second and following builds are failing with java.nio.file.FileAlreadyExistsException
  • if I clean the build (gradle clean) - the first following build passes and then the same issue again with the next ones

The "duplicated class" is found in 3 places:

  • /.../projectFolder/app/build/generated/ksp/buildVariant/java/com/...
  • /.../projectFolder/app/build/generated/ksp/buildVariant/java/byRounds/1.com/...
  • /.../projectFolder/app/build/generated/ksp/buildVariant/java/byRounds/2.com.project/...

Other details:

  • Gradle 8.9
  • AGP 8.5.1
  • reproducible with all Dagger versions from 2.49 to 2.52
java.nio.file.FileAlreadyExistsException: /pathToProjectFolder/app/build/generated/ksp/buildVariant/java/com/**/**/**/**_MembersInjector.java
	at java.base/sun.nio.fs.UnixFileSystem.copy(Unknown Source)
	at java.base/sun.nio.fs.UnixFileSystemProvider.copy(Unknown Source)
	at java.base/java.nio.file.Files.copy(Unknown Source)
	at com.google.devtools.ksp.common.IncrementalUtilKt.copyWithTimestamp(IncrementalUtil.kt:80)
	at com.google.devtools.ksp.AbstractKotlinSymbolProcessingExtension.updateFromShadow(KotlinSymbolProcessingExtension.kt:477)
	at com.google.devtools.ksp.AbstractKotlinSymbolProcessingExtension.doAnalysis(KotlinSymbolProcessingExtension.kt:375)
	at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration(TopDownAnalyzerFacadeForJVM.kt:112)
	at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration$default(TopDownAnalyzerFacadeForJVM.kt:75)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.analyze$lambda$12(KotlinToJVMBytecodeCompiler.kt:373)
	at org.jetbrains.kotlin.cli.common.messages.AnalyzerWithCompilerReport.analyzeAndReport(AnalyzerWithCompilerReport.kt:112)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.analyze(KotlinToJVMBytecodeCompiler.kt:364)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.repeatAnalysisIfNeeded(KotlinToJVMBytecodeCompiler.kt:282)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.repeatAnalysisIfNeeded(KotlinToJVMBytecodeCompiler.kt:282)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.runFrontendAndGenerateIrUsingClassicFrontend(KotlinToJVMBytecodeCompiler.kt:195)
	at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli(KotlinToJVMBytecodeCompiler.kt:106)
	at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:170)
	at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:43)
	at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:103)
	at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:49)
	at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:101)
	at org.jetbrains.kotlin.daemon.CompileServiceImpl.compile(CompileServiceImpl.kt:1555)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at java.rmi/sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
	at java.rmi/sun.rmi.transport.Transport$1.run(Unknown Source)
	at java.rmi/sun.rmi.transport.Transport$1.run(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Unknown Source)
	at java.rmi/sun.rmi.transport.Transport.serviceCall(Unknown Source)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Unknown Source)
	at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)

The only "workaround" that I found is to set ksp.incremental=false

@carstenz
Copy link

Regarding FileAlreadyExistsException and Dagger: It happened to one of our files after switching from kapt to ksp, and I managed to fix it in our project by doing 2 things:

  • Convert code from java to kotlin
  • Move the call to ApplicationComponent.inject(this) to init{}. (It was being called on demand)

Hope that helps someone.

@jonamireh
Copy link
Contributor

jonamireh commented Sep 5, 2024

@ting-yuan copyWithTimestamp was added in #1473; when invoking it, the overwrite parameter is hardcoded to false when it should probably be derived from dst.isFile. I don't have a repro project as it's little tricky to produce but a failing integration test might be easier to write

https://github.com/google/ksp/blob/main/compiler-plugin/src/main/kotlin/com/google/devtools/ksp/KotlinSymbolProcessingExtension.kt#L477

Edit: Published a patch of the suggested change to mavenLocal() and tested with our project; it seems to work, I've done a few incremental builds and I haven't run into this issue again

@jonamireh
Copy link
Contributor

jonamireh commented Sep 6, 2024

@ting-yuan copyWithTimestamp was added in #1473; when invoking it, the overwrite parameter is hardcoded to false when it should probably be derived from dst.isFile. I don't have a repro project as it's little tricky to produce but a failing integration test might be easier to write

Actually, I think applying that recommendation is only solving the symptom, not the problem. The actual problem is that since the baseDir for java files now has a parent of javaShadowBase (eg java/byRounds/$round), the CodeGeneratorImpl is associating the source file to the generated file with that parent, instead of the final file, which has a different parent.

The final file is determined in updateFromShadow when the files in javaShadowBase are copied & then deleted at the end of processing, changing their parent. However, since we associated the source files to the now-deleted files within javaShadowBase, those files are presumably restored on the next compilation, causing the javaShadowBase of the current compilation to include rounds it didn't create.

For example, the first compilation might create java/byRounds/1, associate the source files to generated files within this directory, copy them to java/ and then delete byRounds. On the second compilation, java/byRounds/1 is restored, the compilation might create java/byRounds/2 to generate the same files, and then in updateFromShadow, we iterate over all directories in byRounds, causing this exception since we expect each round directory to contain mutually exclusive files.

Inspecting our build after the error occurs in our project shows that our build directory still contains:

  • java/byRounds/1 (deleted after the previous compilation but restored from cache due to this bug)
  • java/byRounds/7 (actually created during this compilation)

and each round contains the offending file, even though the CodeGeneratorImpl should make that impossible, as you can't write the same file twice

@kevinguitar
Copy link

We're facing the same issue as well, the workaround is to disable ksp.incremental temporarily, but it makes our non-abi incremental build speed 2x slower, would be nice to know the timeframe when the fix will be released 🙏

@lauzadis
Copy link

We've encountered a similar issue in our project. Invoking kspCommonMainKotlinMetadata would cause our build to alternate between success and failure. The files we are writing with CodeGenerator.createNewFile were turning into directories, and the next invocation would turn them back into files, and this repeats for subsequent invocations of the task.

Disabling ksp.incremental did not work, we had to disable KSP2: ksp.useKSP2 = false

@jsoberg
Copy link

jsoberg commented Nov 15, 2024

Hey @jonamireh, thanks so much for the insight above! this is the offending line that you noted in your first comment, correct?

This is my first time investigating the KSP source so it may be a naive approach, but this issue has been plaguing my team for quite awhile so I added a PR addressing the symptom that you talked about here, which makes sense to me looking at the point where the exception seems to get thrown (Windows example).

If you have any more insight on the ultimate source problem from your discussion here please let me know; I'll continue digging into the source to see if I can find the originating problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests