-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Kotlin: Support Kotlin 2.3.0-Beta2 #20965
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds support for Kotlin 2.3.0-Beta2 by updating the rules_kotlin Bazel package from version 2.1.3-codeql.1 to 2.2.0-codeql.1. This update necessarily drops support for Kotlin versions 1.6.x and 1.7.x.
Key Changes:
- Updated
rules_kotlinfrom 2.1.3-codeql.1 to 2.2.0-codeql.1 with updated dependencies and patches - Added support for Kotlin 2.3.0-Beta2 with version-specific compatibility files
- Removed support for Kotlin 1.6.0, 1.6.20, 1.7.0, and 1.7.20 (dependencies and version-specific files)
Reviewed changes
Copilot reviewed 29 out of 29 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
misc/bazel/registry/modules/rules_kotlin/metadata.json |
Updated to reference only the new 2.2.0-codeql.1 version |
misc/bazel/registry/modules/rules_kotlin/2.2.0-codeql.1/source.json |
New source configuration for rules_kotlin 2.2.0 with patch references |
misc/bazel/registry/modules/rules_kotlin/2.2.0-codeql.1/patches/*.patch |
Updated patch files for the new rules_kotlin version |
misc/bazel/registry/modules/rules_kotlin/2.2.0-codeql.1/MODULE.bazel |
New MODULE.bazel with updated dependencies (rules_android 0.6.4, protobuf 29.0, etc.) |
misc/bazel/registry/modules/rules_kotlin/2.1.3-codeql.1/source.json |
Removed old version configuration |
java/kotlin-extractor/versions.bzl |
Removed 1.6.x and 1.7.x versions, added 2.3.0-Beta2 |
java/kotlin-extractor/src/main/kotlin/utils/versions/v_2_3_0-Beta2/*.kt |
Added version-specific compatibility files for Kotlin 2.3.0-Beta2 |
java/kotlin-extractor/src/main/kotlin/utils/TypeSubstitution.kt |
Added @Suppress("REDUNDANT_ELSE_IN_WHEN") annotations for Kotlin 2.3 compatibility |
java/kotlin-extractor/src/main/kotlin/KotlinUsesExtractor.kt |
Removed TODO comment and added suppression annotation |
java/kotlin-extractor/src/main/kotlin/KotlinFileExtractor.kt |
Added suppression annotations, explicit return type, and workaround for nullable delegate property |
java/kotlin-extractor/deps/*.jar |
Removed 1.6.x and 1.7.x dependency JARs, added 2.3.0-Beta2 JARs |
java/kotlin-extractor/BUILD.bazel |
Removed resource_strip_prefix configuration |
MODULE.bazel |
Updated rules_kotlin dependency and repository references |
Comments suppressed due to low confidence (1)
misc/bazel/registry/modules/rules_kotlin/2.2.0-codeql.1/MODULE.bazel:10
- There is a commented-out duplicate declaration of
bazel_dep(name = "rules_java", version = "7.2.0")on line 10, with an active identical declaration on line 11. This commented line should be removed unless there's a specific reason to keep it.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
If we're removing support for 1.6 and 1.7, then we should move (into |
...or/src/main/kotlin/utils/versions/v_2_3_0-Beta2/getJvmModuleNameForDeserializedDescriptor.kt
Outdated
Show resolved
Hide resolved
java/kotlin-extractor/src/main/kotlin/utils/versions/v_2_3_0-Beta2/Kotlin2ComponentRegistrar.kt
Outdated
Show resolved
Hide resolved
| // For Kotlin < 2.3, s.delegate is not-nullable. Cast to a be nullable, | ||
| // as a workaround to silence warnings for kotlin < 2.3 about the elvis | ||
| // operator being redundant. | ||
| // For Kotlin >= 2.3, the cast is redundant, so we need to silence that warning |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you know why it is now nullable? Is that actually an error?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tbh, I have not dug into why it can be nullable. Let me take a look at the API to understand if its an error if its null.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot was very convincing in explaining that the delegate may become null after lowering. But as the extractor hooks into the pipeline before lowering, the delegate should always be set.
|
BTW Kotlin 2.3.0 is released, so you can update 2.3.0-Beta2 to 2.3.0 |
|
@andersfugmann we are blocked on this, can i help in any way? i tried to complete it on our own fork but couldn't, for version |
|
Thanks for the offer to help @sgammon. We have been making progress and expect to have a working solution this week. Support for Kotlin 2.3 is expected to be available in CodeQL 2.24.1 (Early February) |
2e8db58 to
be03a72
Compare
|
@igfoo, I'm marking as ready for review.
|
|
I think it would make sense to rebase away the 2.3.0-Beta2 jars. |
0063f48 to
aa9dd59
Compare
java/kotlin-extractor/src/main/kotlin/utils/versions/v_2_3_0-Beta2/IrSimpleTypeImplCompat.kt
Outdated
Show resolved
Hide resolved
java/kotlin-extractor/src/main/kotlin/utils/versions/v_2_3_0-Beta2/IrSimpleTypeImplCompat.kt
Outdated
Show resolved
Hide resolved
java/kotlin-extractor/src/main/kotlin/utils/versions/unused/allOverriddenIncludingSelf.kt
Outdated
Show resolved
Hide resolved
| @@ -1,2 +0,0 @@ | |||
| | CodeQL Kotlin extractor | 5 | | Unbound symbol found, skipping extraction of expression | app/src/main/kotlin/testProject/App.kt:7:1:8:55 | app/src/main/kotlin/testProject/App.kt:7:1:8:55 | | |||
| | CodeQL Kotlin extractor | 5 | | Unbound symbol found, skipping extraction of expression | app/src/main/kotlin/testProject/App.kt:14:1:17:1 | app/src/main/kotlin/testProject/App.kt:14:1:17:1 | | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What happened here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is part of changes made to when deprecating support for Kotlin 1.6.0. See comment below
| @@ -21,35 +21,6 @@ app/src/main/kotlin/testProject/App.kt: | |||
| # 0| 0: [TypeAccess] int | |||
| # 0| 3: [Parameter] serializationConstructorMarker | |||
| # 0| 0: [TypeAccess] SerializationConstructorMarker | |||
| # 7| 5: [BlockStmt] { ... } | |||
| # 7| 0: [ExprStmt] <Expr>; | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like we lost lots of stuff; is that expected?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test was initially run using Kotlin 1.6 for which support has been removed so had to be updated to Kotlin 1.8.
In Kotlin 1.6, the serialization plugin generated method bodies and made them visible to the IR, but in Kotlin 1.8, The plugin now uses a two-pass system (IrPreGenerator + SerializerIrGenerator), so the bodies are generated in a separate IR generation pass which is why we do not extract the bodies anymore.
I've tested the test changes on main, and observe the same, so this is not caused by changes needed to support Kotlin 2.3.
If we want to capture the function bodies, we can force the Kotlin extractor to be loaded last to be able to observe the function bodies.
diff --git a/java/kotlin-extractor/src/main/kotlin/KotlinExtractorComponentRegistrar.kt b/java/kotlin-extractor/src/main/kotlin/KotlinExtractorComponentRegistrar.kt
index 7fd6416dcd6..81e3c2bba36 100644
--- a/java/kotlin-extractor/src/main/kotlin/KotlinExtractorComponentRegistrar.kt
+++ b/java/kotlin-extractor/src/main/kotlin/KotlinExtractorComponentRegistrar.kt
@@ -4,6 +4,7 @@
package com.github.codeql
import com.intellij.mock.MockProject
+import com.intellij.openapi.extensions.LoadingOrder
import org.jetbrains.kotlin.backend.common.extensions.IrGenerationExtension
import org.jetbrains.kotlin.config.CompilerConfiguration
@@ -16,14 +17,18 @@ class KotlinExtractorComponentRegistrar : Kotlin2ComponentRegistrar() {
if (invocationTrapFile == null) {
throw Exception("Required argument for TRAP invocation file not given")
}
- IrGenerationExtension.registerExtension(
- project,
+ // Register with LoadingOrder.LAST to ensure the extractor runs after other
+ // IR generation plugins (like kotlinx.serialization) have generated their code.
+ val extensionPoint = project.extensionArea.getExtensionPoint(IrGenerationExtension.extensionPointName)
+ extensionPoint.registerExtension(
KotlinExtractorExtension(
invocationTrapFile,
configuration[KEY_CHECK_TRAP_IDENTICAL] ?: false,
configuration[KEY_COMPILATION_STARTTIME],
configuration[KEY_EXIT_AFTER_EXTRACTION] ?: false
- )
+ ),
+ LoadingOrder.LAST,
+ project
)
}
}
I can create a follow-up PR for that if we want this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good; we do expect to be running last, so we're analysing the code that's actually executed.
redsun82
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bazel changes and testing some builds locally LGTM. Everything seems in order to me 👍
| ("%s/" % package_name() if package_name() else "") + | ||
| v | ||
| ), | ||
| resource_strip_prefix = v, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm guessing the situation has improved since I did this, and the simple v is enough. making it work from @codeql is a nice to have, but it has to work from @codeql_kotlin_embeddable
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not familiar with the @codeql_kotlin_embeddable. Where are those needed, and, more importantly, how do I test them?
…hanges introduced in Kotlin 2.3
This change rolls up all files from v1_6_0, v1_6_20, v1_7_0 and v_1_7_20.
In addition, versioned files that are not overridden by any later Kotlin versions (i.e. files that only have one copy under utils/versions) are inlined and removed to simplify list of changes.
List of removed/inlined files:
allOverriddenIncludingSelf.kt
copyTo.kt
ExperimentalCompilerApi.kt
getFileClassFqName.kt
IsUnderscoreParameter.kt
ReferenceEntity.kt
SyntheticBodyKind.kt
Types.kt
withHasQuestionMark.kt
… for Kotlin 1.6 and 1.7
…mpiler 1.8.10 and accept test changes
…o exist in the stdlib
Accept test changes from Kotlin 2.3.0 update Updates expected test outputs for kotlin2 library tests to match actual compiler output. Changes include: - Location adjustments for properties/methods (now point to identifiers) - CastExpr -> ImplicitCastExpr for implicit type casts - Removed duplicate BlockStmt entries in loop ASTs - Super constructor call location changes Note that in Kotlin 2.3.0 super constructor calls now have locations spanning entire class declarations instead of the actual super call site.
1e9864d to
f296f8d
Compare
This PR add support for Kotlin 2.3.0-Beta2.
To support Kotlin 2.3.0-Beta2, and update of the
rules_kotlinbazel package needs to be updated to 2.2.0. Unfortunatly that drops support for Kotlin 1.6.x and 1.7.x.