feat(gazelle): For package mode, resolve dependencies when imports are relative to the package path (#2865)

When `# gazelle:python_generation_mode package` is enabled, relative
imports are currently not being added to the `deps` field of the
generated target.

For example, given the following Python code:
```
from .library import add as _add
from .library import divide as _divide
from .library import multiply as _multiply
from .library import subtract as _subtract
```

The expected py_library rule should include a dependency on the local
library package:
```
py_library(
    name = "py_default_library",
    srcs = ["__init__.py"],
    visibility = ["//visibility:public"],
    deps = [
        "//example/library:py_default_library",
    ],
)
```
However, the actual generated rule is missing the deps entry:

```
py_library(
    name = "py_default_library",
    srcs = ["__init__.py"],
    visibility = ["//visibility:public"],
)
```
This change updates file_parser.go to ensure that relative imports
(those starting with a .) are parsed and preserved. In `Resolve()`,
logic is added to correctly interpret relative paths:

A single dot (.) refers to the current package.

Multiple dots (.., ..., etc.) traverse up parent directories.

The relative import is resolved against the current label.Pkg path that
imports the module and converted into an path relative to the root
before dependency resolution.

As a result, dependencies for relative imports are now correctly added
to the deps field in package generation mode.

Added a directive `# gazelle:experimental_allow_relative_imports true`
to allow this feature to be opt in.

---------

Co-authored-by: yushan <[email protected]>
Co-authored-by: Ignas Anikevicius <[email protected]>
Co-authored-by: Douglas Thor <[email protected]>

Author: 4 people

CHANGELOG.md

@@ -54,6 +54,9 @@ END_UNRELEASED_TEMPLATE
 
 {#v0-0-0-changed}
 ### Changed
+* (gazelle) For package mode, resolve dependencies when imports are relative
+  to the package path. This is enabled via the
+  `# gazelle:experimental_allow_relative_imports` true directive ({gh-issue}`2203`).
 * (gazelle) Types for exposed members of `python.ParserOutput` are now all public.
 
 {#v0-0-0-fixed}

gazelle/README.md

@@ -121,12 +121,12 @@ gazelle_python_manifest(
     requirements = "//:requirements_lock.txt",
     # include_stub_packages: bool (default: False)
     # If set to True, this flag automatically includes any corresponding type stub packages
-    # for the third-party libraries that are present and used. For example, if you have 
+    # for the third-party libraries that are present and used. For example, if you have
     # `boto3` as a dependency, and this flag is enabled, the corresponding `boto3-stubs`
     # package will be automatically included in the BUILD file.
     #
-    # Enabling this feature helps ensure that type hints and stubs are readily available 
-    # for tools like type checkers and IDEs, improving the development experience and 
+    # Enabling this feature helps ensure that type hints and stubs are readily available
+    # for tools like type checkers and IDEs, improving the development experience and
     # reducing manual overhead in managing separate stub packages.
     include_stub_packages = True
 )
@@ -220,6 +220,8 @@ Python-specific directives are as follows:
 | Defines the format of the distribution name in labels to third-party deps. Useful for using Gazelle plugin with other rules with different repository conventions (e.g. `rules_pycross`). Full label is always prepended with (pip) repository name, e.g. `@pip//numpy`.                        |
 | `# gazelle:python_label_normalization`                                                                                                                                                                                                                                                          | `snake_case` |
 | Controls how distribution names in labels to third-party deps are normalized. Useful for using Gazelle plugin with other rules with different label conventions (e.g. `rules_pycross` uses PEP-503). Can be "snake_case", "none", or "pep503".                                                  |
+| `# gazelle:experimental_allow_relative_imports`          | `false` |
+| Controls whether Gazelle resolves dependencies for import statements that use paths relative to the current package. Can be "true" or "false".|
 
 #### Directive: `python_root`:
 
@@ -468,7 +470,7 @@ def py_test(name, main=None, **kwargs):
             name = "__test__",
             deps = ["@pip_pytest//:pkg"],  # change this to the pytest target in your repo.
         )
-    
+
         deps.append(":__test__")
         main = ":__test__.py"
 
@@ -581,6 +583,44 @@ deps = [
 ]
 ```
 
+#### Directive: `experimental_allow_relative_imports`
+Enables experimental support for resolving relative imports in
+`python_generation_mode package`.
+
+By default, when `# gazelle:python_generation_mode package` is enabled,
+relative imports (e.g., from .library import foo) are not added to the
+deps field of the generated target. This results in incomplete py_library
+rules that lack required dependencies on sibling packages.
+
+Example:
+Given this Python file import:
+```python
+from .library import add as _add
+from .library import subtract as _subtract
+```
+
+Expected BUILD file output:
+```starlark
+py_library(
+    name = "py_default_library",
+    srcs = ["__init__.py"],
+    deps = [
+        "//example/library:py_default_library",
+    ],
+    visibility = ["//visibility:public"],
+)
+```
+
+Actual output without this annotation:
+```starlark
+py_library(
+    name = "py_default_library",
+    srcs = ["__init__.py"],
+    visibility = ["//visibility:public"],
+)
+```
+If the directive is set to `true`, gazelle will resolve imports
+that are relative to the current package.
 
 ### Libraries
 

gazelle/python/configure.go

@@ -68,6 +68,7 @@ func (py *Configurer) KnownDirectives() []string {
 		pythonconfig.TestFilePattern,
 		pythonconfig.LabelConvention,
 		pythonconfig.LabelNormalization,
+		pythonconfig.ExperimentalAllowRelativeImports,
 	}
 }
 
@@ -222,6 +223,13 @@ func (py *Configurer) Configure(c *config.Config, rel string, f *rule.File) {
 			default:
 				config.SetLabelNormalization(pythonconfig.DefaultLabelNormalizationType)
 			}
+		case pythonconfig.ExperimentalAllowRelativeImports:
+			v, err := strconv.ParseBool(strings.TrimSpace(d.Value))
+			if err != nil {
+				log.Printf("invalid value for gazelle:%s in %q: %q",
+					pythonconfig.ExperimentalAllowRelativeImports, rel, d.Value)
+			}
+			config.SetExperimentalAllowRelativeImports(v)
 		}
 	}
 

gazelle/python/file_parser.go

@@ -165,7 +165,9 @@ func (p *FileParser) parseImportStatements(node *sitter.Node) bool {
 		}
 	} else if node.Type() == sitterNodeTypeImportFromStatement {
 		from := node.Child(1).Content(p.code)
-		if strings.HasPrefix(from, ".") {
+		// If the import is from the current package, we don't need to add it to the modules i.e. from . import Class1.
+		// If the import is from a different relative package i.e. from .package1 import foo, we need to add it to the modules.
+		if from == "." {
 			return true
 		}
 		for j := 3; j < int(node.ChildCount()); j++ {

gazelle/python/resolve.go

@@ -148,12 +148,61 @@ func (py *Resolver) Resolve(
 		modules := modulesRaw.(*treeset.Set)
 		it := modules.Iterator()
 		explainDependency := os.Getenv("EXPLAIN_DEPENDENCY")
+		// Resolve relative paths for package generation
+		isPackageGeneration := !cfg.PerFileGeneration() && !cfg.CoarseGrainedGeneration()
 		hasFatalError := false
 	MODULES_LOOP:
 		for it.Next() {
 			mod := it.Value().(Module)
-			moduleParts := strings.Split(mod.Name, ".")
-			possibleModules := []string{mod.Name}
+			moduleName := mod.Name
+			// Transform relative imports `.` or `..foo.bar` into the package path from root.
+			if strings.HasPrefix(mod.From, ".") {
+				if !cfg.ExperimentalAllowRelativeImports() || !isPackageGeneration {
+					continue MODULES_LOOP
+				}
+
+				// Count number of leading dots in mod.From (e.g., ".." = 2, "...foo.bar" = 3)
+				relativeDepth := strings.IndexFunc(mod.From, func(r rune) bool { return r != '.' })
+				if relativeDepth == -1 {
+					relativeDepth = len(mod.From)
+				}
+
+				// Extract final symbol (e.g., "some_function") from mod.Name
+				imported := mod.Name
+				if idx := strings.LastIndex(mod.Name, "."); idx >= 0 {
+					imported = mod.Name[idx+1:]
+				}
+
+				// Optional subpath in 'from' clause, e.g. "from ...my_library.foo import x"
+				fromPath := strings.TrimLeft(mod.From, ".")
+				var fromParts []string
+				if fromPath != "" {
+					fromParts = strings.Split(fromPath, ".")
+				}
+
+				// Current Bazel package as path segments
+				pkgParts := strings.Split(from.Pkg, "/")
+
+				if relativeDepth-1 > len(pkgParts) {
+					log.Printf("ERROR: Invalid relative import %q in %q: exceeds package root.", mod.Name, mod.Filepath)
+					continue MODULES_LOOP
+				}
+
+				// Go up relativeDepth - 1 levels
+				baseParts := pkgParts
+				if relativeDepth > 1 {
+					baseParts = pkgParts[:len(pkgParts)-(relativeDepth-1)]
+				}
+				// Build absolute module path
+				absParts := append([]string{}, baseParts...)       // base path
+				absParts = append(absParts, fromParts...)          // subpath from 'from'
+				absParts = append(absParts, imported)              // actual imported symbol
+
+				moduleName = strings.Join(absParts, ".")
+			}
+
+			moduleParts := strings.Split(moduleName, ".")
+			possibleModules := []string{moduleName}
 			for len(moduleParts) > 1 {
 				// Iterate back through the possible imports until
 				// a match is found.

gazelle/python/testdata/relative_imports/README.md

@@ -0,0 +1,2 @@
+# gazelle:python_generation_mode package
+# gazelle:experimental_allow_relative_imports true

gazelle/python/testdata/relative_imports_package_mode/BUILD.in

@@ -0,0 +1,15 @@
+load("@rules_python//python:defs.bzl", "py_binary")
+
+# gazelle:python_generation_mode package
+# gazelle:experimental_allow_relative_imports true
+
+py_binary(
+    name = "relative_imports_package_mode_bin",
+    srcs = ["__main__.py"],
+    main = "__main__.py",
+    visibility = ["//:__subpackages__"],
+    deps = [
+        "//package1",
+        "//package2",
+    ],
+)

gazelle/python/testdata/relative_imports_package_mode/BUILD.out

@@ -0,0 +1,6 @@
+# Resolve deps for relative imports
+
+This test case verifies that the generated targets correctly handle relative imports in
+Python. Specifically, when the Python generation mode is set to "package," it ensures
+that relative import statements such as from .foo import X are properly resolved to
+their corresponding modules.

gazelle/python/testdata/relative_imports_package_mode/README.md

@@ -0,0 +1,5 @@
+from package1.module1 import function1
+from package2.module3 import function3
+
+print(function1())
+print(function3())

gazelle/python/testdata/relative_imports/WORKSPACE renamed to gazelle/python/testdata/relative_imports_package_mode/WORKSPACE

@@ -0,0 +1,11 @@
+load("@rules_python//python:defs.bzl", "py_library")
+
+py_library(
+    name = "package1",
+    srcs = [
+        "__init__.py",
+        "module1.py",
+        "module2.py",
+    ],
+    visibility = ["//:__subpackages__"],
+)

gazelle/python/testdata/relative_imports_package_mode/__main__.py

@@ -0,0 +1,2 @@
+def some_function():
+    pass

gazelle/python/testdata/relative_imports/package2/BUILD.in renamed to gazelle/python/testdata/relative_imports_package_mode/package1/BUILD.in

@@ -0,0 +1,7 @@
+load("@rules_python//python:defs.bzl", "py_library")
+
+py_library(
+    name = "my_library",
+    srcs = ["__init__.py"],
+    visibility = ["//:__subpackages__"],
+)

gazelle/python/testdata/relative_imports_package_mode/package1/BUILD.out

@@ -0,0 +1,7 @@
+load("@rules_python//python:defs.bzl", "py_library")
+
+py_library(
+    name = "my_library",
+    srcs = ["__init__.py"],
+    visibility = ["//:__subpackages__"],
+)

gazelle/python/testdata/relative_imports_package_mode/package1/__init__.py

@@ -0,0 +1,2 @@
+def some_function():
+    return "some_function"

gazelle/python/testdata/relative_imports/package1/module1.py renamed to gazelle/python/testdata/relative_imports_package_mode/package1/module1.py

@@ -0,0 +1,7 @@
+load("@rules_python//python:defs.bzl", "py_library")
+
+py_library(
+    name = "foo",
+    srcs = ["__init__.py"],
+    visibility = ["//:__subpackages__"],
+)

gazelle/python/testdata/relative_imports/package1/module2.py renamed to gazelle/python/testdata/relative_imports_package_mode/package1/module2.py

@@ -0,0 +1,2 @@
+def some_function():
+    return "some_function"

gazelle/python/testdata/relative_imports_package_mode/package1/my_library/BUILD.in

@@ -0,0 +1,10 @@
+load("@rules_python//python:defs.bzl", "py_library")
+
+py_library(
+    name = "subpackage1",
+    srcs = [
+        "__init__.py",
+        "some_module.py",
+    ],
+    visibility = ["//:__subpackages__"],
+)

gazelle/python/testdata/relative_imports_package_mode/package1/my_library/BUILD.out

@@ -0,0 +1,10 @@
+load("@rules_python//python:defs.bzl", "py_library")
+
+py_library(
+    name = "subpackage1",
+    srcs = [
+        "__init__.py",
+        "some_module.py",
+    ],
+    visibility = ["//:__subpackages__"],
+)

gazelle/python/testdata/relative_imports_package_mode/package1/my_library/__init__.py

@@ -0,0 +1,3 @@
+
+def some_init():
+    return "some_init"

gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/BUILD.in

@@ -0,0 +1,3 @@
+
+def some_function():
+    return "some_function"

gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/BUILD.out

@@ -0,0 +1,16 @@
+load("@rules_python//python:defs.bzl", "py_library")
+
+py_library(
+    name = "subpackage2",
+    srcs = [
+        "__init__.py",
+        "script.py",
+    ],
+    visibility = ["//:__subpackages__"],
+    deps = [
+        "//package1/my_library",
+        "//package1/my_library/foo",
+        "//package1/subpackage1",
+        "//package1/subpackage1/subpackage2/library",
+    ],
+)

gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/__init__.py

@@ -0,0 +1,7 @@
+load("@rules_python//python:defs.bzl", "py_library")
+
+py_library(
+    name = "library",
+    srcs = ["other_module.py"],
+    visibility = ["//:__subpackages__"],
+)

gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/BUILD.in

@@ -0,0 +1,11 @@
+from ...my_library import (
+    some_function,
+)  # Import path should be package1.my_library.some_function
+from ...my_library.foo import (
+    some_function,
+)  # Import path should be package1.my_library.foo.some_function
+from .library import (
+    other_module,
+)  # Import path should be package1.subpackage1.subpackage2.library.other_module
+from .. import some_module  # Import path should be package1.subpackage1.some_module
+from .. import some_function  # Import path should be package1.subpackage1.some_function

gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/BUILD.out

@@ -0,0 +1,12 @@
+load("@rules_python//python:defs.bzl", "py_library")
+
+py_library(
+    name = "package2",
+    srcs = [
+        "__init__.py",
+        "module3.py",
+        "module4.py",
+    ],
+    visibility = ["//:__subpackages__"],
+    deps = ["//package2/library"],
+)

gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/__init__.py

@@ -0,0 +1,20 @@
+from .library import add as _add
+from .library import divide as _divide
+from .library import multiply as _multiply
+from .library import subtract as _subtract
+
+
+def add(a, b):
+    return _add(a, b)
+
+
+def divide(a, b):
+    return _divide(a, b)
+
+
+def multiply(a, b):
+    return _multiply(a, b)
+
+
+def subtract(a, b):
+    return _subtract(a, b)

gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/some_module.py

@@ -0,0 +1,7 @@
+load("@rules_python//python:defs.bzl", "py_library")
+
+py_library(
+    name = "library",
+    srcs = ["__init__.py"],
+    visibility = ["//:__subpackages__"],
+)

gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/BUILD.in

@@ -0,0 +1,14 @@
+def add(a, b):
+    return a + b
+
+
+def divide(a, b):
+    return a / b
+
+
+def multiply(a, b):
+    return a * b
+
+
+def subtract(a, b):
+    return a - b

gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/BUILD.out

@@ -0,0 +1,5 @@
+from .library import function5
+
+
+def function3():
+    return "function3 " + function5()