Add package 'name' to lock file #11488

jtcohen6 · 2025-04-11T00:06:19Z

Resolves #11487

Problem

dbt gets the true project name for each configured package (and their dependencies), but it doesn't store that information anywhere, requiring multiple trips to git providers / hub packages.

Solution

Write each package's true project name in package-lock.yml
Without changing the sha1_hash (if possible)

Checklist

I have read the contributing guide and understand what's expected of me.
I have run this code in development, and it appears to resolve the stated issue.
This PR includes tests, or tests are not required or relevant for this PR.
This PR has no interface changes (e.g., macros, CLI, logs, JSON artifacts, config files, adapter interface, etc.) or this PR has already received feedback and approval from Product or DX.
This PR includes type annotations for new and modified functions.

github-actions · 2025-04-11T00:06:33Z

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

zqureshi · 2025-04-11T00:07:53Z

👀

codecov · 2025-04-11T00:09:16Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.88%. Comparing base (7cca847) to head (7da1e0a).
Report is 2 commits behind head on main.

❌ Your patch status has failed because the patch coverage (53.84%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #11488   +/-   ##
=======================================
  Coverage   88.88%   88.88%           
=======================================
  Files         194      194           
  Lines       24553    24564   +11     
=======================================
+ Hits        21824    21834   +10     
- Misses       2729     2730    +1

Flag	Coverage Δ
integration	`85.88% <100.00%> (+<0.01%)`	⬆️
unit	`62.77% <53.84%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
Unit Tests	`62.77% <53.84%> (-0.01%)`	⬇️
Integration Tests	`85.88% <100.00%> (+<0.01%)`	⬆️

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

jtcohen6 · 2025-04-11T00:08:20Z

core/dbt/contracts/project.py

@@ -42,13 +42,19 @@ class Quoting(dbtClassMixin, Mergeable):

 @dataclass
 class Package(dbtClassMixin):
-    pass
+
+    def __post_serialize__(self, data, context: Optional[Dict]):


The intent is to avoid changing the sha1_hash in package-lock.yml just because there's now an optional name attribute in the contract:

{'git': 'https://github.com/dbt-labs/dbt-utils', 'revision': None, 'warn-unpinned': None, 'subdirectory': None, 'unrendered': {'git': 'https://github.com/dbt-labs/dbt-utils'}, 'name': None}

But name actually is a required field for TarballPackage

This would be great to leave as a comment for the method.

I can definitely add a comment. I'm also open to moving this logic elsewhere, so that the intent is clearer and the impact is narrower - perhaps task.deps._create_sha1_hash, where package.to_dict is actually being called?

jtcohen6 · 2025-04-11T00:11:55Z

core/dbt/contracts/project.py



 @dataclass
 class LocalPackage(Package):
    local: str
    unrendered: Dict[str, Any] = field(default_factory=dict)
+    name: Optional[str] = None


In the deps task here, dbt reads package-lock.yml and validates its contents using these contracts. So if name is in there, it needs to be an allowed (optional) field in all Package contracts.

The alternative would be to strip out name when dbt reads that file — but leave it in for TarballPackage only? — which feels trickier.

zqureshi

LGTM but I would have @peterallenwebb or @emmyoop rubber stamp since they are doing infra things.

zqureshi · 2025-04-11T00:11:28Z

core/dbt/task/deps.py

@@ -204,7 +207,7 @@ def run(self) -> None:
        if self.args.add_package:
            self.add()

-        # Check lock file exist and generated by the same pacakges.yml
+        # Check lock file exist and generated by the same packages.yml


I was wondering why this is a diff... until I saw it.

zqureshi · 2025-04-11T00:12:27Z

core/dbt/task/deps.py

-            self.project.packages.packages
-        )
+            package_dict = package.to_dict()
+            package_dict["name"] = package.get_project_name(self.project, renderer)


zqureshi · 2025-04-11T00:14:29Z

tests/functional/dependencies/test_dependency_options.py

@@ -34,9 +34,11 @@ def test_deps_lock(self, clean_start):
        with open("package-lock.yml") as fp:
            contents = fp.read()

-        fivetran_package = "- package: fivetran/fivetran_utils\n    version: 0.4.7"
+        fivetran_package = (


nit: I wonder if this would be clearer to read with a heredoc.

qq chose comma ça ?

fivetran_package = """ - name: fivetran_utils package: fivetran/fivetran_utils version: 0.4.7 """

If you also feel that it increases comprehension, I'm 50/50.

I do think it's easier to read!

zqureshi · 2025-04-15T05:54:23Z

🚀🚀🚀

* Add package 'name' to lock file * PR feedback + changie * Fix test + add comment

Add package 'name' to lock file

1b47605

jtcohen6 requested a review from a team as a code owner April 11, 2025 00:06

cla-bot bot added the cla:yes label Apr 11, 2025

jtcohen6 commented Apr 11, 2025

View reviewed changes

zqureshi approved these changes Apr 11, 2025

View reviewed changes

jtcohen6 added 2 commits April 11, 2025 02:24

PR feedback + changie

d898086

Fix test + add comment

7da1e0a

jtcohen6 requested review from peterallenwebb and emmyoop April 11, 2025 16:35

gshank approved these changes Apr 11, 2025

View reviewed changes

jtcohen6 merged commit 9c91ab2 into main Apr 15, 2025
60 of 61 checks passed

jtcohen6 deleted the jerco/impl-11487 branch April 15, 2025 05:19

QMalcolm pushed a commit that referenced this pull request Apr 17, 2025

Add package 'name' to lock file (#11488)

04f579a

* Add package 'name' to lock file * PR feedback + changie * Fix test + add comment

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add package 'name' to lock file #11488

Add package 'name' to lock file #11488

jtcohen6 commented Apr 11, 2025

github-actions bot commented Apr 11, 2025

zqureshi commented Apr 11, 2025

codecov bot commented Apr 11, 2025 •

edited

Loading

jtcohen6 Apr 11, 2025

zqureshi Apr 11, 2025

jtcohen6 Apr 11, 2025 •

edited

Loading

jtcohen6 Apr 11, 2025

zqureshi left a comment

zqureshi Apr 11, 2025

zqureshi Apr 11, 2025

zqureshi Apr 11, 2025

jtcohen6 Apr 11, 2025

zqureshi Apr 11, 2025

jtcohen6 Apr 11, 2025

zqureshi commented Apr 15, 2025

Add package 'name' to lock file #11488

Add package 'name' to lock file #11488

Conversation

jtcohen6 commented Apr 11, 2025

Problem

Solution

Checklist

github-actions bot commented Apr 11, 2025

zqureshi commented Apr 11, 2025

codecov bot commented Apr 11, 2025 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jtcohen6 Apr 11, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zqureshi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zqureshi commented Apr 15, 2025

codecov bot commented Apr 11, 2025 •

edited

Loading

jtcohen6 Apr 11, 2025 •

edited

Loading