Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-51537][CONNECT][CORE][FOLLOW-UP][TEST] Add test to ensure Spark plugins are not reloaded #50526

Closed
wants to merge 1 commit into from

Conversation

wbo4958
Copy link
Contributor

@wbo4958 wbo4958 commented Apr 7, 2025

What changes were proposed in this pull request?

This PR adds a unit test to verify that Spark plugin JARs specified via --jars are not reloaded.

Why are the changes needed?

This PR is a followup of #50334 (comment)

Does this PR introduce any user-facing change?

No

How was this patch tested?

The test added can pass

Was this patch authored or co-authored using generative AI tooling?

No

@github-actions github-actions bot added the CORE label Apr 7, 2025
@wbo4958
Copy link
Contributor Author

wbo4958 commented Apr 7, 2025

Hi @vicennial @HyukjinKwon, Could you help review this PR, thx very much.

Copy link
Contributor

@vicennial vicennial left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Smart way to test this scenario, nice one!
Additionally verified that this test would indeed fail without the final changes in the parent PR as well as verified that it fails with the changes in the original approach

Thanks for the test!

@wbo4958
Copy link
Contributor Author

wbo4958 commented Apr 9, 2025

Hi @HyukjinKwon, Could you please also take a look at it. Thx very much.

@wbo4958
Copy link
Contributor Author

wbo4958 commented Apr 10, 2025

Hi @HyukjinKwon, it would be great it you can help on this PR. Thx very much.

HyukjinKwon pushed a commit that referenced this pull request Apr 10, 2025
…k plugins are not reloaded

### What changes were proposed in this pull request?

This PR adds a unit test to verify that Spark plugin JARs specified via `--jars` are not reloaded.

### Why are the changes needed?

This PR is a followup of #50334 (comment)

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?

The test added can pass

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #50526 from wbo4958/SPARK-51537-followup.

Authored-by: Bobby Wang <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit 622fa35)
Signed-off-by: Hyukjin Kwon <[email protected]>
HyukjinKwon pushed a commit that referenced this pull request Apr 10, 2025
…k plugins are not reloaded

### What changes were proposed in this pull request?

This PR adds a unit test to verify that Spark plugin JARs specified via `--jars` are not reloaded.

### Why are the changes needed?

This PR is a followup of #50334 (comment)

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?

The test added can pass

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #50526 from wbo4958/SPARK-51537-followup.

Authored-by: Bobby Wang <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit 622fa35)
Signed-off-by: Hyukjin Kwon <[email protected]>
@HyukjinKwon
Copy link
Member

Merged to master, branch-4.0 and branch-3.5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants