Skip to content

Merge main #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 140 commits into
base: dap-main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
140 commits
Select commit Hold shift + click to select a range
7bef2d1
Add force on cluster config option
benjamin-awd Jul 28, 2024
a99ef52
Fix quotation in split_part macro
3fonov Aug 14, 2024
1ec00fe
Remove stack trace from errors
3fonov Aug 15, 2024
5302194
Revert "Fix quotation in split_part macro"
3fonov Aug 15, 2024
9281cf3
Fix letters case
3fonov Aug 15, 2024
159a74d
Merge branch 'main' into error-logging
3fonov Aug 15, 2024
678c8b6
Change to splitByString
3fonov Aug 15, 2024
fdcee21
Merge branch 'main' into error-logging
3fonov Aug 15, 2024
5d119a4
Fix formatting
3fonov Aug 21, 2024
24f08db
Merge branch 'main' into error-logging
3fonov Aug 21, 2024
efec997
Fix quotation in split_part test
3fonov Aug 23, 2024
2934c2b
Merge branch 'main' into error-logging
3fonov Aug 26, 2024
6e155e8
feat: support range_hashed and complex_key_range_hashed dictionary ma…
rjoelnorgren Sep 24, 2024
7a84e51
Apply query settings for empty model
alexsubota Sep 25, 2024
d992b9f
Create test_query_settings.py
alexsubota Sep 26, 2024
52cbf4f
fix black test_query_settings.py
alexsubota Sep 27, 2024
58f7fa9
don't drop local tables when there is no distributed table
canbekley Oct 1, 2024
01c2e9f
fix: Filter settings based on Engine
shachibista Oct 11, 2024
8f49ae9
Make linter happy
shachibista Oct 11, 2024
6e92d04
Merge pull request #338 from 3fonov/main
BentsiLeviav Oct 30, 2024
f5c00bb
Merge pull request #361 from rjoelnorgren/feat-add-range-layout-dicti…
BentsiLeviav Oct 30, 2024
0198fc8
Merge pull request #362 from alexsubota/apply-query-settings-for-empt…
BentsiLeviav Oct 30, 2024
01dd31a
update CHANGELOG
rjoelnorgren Oct 31, 2024
86f8843
add test for range dictionary layout
rjoelnorgren Oct 31, 2024
ae8a7b6
feat: add list support to `primary_key`
lucasfcnunes Nov 3, 2024
90ba920
add pk to tests
lucasfcnunes Nov 3, 2024
065f3a7
Merge pull request #376 from rjoelnorgren/range-dictionary-tests-chan…
BentsiLeviav Nov 3, 2024
d446760
Merge remote-tracking branch 'origin/main' into fix/drop-local-table-…
canbekley Nov 4, 2024
d40abf4
add changelog
canbekley Nov 4, 2024
83564fb
Update CHANGELOG.md
lucasfcnunes Nov 5, 2024
f2488b0
Merge branch 'main' into main
lucasfcnunes Nov 5, 2024
2e87f58
Add in other places
3fonov Nov 11, 2024
9f4bd53
Merge branch 'ClickHouse:main' into fix_cast_bool
3fonov Nov 11, 2024
3f00634
Changelog update
3fonov Nov 11, 2024
e26bacc
Update httpclient
3fonov Nov 11, 2024
4a64149
Created ENV VAR to control behavior
3fonov Nov 11, 2024
7a1906c
Updated CHANGELOG
3fonov Nov 11, 2024
ba346f3
Fix linting
3fonov Nov 13, 2024
798aca9
Allow multiple materialized views to write to same target (#280)
the4thamigo-uk Oct 8, 2024
5d2933c
Update CHANGELOG.md (#280)
the4thamigo-uk Nov 13, 2024
064ed53
Add note to README.md (#364)
the4thamigo-uk Nov 13, 2024
bd45c3c
Merge pull request #382 from 3fonov/main
BentsiLeviav Nov 13, 2024
210d532
fix: create materialized view on cluster
flytrap Nov 14, 2024
f99d0d0
warn about potential hanging mvs that need to be dropped manually
BentsiLeviav Nov 19, 2024
15125f0
update warning message
BentsiLeviav Nov 19, 2024
adbcb4e
update readme file regarding a potential hanging mv
BentsiLeviav Nov 19, 2024
f7e6db1
add tests to cover update-schema situation and full refresh
BentsiLeviav Nov 19, 2024
0b54910
fix lint
BentsiLeviav Nov 19, 2024
d3271c8
Merge pull request #364 from the4thamigo-uk/multiple-materialized-vie…
BentsiLeviav Nov 19, 2024
8eabe42
update CHANGELOG.md for version 1.8.4
BentsiLeviav Nov 19, 2024
9be339a
Merge pull request #386 from ClickHouse/release-1.8.4-housekeeping
BentsiLeviav Nov 19, 2024
9ca5578
update CHANGELOG.md for version 1.8.5
BentsiLeviav Nov 19, 2024
edad093
Merge pull request #387 from ClickHouse/v1.8.5-housekeeping
BentsiLeviav Nov 19, 2024
26eb772
Prevent data loss when updating materialized view (#383)
the4thamigo-uk Nov 21, 2024
2a7a138
Use `CREATE OR REPLACE DICTIONARY` to avoid data loss (#392)
the4thamigo-uk Nov 26, 2024
94e9242
1) Support of insert_overwrite on cluster
pheepa Nov 28, 2024
89e865e
Protect against no columns returned in materialized-view (#395)
the4thamigo-uk Nov 29, 2024
c262aeb
CHANGELOG.md updated
pheepa Dec 2, 2024
29f2551
create indexes
stephennancekivell Dec 3, 2024
5848fcc
update changelog
stephennancekivell Dec 3, 2024
2575f77
make mv data catchup configurable
BentsiLeviav Dec 5, 2024
501e0a2
split mv test file + add catchup test
BentsiLeviav Dec 5, 2024
69693eb
fix lint
BentsiLeviav Dec 5, 2024
4cbf8cb
fix lint
BentsiLeviav Dec 5, 2024
d31591c
update readme file
BentsiLeviav Dec 5, 2024
e2ab918
version 1.8.6 housekeeping
BentsiLeviav Dec 5, 2024
73fe825
Merge pull request #399 from ClickHouse/make-mv-data-catchup-configur…
BentsiLeviav Dec 9, 2024
5a5f184
Merge branch 'main' into create-or-replace-dictionary
BentsiLeviav Dec 15, 2024
e541907
Merge pull request #393 from the4thamigo-uk/create-or-replace-dictionary
BentsiLeviav Dec 15, 2024
6f63e56
Merge pull request #377 from lucasfcnunes/main
BentsiLeviav Dec 15, 2024
a8c7986
Revert "Add list support to `primary_key`"
BentsiLeviav Dec 15, 2024
806fe34
Merge pull request #400 from ClickHouse/revert-377-main
BentsiLeviav Dec 15, 2024
173d9f2
Merge pull request #396 from the4thamigo-uk/protect-against-empty-col…
BentsiLeviav Dec 17, 2024
86ad380
support refreshable materialized views
BentsiLeviav Dec 26, 2024
c43e2a3
fix: remove python 3.8 from various places, #385
lkhagvadorj-amp Dec 27, 2024
24a9714
Add basic refreshable mv tests
BentsiLeviav Dec 29, 2024
125da93
Add docs in README.md + indentation
BentsiLeviav Dec 29, 2024
f8c43a4
update CHANGELOG.md
BentsiLeviav Dec 29, 2024
7ceed41
disable the dependencies validation by default
BentsiLeviav Dec 29, 2024
fa1d9ac
fix lint
BentsiLeviav Dec 29, 2024
bd2ae04
add limitation section to refreshable mv
BentsiLeviav Dec 29, 2024
e222d83
fix lint
BentsiLeviav Dec 29, 2024
63b8d79
use count due to differences between native and http when getting emp…
BentsiLeviav Dec 30, 2024
34a7280
use count due to differences between native and http when getting emp…
BentsiLeviav Dec 30, 2024
289eb89
add a new line between refresh and randomize clauses
BentsiLeviav Dec 30, 2024
c246354
Merge pull request #402 from lkhagvadorj-amp/fix/drop_python_3.8
BentsiLeviav Jan 1, 2025
0147955
Merge pull request #401 from ClickHouse/support-refreshable-mv
BentsiLeviav Jan 5, 2025
3ad5d82
organize the CHANGELOG.md release section of version 1.8.7
BentsiLeviav Jan 5, 2025
35b70af
increase version
BentsiLeviav Jan 5, 2025
d2a63b0
update CHANGELOG.md
BentsiLeviav Jan 5, 2025
58177fc
Merge branch 'main' into feature/create-indexes
BentsiLeviav Jan 5, 2025
81def6c
Merge branch 'main' into add-force-on-cluster
BentsiLeviav Jan 6, 2025
b9f9191
Merge pull request #328 from benjamin-awd/add-force-on-cluster
BentsiLeviav Jan 6, 2025
0a6cc4d
add indication for indexes in the README.md file
BentsiLeviav Jan 6, 2025
82a405e
Merge pull request #397 from stephen-up/feature/create-indexes
BentsiLeviav Jan 6, 2025
a430525
Change white-list to black-list
shachibista Jan 6, 2025
78b9222
Merge branch 'main' into filter-settings-by-engine
BentsiLeviav Jan 7, 2025
e299ecd
Fixes from review
shachibista Jan 8, 2025
67bb498
Merge pull request #367 from shachibista/filter-settings-by-engine
BentsiLeviav Jan 9, 2025
53b20a2
chore: allow optional whitespace in multi mv implementation
benjamin-awd Jan 17, 2025
47dc632
docs: readme
operagxoksana Jan 21, 2025
8f04ba3
Merge pull request #411 from operagxoksana/patch-1
BentsiLeviav Jan 21, 2025
12f26c4
Merge pull request #409 from benjamin-awd/support-multi-mv-whitespace
BentsiLeviav Jan 29, 2025
1f3da05
Merge branch 'main' into user-alter-table-modify-query
BentsiLeviav Jan 29, 2025
d5ba125
fix merge - refreshable mvs
BentsiLeviav Jan 30, 2025
486d5e9
Merge pull request #390 from the4thamigo-uk/user-alter-table-modify-q…
BentsiLeviav Jan 30, 2025
dc6948d
remove unreachable exchange code + support replace view
BentsiLeviav Feb 4, 2025
14fd131
add view tests for creat and update
BentsiLeviav Feb 4, 2025
8224eee
fix lint
BentsiLeviav Feb 4, 2025
ff5b6b4
fix lint
BentsiLeviav Feb 4, 2025
9a4b435
update CHANGELOG.md
BentsiLeviav Feb 4, 2025
41affa4
Fix docker user setup due to ClickHouse/ClickHouse/pull/75259
BentsiLeviav Feb 5, 2025
001047b
Fix docker user setup due to ClickHouse/ClickHouse/pull/75259
BentsiLeviav Feb 5, 2025
e4ae3ec
updates for release 1.8.8
BentsiLeviav Feb 5, 2025
969c6c3
Merge pull request #412 from ClickHouse/support-replace-view
BentsiLeviav Feb 5, 2025
7b959ae
revert readme changes
BentsiLeviav Feb 5, 2025
0c4ee6b
Merge pull request #414 from ClickHouse/housekeeping-v1.8.8
BentsiLeviav Feb 5, 2025
c28b9aa
Merge pull request #384 from flytrap/main
BentsiLeviav Feb 5, 2025
984e0a0
Revert "fix: create materialized view on cluster"
BentsiLeviav Feb 6, 2025
86724ce
Merge pull request #416 from ClickHouse/revert-384-main
BentsiLeviav Feb 6, 2025
bc33015
feat: allow using TLS client certificates
bartoszwjn Feb 4, 2025
15a4e8c
Update CHANGELOG.md
bartoszwjn Feb 4, 2025
57db2b2
Merge pull request #413 from qedsoftware/tls-client-cert-pr
BentsiLeviav Feb 6, 2025
c36c49f
Add workflow dispatch
BentsiLeviav Feb 12, 2025
3d67aa9
Merge pull request #363 from canbekley/fix/drop-local-table-issue
BentsiLeviav Feb 13, 2025
256f238
optimise adding projections and indexes in one statement, to avoid er…
stephennancekivell Feb 14, 2025
e572708
Merge pull request #394 from pheepa/feature/insert-replace-on-cluster
BentsiLeviav Feb 16, 2025
e8e65b4
Fix lint
BentsiLeviav Feb 16, 2025
dca14b3
Merge pull request #422 from ClickHouse/fix-lint
BentsiLeviav Feb 16, 2025
3079f7b
Merge pull request #421 from stephen-up/avoid-err-517
BentsiLeviav Feb 16, 2025
175500d
increase version
BentsiLeviav Feb 16, 2025
dff943d
Update CHANGELOG.md for version v1.8.9
BentsiLeviav Feb 16, 2025
8897ecb
Merge pull request #425 from ClickHouse/v1.8.9-housekeeping
BentsiLeviav Feb 16, 2025
a0491b1
Merge branch 'main' into merge_main
Magicbeanbuyer Feb 20, 2025
1a9aed5
chore: https://github.com/ClickHouse/dbt-clickhouse/commit/67cf9db528…
Magicbeanbuyer Feb 20, 2025
7f30e32
feat: change test branch
Magicbeanbuyer Feb 21, 2025
ff977b7
ci: reduce test senarios
Magicbeanbuyer Feb 21, 2025
5f974e7
fix: remove quotes around cluster
Magicbeanbuyer Feb 21, 2025
3e4cc12
fix: view
Magicbeanbuyer Feb 21, 2025
fb00685
fix: insert overwrite signature
Magicbeanbuyer Feb 21, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 3 additions & 9 deletions .github/workflows/test_matrix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: "test_matrix"

on: # yamllint disable-line rule:truthy
pull_request:
branches: main
branches: dap-main
push:
branches-ignore:
- '*_test'
Expand All @@ -12,6 +12,7 @@ on: # yamllint disable-line rule:truthy
paths-ignore:
- '**.md'
- 'LICENSE'
workflow_dispatch:

jobs:
tests:
Expand All @@ -23,16 +24,9 @@ jobs:
strategy:
matrix:
python-version:
- '3.9'
- '3.10'
- '3.11'
- '3.12'
clickhouse-version:
- '23.8'
- '24.1'
- '24.2'
- '24.3'
- latest
- '24.12'

steps:
- name: Checkout
Expand Down
77 changes: 76 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,78 @@
### Release [1.8.9], 2025-02-16

#### Improvements
* It is now possible to configure a TLS client certificate using `client_cert` and `client_cert_key` profile parameters. ([#413](https://github.com/ClickHouse/dbt-clickhouse/pull/413))
* Added Support of insert_overwrite in cluster setup with incremental and distributed_incremental materializations ([#394](https://github.com/ClickHouse/dbt-clickhouse/pull/394))
* Improve index and projections creation process ([#421](https://github.com/ClickHouse/dbt-clickhouse/pull/421))

#### Bugs
* Reverted breaking changes in MV materialization ([#416](https://github.com/ClickHouse/dbt-clickhouse/pull/416))
* A fix was introduced for distributed tables, where an incremental local table could have been dropped if the distributed table was missing. ([#363](https://github.com/ClickHouse/dbt-clickhouse/pull/363))


### Release [1.8.8], 2025-02-05
### Improvements
* Materialized view now attempts to use `ALTER TABLE...MODIFY QUERY` to update existing materialized views. This is an atomic operation so data is not lost. ([#390](https://github.com/ClickHouse/dbt-clickhouse/pull/390))
* Make view materialization updates atomic. ([#412](https://github.com/ClickHouse/dbt-clickhouse/pull/412))
* Create a black list settings to ignore based on the configured Engine. ([#367](https://github.com/ClickHouse/dbt-clickhouse/pull/367))

#### New Features
* [ClickHouse indexes](https://clickhouse.com/docs/en/optimize/sparse-primary-indexes) are now fully supported for `table` materialization.


#### New Features
* [ClickHouse indexes](https://clickhouse.com/docs/en/optimize/sparse-primary-indexes) are now fully supported for `table` materialization.
The index config should be added to the model config. for instance:
```python
{{ config(
materialized='%s',
indexes=[{
'name': 'your_index_name',
'definition': 'your_column TYPE minmax GRANULARITY 2'
}]
) }}
```

### Release [1.8.7], 2025-01-05

### New Features
* Added support for [refreshable materialized view](https://clickhouse.com/docs/en/materialized-view/refreshable-materialized-view) ([#401](https://github.com/ClickHouse/dbt-clickhouse/pull/401))

### Improvement
* Avoid potential data loss by using `CREATE OR REPLACE DICTIONARY` to atomically update a dictionary ([#393](https://github.com/ClickHouse/dbt-clickhouse/pull/393))
* Removed support in python 3.8 as it is no longer supported by dbt ([#402](https://github.com/ClickHouse/dbt-clickhouse/pull/402)

### Bug Fixes
* Fix a minor bug related to validating existence of an old hanging mv ([#396]())

### Release [1.8.6], 2024-12-05

### Improvement
* Today, on mv model creation, the target table is being populated with the historical data based on the query provided in the mv creation. This catchup mechanism is now behind a config flag and enabled by default (as is today). ([#399](https://github.com/ClickHouse/dbt-clickhouse/pull/399))


### Release [1.8.5], 2024-11-19

### New Features
* Added support for the creation of more than one materialized view inserting records into the same target table. ([#360](https://github.com/ClickHouse/dbt-clickhouse/pull/364))

### Improvement
* Added support for [range_hashed](https://clickhouse.com/docs/en/sql-reference/dictionaries#range_hashed) and [complex_key_range_hashed](https://clickhouse.com/docs/en/sql-reference/dictionaries#complex_key_range_hashed) layouts to the dictionary materialization. ([#361](https://github.com/ClickHouse/dbt-clickhouse/pull/361))
* Truncated stack trace for database errors for cleaner output when HIDE_STACK_TRACE variable is set to any value. ([#382](https://github.com/ClickHouse/dbt-clickhouse/pull/382))
* It is now possible to pass query settings not only on table creation but also on query. ([#362](https://github.com/ClickHouse/dbt-clickhouse/pull/362))


### Bug Fixes
* Before this version, `split_part` macro used to add an extra quotation. that was fixed in ([#338](https://github.com/ClickHouse/dbt-clickhouse/pull/338))

### Bug Fixes
* Existing local tables are no longer dropped/recreated in case of missing distributed tables in `distributed_incremental` materialization mode. ([#363](https://github.com/ClickHouse/dbt-clickhouse/pull/363))

### Release [1.8.4], 2024-09-17
### Improvement
* The S3 help macro now support a `role_arn` parameter as an alternative way to provide authentication for S3 based models. Thanks to
[Mitchell Bregman](https://github.com/mitchbregs) for the contribution!

### Release [1.8.3], 2024-09-01
### Bug Fixes
* An [issue](https://github.com/ClickHouse/dbt-clickhouse/issues/348) was detected when using multiple projections. We solved it and added a test to cover that use case. ([#349](https://github.com/ClickHouse/dbt-clickhouse/pull/349))
Expand Down Expand Up @@ -522,4 +597,4 @@ for Replicated tables that use the `{uuid}` macro in the path to avoid name conf
[0.19.1]: https://github.com/ClickHouse/dbt-clickhouse/compare/v0.19.0.2...v0.19.1
[0.19.0.2]: https://github.com/ClickHouse/dbt-clickhouse/compare/v0.19.0.1...v0.19.0.2
[0.19.0.1]: https://github.com/ClickHouse/dbt-clickhouse/compare/v0.19.0...v0.19.0.1
[0.19.0]: https://github.com/ClickHouse/dbt-clickhouse/compare/eb3020a...v0.19.0
[0.19.0]: https://github.com/ClickHouse/dbt-clickhouse/compare/eb3020a...v0.19.0
Loading
Loading