-
Notifications
You must be signed in to change notification settings - Fork 125
feat: standard database functions everywhere #1750
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 14 commits
8cdad22
dc8994b
fa71952
f6a4de8
cea35b4
4ce5eaf
ad564e1
7cb9ea3
6c4b0c9
655939d
9bf08ae
2ba678d
fba6aab
096010c
46385f9
e9a63f8
1fc02aa
9696cb6
bdabd63
2322e91
0704002
0bdfb5a
31e9d4d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -22,9 +22,9 @@ impl-variants: true | |||||
|
||||||
<div class="impl node"> | ||||||
|
||||||
### Migrating to New Database Services? {.node} | ||||||
### Migrating to the `@cap-js/` Database Services? {.node} | ||||||
|
||||||
With CDS 8, the new database services for SQLite, PostgreSQL, and SAP HANA are now generally available. It's highly recommended to migrate. You can find instructions in the [migration guide](databases-sqlite#migration). Although the guide is written in the context of the new SQLite Service, the same hints apply to PostgreSQL and SAP HANA. | ||||||
With CDS 8, the [`@cap-js`](https://github.com/cap-js/cds-dbs) database services for SQLite, PostgreSQL, and SAP HANA are generally available. It's highly recommended to migrate. You can find instructions in the [migration guide](databases-sqlite#migration). Although the guide is written in the context of the SQLite Service, the same hints apply to PostgreSQL and SAP HANA. | ||||||
|
||||||
### Adding Database Packages {.node} | ||||||
|
||||||
|
@@ -358,63 +358,6 @@ The operator mappings are available for runtime queries only, but not in CDS fil | |||||
::: | ||||||
|
||||||
|
||||||
### Functions Mappings for Runtime Queries {.node} | ||||||
|
||||||
A specified set of standard functions is supported in a **database-agnostic**, hence portable way, and translated to database-specific variants or polyfills. | ||||||
Note that these functions are only supported within runtime queries, but not in CDS files. | ||||||
This set of functions are by large the same as specified in OData: | ||||||
|
||||||
* `concat(x,y,...)` — concatenates the given strings or numbers | ||||||
* `trim(x)` — removes leading and trailing whitespaces | ||||||
* `contains(x,y)` — checks whether `y` is contained in `x`, may be fuzzy | ||||||
* `startswith(x,y)` — checks whether `y` starts with `x` | ||||||
* `endswith(x,y)` — checks whether `y` ends with `x` | ||||||
* `matchespattern(x,y)` — checks whether `x` matches regex `y` | ||||||
* `substring(x,i,n?)` <sup>1</sup> — | ||||||
Extracts a substring from `x` starting at index `i` (0-based) with optional length `n`. | ||||||
* **`i`**: Positive starts at `i`, negative starts `i` before the end. | ||||||
* **`n`**: Positive extracts `n` items; omitted extracts to the end; negative is invalid. | ||||||
* `indexof(x,y)` <sup>1</sup> — returns the index of the first occurrence of `y` in `x` | ||||||
* `length(x)` — returns the length of string `x` | ||||||
* `tolower(x)` — returns all-lowercased `x` | ||||||
* `toupper(x)` — returns all-uppercased `x` | ||||||
* `ceiling(x)` — rounds the input numeric parameter up to the nearest numeric value | ||||||
* `floor(x)` — rounds the input numeric parameter down to the nearest numeric value | ||||||
* `round(x)` — rounds the input numeric parameter to the nearest numeric value. | ||||||
The mid-point between two integers is rounded away from zero, i.e. 0.5 is rounded to 1 and ‑0.5 is rounded to -1. | ||||||
* `year(x)` `month(x)`, `day(x)`, `hour(x)`, `minute(x)`, `second(x)` — | ||||||
returns parts of a datetime for a given `cds.DateTime` / `cds.Date` / `cds.Time` | ||||||
* `time(x)`, `date(x)` - returns a string representing the `time` / `date` for a given `cds.DateTime` / `cds.Date` / `cds.Time` | ||||||
* `fractionalseconds(x)` - returns a a `Decimal` representing the fractions of a second for a given `cds.Timestamp` | ||||||
* `maxdatetime()` - returns the latest possible point in time: `'9999-12-31T23:59:59.999Z'` | ||||||
* `mindatetime()` — returns the earliest possible point in time: `'0001-01-01T00:00:00.000Z'` | ||||||
* `totalseconds(x)` — returns the duration of the value in total seconds, including fractional seconds. The [OData spec](https://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part2-url-conventions.html#sec_totalseconds) defines the input as EDM.Duration: `P12DT23H59M59.999999999999S` | ||||||
* `now()` — returns the current datetime | ||||||
* `min(x)` `max(x)` `sum(x)` `average(x)` `count(x)`, `countdistinct(x)` — aggregate functions | ||||||
* `search(xs,y)` — checks whether `y` is contained in any of `xs`, may be fuzzy → [see Searching Data](../guides/providing-services#searching-data) | ||||||
* `session_context(v)` — with standard variable names → [see Session Variables](#session-variables) | ||||||
> <sup>1</sup> These functions work zero-based. E.g., `substring('abcdef', 1, 3)` returns 'bcd' | ||||||
|
||||||
> You have to write these functions exactly as given; all-uppercase usages aren't supported. | ||||||
|
||||||
In addition to the standard functions, which all `@cap-js` database services support, `@cap-js/sqlite` and `@cap-js/postgres` also support these common SAP HANA functions, to further increase the scope for portable testing: | ||||||
|
||||||
* `years_between` — Computes the number of years between two specified dates. | ||||||
* `months_between` — Computes the number of months between two specified dates. | ||||||
* `days_between` — Computes the number of days between two specified dates. | ||||||
* `seconds_between` — Computes the number of seconds between two specified dates. | ||||||
* `nano100_between` — Computes the time difference between two dates to the precision of 0.1 microseconds. | ||||||
|
||||||
The database service implementation translates these to the best-possible native SQL functions, thus enhancing the extent of **portable** queries. | ||||||
With open source and the new database service architecture, we also have methods in place to enhance this list by custom implementation. | ||||||
|
||||||
> For the SAP HANA functions, both usages are allowed: all-lowercase as given above, as well as all-uppercase. | ||||||
|
||||||
::: warning Runtime Only | ||||||
The function mappings are available for runtime queries only, but not in CDS files. | ||||||
::: | ||||||
|
||||||
|
||||||
### Session Variables {.node} | ||||||
|
||||||
The API shown below, which includes the function `session_context()` and specific pseudo variable names, is supported by **all** new database services, that is, *SQLite*, *PostgreSQL* and *SAP HANA*. | ||||||
|
@@ -940,7 +883,173 @@ Instead, they protect the integrity of your data in the database layer against p | |||||
→ Use [`@assert.target`](providing-services#assert-target) for corresponding input validations. | ||||||
::: | ||||||
|
||||||
## Standard Database Functions | ||||||
{ #functions-mappings-for-runtime-queries } | ||||||
|
||||||
A specified set of standard functions - inspired by [OData](https://docs.oasis-open.org/odata/odata/v4.01/odata-v4.01-part2-url-conventions.html#sec_StringandCollectionFunctions) and [SAP HANA](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-sql-reference-guide/alphabetical-list-of-functions?locale=en-US) - is supported in a **database-agnostic**, hence portable way, and translated to the best-possible native SQL functions or polyfills. | ||||||
|
||||||
Those functions are automatically mapped during runtime (Node.js) already today. | ||||||
To switch on the same mappings for your CDL files, you have to set: | ||||||
|
||||||
<Config>cds.cdsc.standardDatabaseFunctions = true</Config> | ||||||
|
||||||
::: tip | ||||||
this will be the default starting with `@sap/cds-compiler >= 9` and hence is only needed for earlier versions. | ||||||
::: | ||||||
|
||||||
<!-- | ||||||
TODO: remove the above with cds9 | ||||||
--> | ||||||
patricebender marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
### OData standard functions | ||||||
|
||||||
The `@sap/cds-compiler` and all CAP Node.js database services come with out of the box support for common OData functions. | ||||||
|
||||||
::: warning Case Sensitivity | ||||||
The OData function mappings are case-sensitive and must be written as in the list below. | ||||||
::: | ||||||
|
||||||
e.g. | ||||||
|
||||||
```cds | ||||||
entity V as select from Books { | ||||||
startswith(title, 'Raven') as lowerCase, // mapped to native SQL equivalent | ||||||
startsWith(title, 'Raven') as camelCase, // passed as-is | ||||||
} | ||||||
``` | ||||||
|
||||||
`❯ cds compile -2 sql --dialect hana` | ||||||
patricebender marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
```sql | ||||||
CREATE VIEW V AS SELECT | ||||||
(CASE WHEN locate(title, 'Raven') = 1 THEN TRUE ELSE FALSE END) AS lowerCase, | ||||||
-- the below will most likely fail on SAP HANA | ||||||
startsWith(title, 'Raven') AS camelCase | ||||||
FROM Books; | ||||||
``` | ||||||
|
||||||
💡 If you have your own User-Defined Functions (UDFs) with the same name, you can still use them, | ||||||
by deviating from the casing given below. | ||||||
renejeglinsky marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
#### String Functions | ||||||
|
||||||
- `concat(x, y, ...)` | ||||||
Concatenates the given strings or numbers. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes we support the concat operator |
||||||
|
||||||
- `trim(x)` | ||||||
Removes leading and trailing whitespaces. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. According to OData, this should be whitespace characters, according to Unicode rules. I think these are:
@patricebender - do you trim these characters or just whatever the DB considers to be a whitespace? I think the latter would be fine for most practical purposes. (Although we recently had an issue with a Chinese WS character (!)). At least we need to document which interpretation of whitespace character we apply. |
||||||
|
||||||
- `contains(x, y)` | ||||||
Checks whether `y` is contained in `x` (case-sensitive). | ||||||
|
||||||
- `startswith(x, y)` | ||||||
Checks whether `y` starts with `x` (case-sensitive). | ||||||
|
||||||
- `endswith(x, y)` | ||||||
Checks whether `y` ends with `x` (case-sensitive). | ||||||
|
||||||
- `matchespattern(x, y)` | ||||||
Checks whether `x` matches the regular expression `y`. | ||||||
|
||||||
- `indexof(x, y)` <sup>1</sup> | ||||||
Returns the index of the first occurrence of `y` in `x` (case-sensitive). | ||||||
|
||||||
- `substring(x, i, n?)` <sup>1</sup> | ||||||
Extracts a substring from `x` starting at index `i` (0-based) with an optional length `n`. | ||||||
- `i`: | ||||||
- Positive: starts at index `i` | ||||||
- Negative: starts `i` positions before the end | ||||||
patricebender marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
- `n`: | ||||||
- Positive: extracts `n` characters | ||||||
- Omitted: extracts until the end of the string | ||||||
- Negative: invalid | ||||||
patricebender marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
- `length(x)` | ||||||
Returns the length of the string `x`. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. currently |
||||||
|
||||||
- `tolower(x)` | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||||||
Converts all characters in `x` to lowercase. | ||||||
|
||||||
- `toupper(x)` | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||||||
Converts all characters in `x` to uppercase. | ||||||
|
||||||
<sup>1</sup> These functions work zero-based. E.g., `substring('abcdef', 1, 3)` returns 'bcd' | ||||||
|
||||||
#### Numeric Functions | ||||||
|
||||||
- `ceiling(x)` | ||||||
Rounds the numeric parameter up to the nearest integer. | ||||||
agoerler marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
- `floor(x)` | ||||||
Rounds the numeric parameter down to the nearest integer. | ||||||
|
||||||
- `round(x)` | ||||||
Rounds the numeric parameter to the nearest integer. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
https://www.postgresql.org/docs/16/functions-math.html Rounds to nearest integer. For numeric, ties are broken by rounding away from zero. For double precision, the tie-breaking behavior is platform dependent, but “round to nearest even” is the most common rule. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Java:
|
||||||
The midpoint between two integers is rounded away from zero (e.g., `0.5` → `1` and `-0.5` → `-1`). | ||||||
|
||||||
::: warning `round` function with more than one argument | ||||||
Note that most databases support `round` functions with multiple arguments, | ||||||
the second parameter being the precision. SAP HANA even has a third argument which for the rounding mode. | ||||||
If you provide more than one argument, the `round` function may behave differently depending on the database. | ||||||
::: | ||||||
|
||||||
#### Date and Time Functions | ||||||
|
||||||
- `year(x)`, `month(x)`, `day(x)`, `hour(x)`, `minute(x)`, `second(x)` | ||||||
Extracts and returns specific date / time parts as integer value from a given `cds.DateTime`, `cds.Date`, or `cds.Time`. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
an
could you elaborate? It always returns the requested part of the date as it is written in the given |
||||||
|
||||||
- `time(x)`, `date(x)` | ||||||
Extracts and returns a time or date from a given `cds.DateTime`, `cds.Date`, or `cds.Time`. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||||||
|
||||||
- `fractionalseconds(x)` | ||||||
Returns a `Decimal` representing the fractional seconds for a given `cds.Timestamp`. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. n/a in CAP Java |
||||||
|
||||||
- `maxdatetime()` | ||||||
Returns the latest possible point in time: `'9999-12-31T23:59:59.999Z'`. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. in OData, this functions returns an |
||||||
|
||||||
- `mindatetime()` | ||||||
Returns the earliest possible point in time: `'0001-01-01T00:00:00.000Z'`. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. in OData, this functions returns an DateTimeOffset which is defined to contain the fractional part: https://docs.oasis-open.org/odata/odata-csdl-xml/v4.01/csprd02/odata-csdl-xml-v4.01-csprd02.html#sec_DateTimeOffset |
||||||
|
||||||
#### Aggregate Functions | ||||||
|
||||||
- `min(x)`, `max(x)`, `sum(x)`, `average(x)`, `count(x)`, `countdistinct(x)` | ||||||
Standard aggregate functions used to calculate minimum, maximum, sum, average, count, and distinct count of values. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. returns
It would be useful to have a type |
||||||
|
||||||
|
||||||
### SAP HANA Functions | ||||||
|
||||||
In addition to the OData standard functions, the cds-compiler and all CAP Node.js database services come with | ||||||
out of the box support for some common SAP HANA functions, to further increase the scope for portable testing: | ||||||
|
||||||
::: warning Upper- and Lowercase are supported | ||||||
For the SAP HANA functions, both usages are allowed: all-lowercase as given above, as well as all-uppercase. | ||||||
::: | ||||||
|
||||||
- `years_between` | ||||||
Computes the number of years between two specified dates. ([link](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-sql-reference-guide/years-between-function-datetime?locale=en-US)) | ||||||
- `months_between` | ||||||
Computes the number of months between two specified dates. ([link](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-sql-reference-guide/months-between-function-datetime?locale=en-US)) | ||||||
- `days_between` | ||||||
Computes the number of days between two specified dates. ([link](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-sql-reference-guide/days-between-function-datetime?locale=en-US)) | ||||||
- `seconds_between` | ||||||
Computes the number of seconds between two specified dates. ([link](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-sql-reference-guide/seconds-between-function-datetime?locale=en-US)) | ||||||
- `nano100_between` | ||||||
Computes the time difference between two dates to the precision of 0.1 microseconds. ([link](https://help.sap.com/docs/hana-cloud-database/sap-hana-cloud-sap-hana-database-sql-reference-guide/nano100-between-function-datetime?locale=en-US)) | ||||||
|
||||||
### Special Runtime Functions | ||||||
|
||||||
In addition to the OData and SAP HANA standard functions, the **CAP runtimes** provides special functions that are only available for runtime queries: | ||||||
|
||||||
- `search(x, y)` | ||||||
Checks whether `y` is contained in any element of `x` (fuzzy matching may apply). | ||||||
See [Searching Data](../guides/providing-services#searching-data) for more details. | ||||||
|
||||||
- `session_context(<var>)` | ||||||
Utilizes standard variable names to maintain session context. | ||||||
Refer to [Session Variables](#session-variables) for additional information. | ||||||
|
||||||
- `now()` | ||||||
Returns the current timestamp. | ||||||
|
||||||
## Using Native Features { #native-db-functions} | ||||||
|
||||||
|
Uh oh!
There was an error while loading. Please reload this page.