Skip to content

Latest commit

 

History

History
34 lines (32 loc) · 8.67 KB

query.md

File metadata and controls

34 lines (32 loc) · 8.67 KB

QUERY

Below is a table containing all the details for the property group: Query

Property Name Description Default Value Run CdkDeploy When Changed
sleeper.query.s3.max-connections The maximum number of simultaneous connections to S3 from a single query runner. This is separated from the main one as it's common for a query runner to need to open more files at once. 1024 false
sleeper.query.processor.memory.mb The amount of memory in MB for the lambda that executes queries. true
sleeper.query.processor.timeout.seconds The timeout for the lambda that executes queries in seconds. 900 true
sleeper.query.processor.state.refresh.period.seconds The frequency with which the query processing lambda refreshes its knowledge of the system state (i.e. the partitions and the mapping from partition to files), in seconds. 60 false
sleeper.query.processor.results.batch.size The maximum number of records to include in a batch of query results send to the results queue from the query processing lambda. 2000 false
sleeper.query.processor.record.retrieval.threads The size of the thread pool for retrieving records in a query processing lambda. 10 false
sleeper.query.processor.cache.timeout.seconds The default amount of time in seconds the query executor's cache of partition and file reference information is valid for. After this it will time out and need refreshing. 60 false
sleeper.query.tracker.ttl.days This value is used to set the time-to-live on the tracking of the queries in the DynamoDB-based query tracker. 1 false
sleeper.query.results.bucket.expiry.days The length of time the results of queries remain in the query results bucket before being deleted. 7 true
sleeper.query.results.queue.visibility.timeout.seconds The visibility timeout in seconds of the query results queue. 900 true
sleeper.default.query.results.rowgroup.size The default value of the rowgroup size used when the results of queries are written to Parquet files. The value given below is 8MiB. This value can be overridden using the query config. 8388608 false
sleeper.default.query.results.page.size The default value of the page size used when the results of queries are written to Parquet files. The value given below is 128KiB. This value can be overridden using the query config. 131072 false
sleeper.query.warm.lambda.period.minutes The rate at which the query lambda runs to keep it warm (in minutes, must be >=1). This only applies when the KeepLambdaWarmStack is enabled 5 true
sleeper.query.role.arn The ARN for the role with required permissions to query sleeper. true
sleeper.query.websocket.api.url The URL of the WebSocket API for querying sleeper. true
sleeper.query.queue.url The URL of the queue responsible for sending a query to sleeper. true
sleeper.query.queue.arn The ARN of the queue responsible for sending a query to sleeper. true
sleeper.query.dlq.url The URL of the dead letter queue used when querying sleeper. true
sleeper.query.dlq.arn The ARN of the dead letter queue used when querying sleeper. true
sleeper.query.results.queue.url The URL of the queue responsible for retrieving results from a query sent to sleeper. true
sleeper.query.results.queue.arn The ARN of the queue responsible for retrieving results from a query sent to sleeper. true
sleeper.query.results.bucket The S3 Bucket name of the query results bucket. true
sleeper.query.tracker.table.name The name of the table responsible for tracking query progress. true
sleeper.query.warm.lambda.rule The name of the CloudWatch rule to trigger the query lambda to keep it warm. true
sleeper.query.leaf.partition.queue.url The URL of the queue responsible for sending a leaf partition query to sleeper. true
sleeper.query.leaf.partition.queue.arn The ARN of the queue responsible for sending a leaf partition query to sleeper. true
sleeper.query.leaf.partition.dlq.url The URL of the dead letter queue used when leaf partition querying sleeper. true
sleeper.query.leaf.partition.dlq.arn The ARN of the dead letter queue used when leaf partition querying sleeper. true