Skip to content

Commit f0f51a0

Browse files
Create sparksession.md
1 parent 87980a6 commit f0f51a0

File tree

1 file changed

+22
-0
lines changed

1 file changed

+22
-0
lines changed

docs/sparksession.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# SparkSession Function
2+
Submits application to Apache Spark cluster with config options.
3+
4+
# Arguments
5+
- `master::String`: the URL to the Spark cluster master.
6+
- `appName::String`: the name of the application.
7+
- `config::Dict{String, String}()`: one or more config option for the SparkSession.
8+
9+
# Examples
10+
Basic spark session:
11+
```
12+
spark = SparkSession("spark://example.com:7077", "Julia SparkSQL Example App")
13+
```
14+
15+
Spark Session with Hive config:
16+
```
17+
spark = SparkSession("spark://example.com:7077", "Julia SparkSQL Example App", Dict{String,String}("spark.sql.warehouse.dir"=>"LOCATION","spark.sql.catalogImplementation"=>"hive"))
18+
```
19+
Spark Session with Delta Lake config:
20+
```
21+
spark = SparkSession("spark://example.com:7077", "Julia SparkSQL Example App", Dict{String,String}("spark.sql.warehouse.dir"=>"LOCATION", "spark.sql.extensions"=>"io.delta.sql.DeltaSparkSessionExtension", "spark.sql.catalog.spark_catalog"=>"org.apache.spark.sql.delta.catalog.DeltaCatalog"))
22+
```

0 commit comments

Comments
 (0)