[SPARK-39577][SQL][DOCS] Add SQL reference for built-in functions

### What changes were proposed in this pull request?
Currently, Spark SQL reference missing many functions.
Users cannot find the needed functions.

### Why are the changes needed?
Add SQL reference for built-in functions

### Does this PR introduce _any_ user-facing change?
'Yes'.
Users can find needed functions in SQL reference.
Before this PR, the built-in functions show below.
![image](https://user-images.githubusercontent.com/8486025/175505440-a375dbe6-988c-4647-836d-746c681be19a.png)

After this PR, the built-in functions show below.
![image](https://user-images.githubusercontent.com/8486025/175916778-d55ab1eb-fd28-4362-a0b7-7e33f37eacb4.png)

The part of Mathematical Functions show below.
![image](https://user-images.githubusercontent.com/8486025/175760329-1b185eca-3d92-4c4f-ba2d-980200bf1a5b.png)
![image](https://user-images.githubusercontent.com/8486025/175760340-e93ea083-90e7-4710-9bf4-f45c2b57f8bc.png)
![image](https://user-images.githubusercontent.com/8486025/175760345-638b8fb2-d7f6-4e51-943e-7915583c03db.png)
![image](https://user-images.githubusercontent.com/8486025/175760355-99d7125a-dcdb-407f-8c8d-6fbd2ca60801.png)

The part of String Functions show below.
![image](https://user-images.githubusercontent.com/8486025/175760377-2e26454f-75d1-4ad4-9fdd-060a0460d439.png)
![image](https://user-images.githubusercontent.com/8486025/175760387-125483a4-0f13-45f8-9e60-4c66f1f3dc6f.png)
![image](https://user-images.githubusercontent.com/8486025/175760396-44d0f0b2-645b-408b-bdd7-ac167f98a30a.png)
![image](https://user-images.githubusercontent.com/8486025/175760405-c44b0661-c73f-437f-aa99-113dd25bb3fb.png)
![image](https://user-images.githubusercontent.com/8486025/175760416-66af659d-7db8-4c89-8686-9bf8bf1fec20.png)
![image](https://user-images.githubusercontent.com/8486025/175760422-6105ea4a-5ae1-42ef-aee2-1a88984b2c10.png)
![image](https://user-images.githubusercontent.com/8486025/175760428-5ab34058-ac78-48c6-aaa0-9a21b324822f.png)

The part of Bitwise Functions show below.
![image](https://user-images.githubusercontent.com/8486025/175760440-10623eff-6330-4407-b069-f74e29966b64.png)

The part of Conversion Functions show below.
![image](https://user-images.githubusercontent.com/8486025/175760458-384267f7-b7bb-44a7-8200-3c5e04dbd71a.png)

The part of Conditional Functions show below.
![image](https://user-images.githubusercontent.com/8486025/175762759-7546e799-d530-47ea-ac22-27dabd1fcf4a.png)

The part of Predicate Functions show below.
![image](https://user-images.githubusercontent.com/8486025/175760534-7a996e8e-188c-44de-b8e1-e36f346ae58e.png)
![image](https://user-images.githubusercontent.com/8486025/175760545-96222945-0dbf-4885-b23e-3043791f13d5.png)

The part of Csv Functions show below.
![image](https://user-images.githubusercontent.com/8486025/175760588-e0860cf4-0457-4f22-a21c-880c7ef92db2.png)

The part of Misc Functions show below.
![image](https://user-images.githubusercontent.com/8486025/175760614-93b5d9da-fed3-41d6-84c5-4d207b2f1175.png)
![image](https://user-images.githubusercontent.com/8486025/175760623-fb699de6-6174-496f-b15d-43155f223ee6.png)

The part of Generator Functions show below.
![image](https://user-images.githubusercontent.com/8486025/175760569-5f14bf3b-5844-4821-acce-232f6ce21372.png)

### How was this patch tested?
N/A

Closes apache#36976 from beliefer/SPARK-39577.

Authored-by: Jiaan Geng <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>

Author: beliefer

docs/sql-ref-functions-builtin.md

@@ -77,3 +77,92 @@ license: |
     {% endif %}
 {% endfor %}
 
+{% for static_file in site.static_files %}
+    {% if static_file.name == 'generated-math-funcs-table.html' %}
+### Mathematical Functions
+{% include_relative generated-math-funcs-table.html %}
+#### Examples
+{% include_relative generated-math-funcs-examples.html %}
+        {% break %}
+    {% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+    {% if static_file.name == 'generated-string-funcs-table.html' %}
+### String Functions
+{% include_relative generated-string-funcs-table.html %}
+#### Examples
+{% include_relative generated-string-funcs-examples.html %}
+        {% break %}
+    {% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+    {% if static_file.name == 'generated-conditional-funcs-table.html' %}
+### Conditional Functions
+{% include_relative generated-conditional-funcs-table.html %}
+#### Examples
+{% include_relative generated-conditional-funcs-examples.html %}
+        {% break %}
+    {% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+    {% if static_file.name == 'generated-bitwise-funcs-table.html' %}
+### Bitwise Functions
+{% include_relative generated-bitwise-funcs-table.html %}
+#### Examples
+{% include_relative generated-bitwise-funcs-examples.html %}
+        {% break %}
+    {% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+    {% if static_file.name == 'generated-conversion-funcs-table.html' %}
+### Conversion Functions
+{% include_relative generated-conversion-funcs-table.html %}
+#### Examples
+{% include_relative generated-conversion-funcs-examples.html %}
+        {% break %}
+    {% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+    {% if static_file.name == 'generated-predicate-funcs-table.html' %}
+### Predicate Functions
+{% include_relative generated-predicate-funcs-table.html %}
+#### Examples
+{% include_relative generated-predicate-funcs-examples.html %}
+        {% break %}
+    {% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+    {% if static_file.name == 'generated-csv-funcs-table.html' %}
+### Csv Functions
+{% include_relative generated-csv-funcs-table.html %}
+#### Examples
+{% include_relative generated-csv-funcs-examples.html %}
+        {% break %}
+    {% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+    {% if static_file.name == 'generated-misc-funcs-table.html' %}
+### Misc Functions
+{% include_relative generated-misc-funcs-table.html %}
+#### Examples
+{% include_relative generated-misc-funcs-examples.html %}
+        {% break %}
+    {% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+    {% if static_file.name == 'generated-generator-funcs-table.html' %}
+### Generator Functions
+{% include_relative generated-generator-funcs-table.html %}
+#### Examples
+{% include_relative generated-generator-funcs-examples.html %}
+        {% break %}
+    {% endif %}
+{% endfor %}

docs/sql-ref-functions.md

@@ -32,11 +32,22 @@ This subsection presents the usages and descriptions of these functions.
  * [Map Functions](sql-ref-functions-builtin.html#map-functions)
  * [Date and Timestamp Functions](sql-ref-functions-builtin.html#date-and-timestamp-functions)
  * [JSON Functions](sql-ref-functions-builtin.html#json-functions)
+ * [Mathematical Functions](sql-ref-functions-builtin.html#mathematical-functions)
+ * [String Functions](sql-ref-functions-builtin.html#string-functions)
+ * [Bitwise Functions](sql-ref-functions-builtin.html#bitwise-functions)
+ * [Conversion Functions](sql-ref-functions-builtin.html#conversion-functions)
+ * [Conditional Functions](sql-ref-functions-builtin.html#conditional-functions)
+ * [Predicate Functions](sql-ref-functions-builtin.html#predicate-functions)
+ * [Csv Functions](sql-ref-functions-builtin.html#csv-functions)
+ * [Misc Functions](sql-ref-functions-builtin.html#misc-functions)
 
 #### Aggregate-like Functions
  * [Aggregate Functions](sql-ref-functions-builtin.html#aggregate-functions)
  * [Window Functions](sql-ref-functions-builtin.html#window-functions)
 
+#### Generator Functions
+* [Generator Functions](sql-ref-functions-builtin.html#generator-functions)
+
 ### UDFs (User-Defined Functions)
 
 User-Defined Functions (UDFs) are a feature of Spark SQL that allows users to define their own functions when the system's built-in functions are not enough to perform the desired task. To use UDFs in Spark SQL, users must first define the function, then register the function with Spark, and finally call the registered function. The User-Defined Functions can act on a single row or act on multiple rows at once. Spark SQL also supports integration of existing Hive implementations of UDFs, UDAFs and UDTFs.

sql/gen-sql-functions-docs.py

@@ -31,6 +31,9 @@
 groups = {
     "agg_funcs", "array_funcs", "datetime_funcs",
     "json_funcs", "map_funcs", "window_funcs",
+    "math_funcs", "conditional_funcs", "generator_funcs",
+    "predicate_funcs", "string_funcs", "misc_funcs",
+    "bitwise_funcs", "conversion_funcs", "csv_funcs",
 }
 
 
@@ -45,6 +48,8 @@ def _list_grouped_function_infos(jvm):
 
     for jinfo in filter(lambda x: x.getGroup() in groups, jinfos):
         name = jinfo.getName()
+        if (name == "raise_error"):
+            continue
         usage = jinfo.getUsage()
         usage = usage.replace("_FUNC_", name) if usage is not None else usage
         infos.append(ExpressionInfo(
@@ -108,7 +113,12 @@ def _make_pretty_usage(infos):
         # Expected formats are as follows;
         #  - `_FUNC_(...) - description`, or
         #  - `_FUNC_ - description`
-        usages = iter(re.split(r"(%s.*) - " % info.name, info.usage.strip())[1:])
+        func_name = info.name
+        if (info.name == "*" or info.name == "+"):
+            func_name = "\\" + func_name
+        elif (info.name == "when"):
+            func_name = "CASE WHEN"
+        usages = iter(re.split(r"(.*%s.*) - " % func_name, info.usage.strip())[1:])
         for (sig, description) in zip(usages, usages):
             result.append("    <tr>")
             result.append("      <td>%s</td>" % sig)