[8.7](backport #2729) [synthetics] Add lightweight monitor configuration options

docs/en/observability/synthetics-lightweight.asciidoc

@@ -3,33 +3,44 @@
 
 beta[]
 
-To configure lightweight monitors define a set of `monitors` to check your remote hosts.
-Each `monitor` item is an entry in a YAML list and begins with a dash (`-`).
-You can define the type of monitor to use, the hosts to check, and other
-optional settings.
+In the {synthetics-app} you can monitor the status of network endpoints using the following lightweight checks:
+
+* *HTTP*: Monitor your website. The HTTP monitor checks to make sure specific endpoints return the correct
+  status code and display the correct text.
+* *ICMP*: Check the availability of your hosts. The ICMP monitor uses ICMP (v4 and v6) Echo
+  Requests to check the network reachability of the hosts you are pinging. This will tell you whether the
+  host is available and connected to the network, but doesn't tell you if a service on the host is running or
+  not.
+* *TCP*: Monitor the services running on your hosts. The TCP monitor checks individual ports
+  to make sure the service is accessible and running.
 
-Lightweight monitors can be configured using any of the following:
+Lightweight monitors can be configured using either the <<synthetics-lightweight-ui>>
+or <<synthetics-lightweight-projects>>.
 
-* *YAML files in {project-monitors}*.
-  <<synthetics-get-started-project,Set up a project>> and configure monitors in YAML
-  files in the `lightweight` directory.
-* *{synthetics-app}*.
-  <<synthetics-get-started-ui,Go to the {synthetics-app}>> in {kib} and configure monitors.
+[discrete]
+[[synthetics-lightweight-ui]]
+= {synthetics-app}
+
+To use the UI in the {synthetics-app}, go to the {synthetics-app} in {kib} to create and configure monitors.
+For step-by-step instructions, refer to <<synthetics-get-started-ui>>.
+
+[role="screenshot"]
+image::images/uptime-set-up-ui.asciidoc.png[Synthetics Create monitor UI]
 
 [discrete]
-[[synthetics-monitor-types]]
-= Monitor types
+[[synthetics-lightweight-projects]]
+= {project-monitors-cap}
+
+To use YAML files in to create {project-monitors}, <<synthetics-get-started-project,set up a project>>
+and configure monitors in YAML files in the `lightweight` directory.
 
-`http`:: Connects via HTTP and optionally verifies that the host returns the
-expected response. Will use `Elastic-Heartbeat` as
-the user agent product.
-`icmp`:: Uses an ICMP (v4 and v6) Echo Request to ping the configured hosts.
-Requires special permissions or root access.
-`tcp`:: Connects via TCP and optionally verifies the endpoint by sending and/or
-receiving a custom payload.
+In each YAML file, define a set of `monitors` to check your remote hosts.
+Each `monitor` item is an entry in a YAML list and begins with a dash (`-`).
+You can define the type of monitor to use, the hosts to check, and other
+optional settings.
 
 The following example configures three monitors checking via the `http`, `icmp`, and `tcp`
-protocols and demonstrates how to use TCP Echo and HTTP response verification:
+protocols and demonstrates how to use TCP Echo response verification:
 
 [source,yaml]
 ----
@@ -51,32 +62,160 @@ heartbeat.monitors:
   check.send: "Check"
   check.receive: "Check"
   schedule: '@every 5s'
-heartbeat.scheduler:
-  limit: 10
 ----
 
-[discrete]
 [[synthetics-monitor-options]]
-= Configuration options
+There are some common monitor configuration options that are the same for all lightweight monitor types.
+For a complete list, refer to <<synthetics-lightweight-common-options>>.
 
-You can find a complete reference of available configuration options in
-the {heartbeat} documentation.
+Each monitor type also has additional configuration options that are specific to that type.
+Refer to:
 
-[NOTE]
-====
-While the most commonly used {heartbeat} configuration options are supported,
-some configuration options are not yet supported in {project-monitors} and the {synthetics-app}.
-====
+* <<synthetics-lightweight-http>>
+* <<synthetics-lightweight-icmp>>
+* <<synthetics-lightweight-tcp>>
 
-There are some common monitor configuration options that are the same for all monitors.
-For a complete list, refer to {heartbeat-ref}/monitor-options.html[Common monitor options].
+The `tcp` and `http` monitor types both support SSL/TLS and some proxy settings.
 
-Each monitor type has additional configuration options that are specific to that monitor type.
-Refer to:
+[discrete]
+[[synthetics-lightweight-common-options]]
+== Common options
+
+You can specify the following options when defining a synthetic monitor in any location.
+These options are the same for all monitors. Each monitor type has additional configuration
+options that are specific to that monitor type.
+
+// Reference table
+include::synthetics-reference/lightweight-config/common.asciidoc[]
+
+[discrete]
+[[synthetics-lightweight-http]]
+== HTTP options
+
+The options described here configure Synthetics to connect via HTTP and
+optionally verify that the host returns the expected response.
+
+Valid options for HTTP monitors include <<synthetics-lightweight-common-options,all common options>>
+and the following HTTP-specific options:
+
+// Reference table
+include::synthetics-reference/lightweight-config/http.asciidoc[]
+
+[discrete]
+[[synthetics-lightweight-icmp]]
+== ICMP options
+
+The options described here configure Synthetics to use ICMP (v4 and v6) Echo
+Requests to check the configured hosts. On most platforms you must execute
+Synthetics with elevated permissions to perform ICMP pings.
+
+On Linux, regular users may perform pings if the right file capabilities are set. Run
+`sudo setcap cap_net_raw+eip /path/to/heartbeat` to  grant Synthetics ping capabilities on Linux.
+Alternatively, you can grant ping permissions to the user being used to run Synthetics.
+To grant ping permissions in this way, run `sudo sysctl -w net.ipv4.ping_group_range='myuserid myuserid'`.
+
+Other platforms may require Synthetics to run as root or administrator to execute pings.
+
+Valid options for ICMP monitors include <<synthetics-lightweight-common-options,all common options>>
+and the following ICMP-specific options:
+
+// Reference table
+include::synthetics-reference/lightweight-config/icmp.asciidoc[]
+
+[discrete]
+[[synthetics-lightweight-tcp]]
+== TCP options
+
+The options described here configure Synthetics to connect via TCP and
+optionally verify the endpoint by sending and/or receiving a custom payload.
+
+Valid options for TCP monitors include <<synthetics-lightweight-common-options,all common options>>
+and the following TCP-specific options:
+
+// Reference table
+include::synthetics-reference/lightweight-config/tcp.asciidoc[]
+
+[discrete]
+[[synthetics-lightweight-data-types]]
+== Data types reference
+
+Values of configuration settings are interpreted as required by Synthetics.
+If a value can't be correctly interpreted as the required type - for example a
+string is given when a number is required - Synthetics will fail to start up.
+
+[discrete]
+[[synthetics-lightweight-data-bool]]
+=== Boolean
+
+Boolean values can be either `true` or `false`. Alternative names for `true` are
+`yes` and `on`. Instead of `false` the values `no` and `off` can be used.
+
+[source,yaml]
+-----
+enabled: true
+disabled: false
+-----
+
+[discrete]
+[[synthetics-lightweight-data-numbers]]
+=== Number
+
+Number values require you to enter the number _without_ single or
+double quotes. 
+
+[source,yaml]
+-----
+integer: 123
+negative: -1
+float: 5.4
+-----
+
+NOTE: Some settings only support a restricted number range.
+
+[discrete]
+[[synthetics-lightweight-data-string]]
+=== String
+
+In http://www.yaml.org[YAML], multiple styles of string definitions are supported:
+double-quoted, single-quoted, unquoted.
+
+The double-quoted style is specified by surrounding the string with `"`. This
+style provides support for escaping unprintable characters using `\`, but comes
+at the cost of having to escape `\` and `"` characters.
+
+The single-quoted style is specified by surrounding the string with `'`. This
+style supports no escaping (use `''` to quote a single quote). Only printable
+characters can be used when using this form.
+
+Unquoted style requires no quotes, but does not support any escaping and can't
+include any symbol that has a special meaning in YAML.
+
+NOTE: Single-quoted style is recommended when defining regular expressions,
+event format strings, windows file paths, or non-alphabetical symbolic characters.
+
+[discrete]
+[[synthetics-lightweight-data-duration]]
+=== Duration
+
+Durations require a numeric value with optional fraction and required unit.
+Valid time units are `ns`, `us`, `ms`, `s`, `m`, `h`. Sometimes features based
+on durations can be disabled by using zero or negative durations.
+
+[source,yaml]
+-----
+duration1: 2.5s
+duration2: 6h
+duration_disabled: -1s
+-----
+
+[discrete]
+[[synthetics-lightweight-data-regex]]
+=== Regular expression
 
-* {heartbeat-ref}/monitor-http-options.html[HTTP monitors]
-* {heartbeat-ref}/monitor-tcp-options.html[ICMP monitors]
-* {heartbeat-ref}/monitor-tcp-options.html[TCP monitors]
+Regular expressions are special strings that are compiled into regular
+expressions at load time.
 
-The `tcp` and `http` monitor types both support SSL/TLS and some proxy
-settings.
+As regular expressions and YAML use `\` for escaping
+characters in strings, it's highly recommended to use single quoted strings when
+defining regular expressions. When single quoted strings are used, the `\` character
+is not interpreted by YAML parser as an escape symbol.

docs/en/observability/synthetics-reference/lightweight-config/common.asciidoc

@@ -0,0 +1,140 @@
+:hardbreaks-option:
+
+[%header]
+|===
+| Option (type) | Description
+
+// type
+| [[monitor-type]] *`type`*
+(`"http"` \| `"icmp"` \| `"tcp"`)
+a| *Required*. The type of monitor to run. One of:
+
+* `http`: Connects via HTTP and optionally verifies that the host returns the expected response.
+* `icmp`: Uses an ICMP (v4 and v6) Echo Request to ping the configured hosts. Requires special permissions or root access.
+* `tcp`: Connects via TCP and optionally verifies the endpoint by sending and/or receiving a custom payload.
+
+////////////////////////
+id
+////////////////////////
+| [[monitor-id]] *`id`*
+(<<synthetics-lightweight-data-string,string>>)
+a| *Required*. A unique identifier for this configuration. This should not change with edits to the monitor configuration regardless of changes to any config fields.
+
+*Examples*:
+
+[source,yaml]
+----
+id: uploader-service
+----
+
+[source,yaml]
+----
+id: http://example.net
+----
+
+[NOTE]
+====
+When querying against indexed monitor data this is the field you will be aggregating with. It appears in the exported fields as `monitor.id`.
+
+If you do not set an `id` explicitly, the monitor's config will be hashed and a generated value will be used. This value will change with any options change to this monitor making aggregations over time between changes impossible. For this reason, it's recommended that you set this manually.
+====
+
+////////////////////////
+name
+////////////////////////
+| [[monitor-name]] *`name`*
+(<<synthetics-lightweight-data-string,string>>)
+a| Human readable name for this monitor.
+
+*Examples*: 
+
+[source,yaml]
+----
+name: Uploader service
+----
+
+[source,yaml]
+----
+name: Example website
+----
+
+////////////////////////
+service.name
+////////////////////////
+| [[monitor-service_name]] *`service.name`*
+(<<synthetics-lightweight-data-string,string>>)
+a| APM service name for this monitor. Corresponds to the `service.name` ECS field. Set this when monitoring an app that is also using APM to enable integrations between Synthetics and APM data in Kibana.
+
+////////////////////////
+enabled
+////////////////////////
+| [[monitor-enabled]] *`enabled`*
+(<<synthetics-lightweight-data-bool,boolean>>)
+a| Whether the module is enabled.
+
+*Default*: `true`
+
+*Example*:
+
+[source,yaml]
+----
+enabled: false
+----
+
+////////////////////////
+schedule
+////////////////////////
+| [[monitor-schedule]] *`schedule`*
+(<<synthetics-lightweight-data-duration,duration>>)
+a| *Required*. The task schedule.
+
+NOTE: Schedules with less than 1 minute resolution will be saved to the nearest minute. For example, `@every 5s` will be changed to `@every 60s` when the monitor is pushed using the CLI.
+
+*Example*:
+Run the task every 5 minutes from the time the monitor was started.
+
+[source,yaml]
+----
+schedule: @every 5m
+----
+
+////////////////////////
+timeout
+////////////////////////
+| [[monitor-timeout]] *`timeout`*
+(<<synthetics-lightweight-data-duration,duration>>)
+a| The total running time for each ping test. This is the total time allowed for testing the connection and exchanging data.
+
+*Default*: `16s`
+
+*Example*:
+
+[source,yaml]
+----
+timeout: 2m
+----
+
+////////////////////////
+tags
+////////////////////////
+| [[monitor-tags]] *`tags`*
+(list of <<synthetics-lightweight-data-string,string>>s)
+a| A list of tags that will be sent with the monitor event.
+
+*Examples*:
+
+[source,yaml]
+----
+tags:
+  - tag one
+  - tag two
+----
+
+[source,yaml]
+----
+tags: ["tag one", "tag two"]
+----
+
+|===
+
+:!hardbreaks-option: