Skip to content

Commit 19a96ad

Browse files
authored
Document automatically create network policies to allow Prometheus scraping traffic for workloads annotated with prometheus.io/scrape (#292)
1 parent efa24cc commit 19a96ad

File tree

4 files changed

+103
-35
lines changed

4 files changed

+103
-35
lines changed

docs/features/network-mapping-network-policies/reference/Network-Policies-Deep-Dive.mdx

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -178,6 +178,51 @@ would block existing traffic like **loadgenerator** -> **frontend**.
178178
179179
180180
181+
## Auto-generating network policies for Prometheus server
182+
When an intent generates a network policy that can block Prometheus from querying metrics, the intents operator by default
183+
will automatically generate network policies to enable Prometheus to access the metrics port on this workload.
184+
185+
The way Otterize identifies that a workloads (pods, services) would be blocked from Prometheus is composed of 2 conditions:
186+
1. The workload has the `prometheus.io/scrape` annotation set to `true`
187+
2. `prometheusServerConfigs` is set in the intents operator configuration to identify the Prometheus server, as [documented in the intents operator](/reference/intents-operator).
188+
189+
To disable this feature, consult the [documentation for the intents operator](/reference/intents-operator).
190+
191+
192+
Let's look at an example.
193+
In our cluster we have client, server and Prometheus server.
194+
The server has the annotations `prometheus.io/scrape: "true"` and `prometheus.io/port: "8080"`.
195+
196+
By applying the following intents file:
197+
```yaml
198+
apiVersion: k8s.otterize.com/v2beta1
199+
kind: ClientIntents
200+
metadata:
201+
name: client
202+
namespace: prometheus-test
203+
spec:
204+
workload:
205+
name: client
206+
targets:
207+
- service:
208+
name: server
209+
```
210+
211+
Otterize will generate a network policy allowing access from the **client** workload to the **server** service.
212+
Once a network policy matches a pod, other traffic not allowed by existing network policies to the **server** will be blocked.
213+
In our case, this means that the Prometheus server will not be able to communicate with the server and scrape its metrics.
214+
215+
To resolve this, Otterize will automatically generate a network policy allowing traffic from the **Prometheus server** to the **server**.
216+
This is based on the presence of the `prometheus.io/scrape: "true"` annotation in the **server** and the `prometheusServerConfigs` configuration
217+
that identifies the Prometheus server.
218+
219+
Important note:
220+
If the **server** does not have the `prometheus.io/port` annotation, the network policy generated by Otterize will allow communication between the
221+
**Prometheus server** and the **server** on all ports exposed by the server. This is not a best practice, so it's recommended to set the port explicitly
222+
if known.
223+
224+
225+
181226
## How intents translate to network policies
182227
Let's follow an example scenario and track how Otterize configures network policies when we apply intents.
183228
### Deploy example

docs/reference/intents-operator/README.mdx

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,11 @@ to access the server. This label is used as the selector to determine which clie
4141
pod.
4242

4343
Learn more: [Network policies deep dive](/features/network-mapping-network-policies/reference/Network-Policies-Deep-Dive)
44+
45+
### Handling Third Party or Infrastructure traffic
46+
Otterize automates network policies that enables third party or infrastructure traffic.
47+
Otterize can currently automatically handle [external traffic](#Handling external traffic) as well as [Prometheus monitoring traffic](#Handling Prometheus traffic).
48+
4449
#### Handling external traffic
4550
The intents operator has automatic behavior for allowing external traffic for pods which have indicated that they are supposed to accept external traffic, such as by creating a `Service` of type `NodePort` or `LoadBalancer`, or an `Ingress` resource.
4651

@@ -50,7 +55,23 @@ As the intents operator creates network policies, and the semantics of network p
5055

5156
This meant that if you had no network policies on a pod, and created ClientIntents for that pod, then external traffic would be blocked. To make it easy to enable pod-to-pod traffic without affecting expected external traffic, the intents operator automatically detects resources of kind `Service` of type `NodePort` or `LoadBalancer`, or an `Ingress` resource, and if it creates the first network policy to affect those pods, it also creates a network policy that allows external traffic to those pods, as specified by the external `Service`/`Ingress` - for example, it only allows traffic to the specified ports, not all traffic.
5257

53-
This behavior can be disabled using the Helm chart's values.
58+
This behavior can be disabled using the Helm chart's values - `automateThirdPartyNetworkPolicies`.
59+
60+
#### Handling Prometheus traffic
61+
The intents operator has automatic behavior for allowing Prometheus traffic to pods which have indicated that they are supposed to accept Prometheus traffic, by adding the `prometheus.io/scrape: "true"` annotation to the pod or service.
62+
63+
As the intents operator creates network policies, and the semantics of network policies dictate that:
64+
1. if no network policies apply to a pod, then all traffic is allowed.
65+
2. once any network policy applies to a pod, only the traffic explicitly allowed in the policy is allowed
66+
67+
This meant that if you had no network policies on a pod, and created ClientIntents for that pod, then Prometheus scraping would be blocked. To make it easy to enable traffic for Prometheus metrics scraping, the intents operator automatically detects resources with `prometheus.io/scrape: "true"` annotation, and if it creates the first network policy to affect those pods, it also creates a network policy that allows Prometheus scraping.
68+
69+
You must set `prometheusServerConfigs` in the Helm chart, so Otterize would be able to identify the Prometheus server in the cluster and restrict the network policy to only allow traffic from Prometheus.
70+
71+
This behavior can be disabled using the Helm chart's values - `automateThirdPartyNetworkPolicies`, or to only disable Prometheus, make sure `prometheusServerConfigs` is not set.
72+
73+
**Note**: If you haven't specified `prometheus.io/port` in the pod or service, Otterize will match Prometheus' behavior by allowing all ports.
74+
To improve security, specify `prometheus.io/port` to require Prometheus to only connect to that port. Otterize will the only allow access to that port.
5475

5576
### AWS IAM
5677
The intents operator enables automatic management of AWS IAM roles and policies according to the

0 commit comments

Comments
 (0)