Skip to content

Commit 2c9a051

Browse files
ikheifets-splunkikheifets-splunk
authored
docs: add k8s scaling guide (#1087)
Signed-off-by: Ilya Kheifets <ikheifets@splunk.com>
1 parent 9433280 commit 2c9a051

File tree

4 files changed

+208
-11
lines changed

4 files changed

+208
-11
lines changed

‎docs/gettingstarted/mk8s/k8s-microk8s-scaling.md‎

Lines changed: 203 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,203 @@
1+
# Scaling SNMP with microk8s
2+
3+
The following guide is to present how to bootstrap master and replica nodes for microk8s cluster and to explore the possibilities of scaling SC4SNMP.
4+
5+
## Scaling criteria
6+
7+
Below is the formula that can help with deciding when to scale the system.
8+
9+
`2 * periodic_task_exec_time * inventory_size >= workers_count * task_period`
10+
11+
where:
12+
13+
* `inventory_size` - How many items we have on inventory (`values.yaml`).
14+
* `workers_count` - How many workers for `polling` / `walk` we have (pod workers).
15+
* `task_period` - `walk` / `polling` period time (`values.yaml`).
16+
* `periodic_task_exec_time` - Execution time of `polling` / `walk` task (metrics at screenshot).
17+
18+
![task exec time](../../images/sc4snmp_task_execution.png)
19+
20+
If the left side of equation is higher you need to scale `workers_count` or increase `task_period`.
21+
22+
23+
## Make microk8s cluster
24+
25+
### Bootstrap master node
26+
27+
1. Setup master node using following [guide](./k8s-microk8s.md).
28+
29+
2. Generate joining token for replicas:
30+
31+
```bash
32+
microk8s add-node
33+
```
34+
35+
3. After running `add-node` on stdout you will get a command (like `microk8s join <master_node>:25000/<token>`) that you need to remember.
36+
37+
### Bootstrap replica nodes
38+
39+
1. Installing microk8s on replica node:
40+
41+
```bash
42+
sudo snap install microk8s --classic --channel=1.30/stable
43+
sudo usermod -a -G microk8s $USER
44+
sudo chown -f -R $USER ~/.kube
45+
su - $USER
46+
```
47+
48+
2. After running `add-node` on **master node** you will get on stdout command that you need to run on your **replica node**:
49+
50+
```bash
51+
microk8s join <master_node>:25000/<token>
52+
```
53+
54+
3. Check that replica joined cluster:
55+
56+
```bash
57+
microk8s kubectl get nodes
58+
```
59+
60+
New nodes should appear:
61+
62+
```bash
63+
NAME STATUS ROLES AGE VERSION
64+
i-05ecfbf799e480188 Ready <none> 25h v1.30.5
65+
i-0733cb329576e6c78 Ready <none> 25h v1.30.5
66+
i-0b27bcc06fc5c660e Ready <none> 25h v1.30.5
67+
```
68+
69+
## Scaling SC4SNMP
70+
71+
1. [Install SC4SNMP](../sc4snmp-installation.md) if it is not installed yet.
72+
73+
2. Add `worker` section on `values.yaml`:
74+
75+
```yaml
76+
worker:
77+
poller:
78+
replicaCount: 4
79+
trap:
80+
replicaCount: 4
81+
sender:
82+
replicaCount: 4
83+
```
84+
85+
3. Add `traps` replica count in `values.yaml`:
86+
87+
```yaml
88+
traps:
89+
replicaCount: 4
90+
```
91+
92+
4. Redeploy SC4SNMP:
93+
94+
```bash
95+
microk8s helm3 upgrade --install snmp -f values.yaml splunk-connect-for-snmp/splunk-connect-for-snmp --namespace=sc4snmp --create-namespace
96+
```
97+
98+
5. Check that SC4SNMP has been scaled:
99+
100+
```bash
101+
microk8s kubectl get pods -n sc4snmp
102+
```
103+
104+
You should get 4 replicas for each worker and traps service:
105+
106+
```bash
107+
NAME READY STATUS RESTARTS AGE
108+
snmp-mibserver-5df74fb678-zkj9m 1/1 Running 0 25h
109+
snmp-mongodb-6dc5c4f74d-xg6p7 2/2 Running 0 25h
110+
snmp-redis-master-0 1/1 Running 0 25h
111+
snmp-splunk-connect-for-snmp-inventory-k9t87 0/1 Completed 0 3m
112+
snmp-splunk-connect-for-snmp-scheduler-76848cf748-57qbx 1/1 Running 0 25h
113+
snmp-splunk-connect-for-snmp-trap-9f55664c4-9dv7d 1/1 Running 0 3m1s
114+
snmp-splunk-connect-for-snmp-trap-9f55664c4-crgld 1/1 Running 0 3m1s
115+
snmp-splunk-connect-for-snmp-trap-9f55664c4-sb768 1/1 Running 0 25h
116+
snmp-splunk-connect-for-snmp-trap-9f55664c4-tkhcp 1/1 Running 0 3m1s
117+
snmp-splunk-connect-for-snmp-worker-poller-7487956697-4hvpl 1/1 Running 0 21h
118+
snmp-splunk-connect-for-snmp-worker-poller-7487956697-8bvnn 1/1 Running 0 3m1s
119+
snmp-splunk-connect-for-snmp-worker-poller-7487956697-9dfgt 1/1 Running 0 3m1s
120+
snmp-splunk-connect-for-snmp-worker-poller-7487956697-hlhvz 1/1 Running 0 24h
121+
snmp-splunk-connect-for-snmp-worker-sender-657589666f-979d2 1/1 Running 0 3m1s
122+
snmp-splunk-connect-for-snmp-worker-sender-657589666f-mrvg9 1/1 Running 0 3m1s
123+
snmp-splunk-connect-for-snmp-worker-sender-657589666f-qtcr8 1/1 Running 0 21h
124+
snmp-splunk-connect-for-snmp-worker-sender-657589666f-tc8sv 1/1 Running 0 24h
125+
snmp-splunk-connect-for-snmp-worker-trap-859dc47d9b-6fbs2 1/1 Running 0 24h
126+
snmp-splunk-connect-for-snmp-worker-trap-859dc47d9b-kdcdb 1/1 Running 0 3m1s
127+
snmp-splunk-connect-for-snmp-worker-trap-859dc47d9b-sfxvb 1/1 Running 0 3m
128+
snmp-splunk-connect-for-snmp-worker-trap-859dc47d9b-xmmwv 1/1 Running 0 21h
129+
```
130+
131+
## Autoscaling SC4SNMP
132+
133+
1. [Install SC4SNMP](../sc4snmp-installation.md) if it is not installed yet.
134+
135+
2. Add autoscaling options to `values.yaml`:
136+
137+
```yaml
138+
worker:
139+
poller:
140+
autoscaling:
141+
enabled: true
142+
minReplicas: 5
143+
maxReplicas: 10
144+
trap:
145+
autoscaling:
146+
enabled: true
147+
minReplicas: 5
148+
maxReplicas: 10
149+
sender:
150+
autoscaling:
151+
enabled: true
152+
minReplicas: 5
153+
maxReplicas: 10
154+
155+
traps:
156+
autoscaling:
157+
enabled: true
158+
minReplicas: 5
159+
maxReplicas: 10
160+
```
161+
162+
3. Redeploy SC4SNMP:
163+
164+
```bash
165+
microk8s helm3 upgrade --install snmp -f values.yaml splunk-connect-for-snmp/splunk-connect-for-snmp --namespace=sc4snmp --create-namespace
166+
```
167+
168+
4. Checked that SC4SNMP scaled:
169+
170+
```bash
171+
microk8s kubectl get po -n sc4snmp
172+
```
173+
174+
After scaling of each worker and trap service 5-10 instances will appear:
175+
176+
```bash
177+
NAME READY STATUS RESTARTS AGE
178+
snmp-mibserver-6fdcdf9ddd-7bvmj 1/1 Running 0 25h
179+
snmp-mongodb-6dc5c4f74d-6b7mf 2/2 Running 0 25h
180+
snmp-redis-master-0 1/1 Running 0 25h
181+
snmp-splunk-connect-for-snmp-inventory-sssgs 0/1 Completed 0 3m37s
182+
snmp-splunk-connect-for-snmp-scheduler-5fcb6dcb44-r79ff 1/1 Running 0 25h
183+
snmp-splunk-connect-for-snmp-trap-5788bc498c-62xsq 1/1 Running 0 2m10s
184+
snmp-splunk-connect-for-snmp-trap-5788bc498c-bmlhg 1/1 Running 0 2m10s
185+
snmp-splunk-connect-for-snmp-trap-5788bc498c-p7mkq 1/1 Running 0 2m10s
186+
snmp-splunk-connect-for-snmp-trap-5788bc498c-t8q9c 1/1 Running 0 2m10s
187+
snmp-splunk-connect-for-snmp-trap-5788bc498c-xjjp2 1/1 Running 0 24h
188+
snmp-splunk-connect-for-snmp-worker-poller-5d76b9b675-25tbf 1/1 Running 0 16m
189+
snmp-splunk-connect-for-snmp-worker-poller-5d76b9b675-dc6zr 1/1 Running 0 16m
190+
snmp-splunk-connect-for-snmp-worker-poller-5d76b9b675-g7vpr 1/1 Running 0 16m
191+
snmp-splunk-connect-for-snmp-worker-poller-5d76b9b675-gdkgq 1/1 Running 0 16m
192+
snmp-splunk-connect-for-snmp-worker-poller-5d76b9b675-pg6cj 1/1 Running 0 24h
193+
snmp-splunk-connect-for-snmp-worker-sender-7757fb7f89-56h9w 1/1 Running 0 24h
194+
snmp-splunk-connect-for-snmp-worker-sender-7757fb7f89-hr54w 1/1 Running 0 16m
195+
snmp-splunk-connect-for-snmp-worker-sender-7757fb7f89-j7wcn 1/1 Running 0 16m
196+
snmp-splunk-connect-for-snmp-worker-sender-7757fb7f89-sgsdg 0/1 Pending 0 16m
197+
snmp-splunk-connect-for-snmp-worker-sender-7757fb7f89-xrpfx 1/1 Running 0 16m
198+
snmp-splunk-connect-for-snmp-worker-trap-6b8fd89868-79x2l 0/1 Pending 0 16m
199+
snmp-splunk-connect-for-snmp-worker-trap-6b8fd89868-br7pf 1/1 Running 0 24h
200+
snmp-splunk-connect-for-snmp-worker-trap-6b8fd89868-cnmh9 0/1 Pending 0 16m
201+
snmp-splunk-connect-for-snmp-worker-trap-6b8fd89868-dhdgg 1/1 Running 0 16m
202+
snmp-splunk-connect-for-snmp-worker-trap-6b8fd89868-wcwq5 0/1 Pending 0 16m
203+
```

‎docs/gettingstarted/mk8s/k8s-microk8s.md‎

Lines changed: 4 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -42,17 +42,6 @@ Wait for Installation of Mk8S to complete:
4242
microk8s status --wait-ready
4343
```
4444

45-
## Add nodes (optional)
46-
47-
* Repeat the steps above for each additional node (with a minimum of 3 nodes).
48-
* On the first node, use the following command to see the instructions to join:
49-
50-
```bash
51-
microk8s add-node
52-
```
53-
54-
* On each additional node, use the output from the command above.
55-
5645
## Install required services for SC4SNMP
5746

5847
The following commands can be issued from any one node in a cluster:
@@ -87,3 +76,7 @@ the same as the primary IP.
8776
microk8s enable metallb
8877
microk8s status --wait-ready
8978
```
79+
80+
## Add nodes (optional)
81+
82+
If you need cluster mode please use following [guide](k8s-microk8s-scaling.md#make-microk8s-cluster).

‎docs/images/sc4snmp_task_execution.png‎

338 KB
Loading

‎mkdocs.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,7 @@ nav:
4242
- Install Splunk OpenTelemetry Collector for Kubernetes: "gettingstarted/sck-installation.md"
4343
- Install SC4SNMP: "gettingstarted/sc4snmp-installation.md"
4444
- Enable IPv6: "gettingstarted/enable-ipv6.md"
45+
- Scaling with Microk8s: "gettingstarted/mk8s/k8s-microk8s-scaling.md"
4546
- Configuration:
4647
- Deployment: "configuration/deployment-configuration.md"
4748
- Configurable values: "configuration/values-params-description.md"

0 commit comments

Comments
 (0)