Increase Pod Memory on the Fly in Rancher
- Select Deployments from menu-sidebar
- Find correct deployment and click on three dots
- Edit Config
- Select Resources from deployment-sidebar
- Modify memory reservation and limit here
- Save
Kubernetes
Kubernetes: Restarting Pods
When making changes to config maps or other things - it’s tempting to restart all the pods for a particular service (select them all, delete, and let them come back up). This may work in the majority of cases, but if something goes wrong, it’ll be hard to tell if the recent change caused it or if it’s unrelated.
Recommendation:
- First check the logs in the pods to restart for any abnormal behavior
- Second restart a single pod before making any changes, make sure it comes up properly
- If the pod cannot restart without making any changes, debug the issue
- If the pod can restart, make the config change.
- Now restart (or kill) pods one at a time, ensuring each one enforces the new change.
The point here isn’t the specific process, but rather having checkpoints when changes are made. Ideally, by making only one change at a time and testing, it’s evident when something is broken which change caused it.
Access Command Line in Kubernetes Rancher
To access a shell in a particular pod:
- Workload-> Pods -> Find pod -> Click Three dots -> Execute Shell
To access a terminal to run kubectl commands in the environment:
- Click the
>_symbol to open a terminal.
Example command:
kubectl describe thingToDescribe -n NameSpace
Monitor Memory and CPU of a Pod in Rancher
If you have a kubernetes pod being terminated due to OOMKilled (Out of Memory Killed), you can monitor the memory usage of the pod during the action that is using too much memory.
- Workload -> Pods -> Find ther POD to monitor and click the name
- Click Metrics
- Set the range and refresh parameters (5s refresh during active monitoring)
- Run process (trigger it through an API call/Swagger endpoint)
- Monitor memory utilization during the process
Kubernetes Pod Dies Unexpectedly With No Error in Console Log
If you have a container in kubernetes unexpectedly disconnect with no error in the console log you can check the pod yaml for the last state of the pod.
Go to your Pod -> (three dots) Edit Yaml -> scroll to status (at the bottom) -> expand.
From here you can see the last status such as:
reason: OOMKilled
Check Spring Profiles in Rancher
Spring profiles can be used to toggle behaviors in a spring app. If your app is running in a container in Rancher (Kubernetes) you may need to see what spring profiles are being applied. To check what spring profiles a service is running in Rancher:
- Click Workload in the navigation bar -> Pods
- Search for your service and click on it
- Click related resources
- Click configMap
- Scroll to SPRING_PROFILES_ACTIVE
