monitoring

HighNodeCPUFrequency

Meaning

This alert triggers when the CPU frequency on a node exceeds 80% of the maximum frequency for more than 5 minutes.

Impact

High CPU frequency can indicate:

• High CPU utilization pushing the processor to higher frequencies
• Potential thermal issues causing frequency scaling
• Power management concerns affecting system stability
• Reduced CPU lifespan due to sustained high-frequency operation

Diagnosis

1. Identify the affected node and CPU:

   $ kubectl get nodes
   

2. Check the current CPU frequency on the node:

   $ kubectl debug node/<node-name> -it --image=registry.redhat.io/ubi8/ubi
   

   Then run inside the debug pod:

   $ cat /proc/cpuinfo | grep -i "cpu mhz"
   

3. Monitor CPU utilization and temperature:

   $ kubectl top nodes
   

   $ kubectl top pods --all-namespaces --sort-by=cpu
   

   Check system temperature (if available):

   $ sensors
   

4. Review node resource allocation:

   $ kubectl describe node <node-name>
   

5. Check for CPU-intensive workloads:

   $ ps aux --sort=-%cpu | head -20
   

Mitigation

Immediate actions:

• Monitor the CPU temperature to ensure it is within safe limits.
• Check if the high frequency is due to legitimate high CPU demand.
• Verify CPU settings if needed.

If the alert is triggered by high CPU utilization:

• Identify and analyze CPU-intensive pods.
• Consider redistributing workloads across nodes.
• Evaluate pod resource limits and requests.
• Scale horizontally if needed.

If you suspect thermal throttling:

• Check system cooling and ventilation.
• Monitor ambient temperature.
• Consider reducing workload temporarily.
• Verify proper thermal management settings.

If you cannot resolve the issue, see the following resources:

• OKD Help
• #virtualization Slack channel

This site is open source. Improve this page.