|
|
| (10 intermediate revisions by the same user not shown) |
| Line 1: |
Line 1: |
| | + | [[Welcome_to_Veloopti_help|Home]] > [[:Category:Administration|Administration]] > [[:Category:Event Administration|Event Administration]] > [[Event storm|Event storm]] |
| | + | ---- |
| | + | |
| | == Overview == | | == Overview == |
| − | An event storm is ''a large number of informational, warning and exception events of the same type from one or more nodes over a relatively short period of time''. The events will appear in your Veloopti organisation either as individual events or as duplicates of events. | + | An event storm is ''a large number of informational, warning and exception events of the same type from one or more nodes over a relatively short period of time''. The events will appear in your Veloopti organisation either as individual events or as duplicates of events. The factors with detecting an event storm are the number of events that match the rule, called the ''Breach count'', and the time window that they are detected within, called the ''Breach duration''. |
| − | | |
| − | The main factors with detecting an event storm are the number of events that match the rule and the time window that they are detected within. | |
| − | | |
| − | Storm events can increase and decrease in severity.
| |
| − | | |
| − | == Where event storms come from ==
| |
| − | Each minute the number of events that are received that match the path of the storm rule are added together. When adding the events together it does not matter whether they are new events or [[Event#Previous events and duplicates|duplicate]].
| |
| − | | |
| − | == How event storms are detected ==
| |
| − | The informational, warning or exception breach duration windows are compared with the total number of actual events that have been received during each specified breach duration time window. If a breach is detected then the one with the highest severity either creates a new storm event or increments the duplicate count of the pre-existing one.
| |
| − | | |
| − | ==== Changing severity while the storm is still active ====
| |
| − | Once a storm rule is breached it will continue to be monitored for the current number of events during its breach duration window to determine whether the severity of the event needs to change.
| |
| − | | |
| − | === Ending the storm ===
| |
| − | When the current event count for the path no longer exceed any of the informational, warning or exception breach duration thresholds the reset conditions are then compared to the current event count to determine if the event can be closed. If no reset condition is met then the storm event remains open with the pre-existing severity.
| |
| − | | |
| − | === Notifications ===
| |
| − | Notifications will occur once with any new storm event or increase of severity level of a current storm event. Therefore a storm event that is opened with a severity of information will initially send a notification when it is first opened. It will also send a notification if the severity is increased to warning and again if the severity is increased to exception. Whereas a storm event that is opened with a severity level of warning would not send a notification if the event decreased severity to information however it would send a notification if the event severity increased to exception. A storm event that is opened as a exception would not notify if it reduces in severity to either warning or normal.
| |
| − | | |
| − | == Event storm properties ==
| |
| − | A storm rule is opened by clicking on its name on the [https://ap1.veloopti.com.au/storm-rules Storm rules] web page which is found under the Events main menu item. These pages are protected using permissions meaning not everyone is able to see them.
| |
| − | | |
| − | When a change is made that does not abide by the logic of the storm rules engine then the text is marked in red indicating that it needs to be changed.
| |
| − | | |
| − | === Overview ===
| |
| − | The overview tab contains the properties of the storm rule that are common to every severity setting.
| |
| − | :'''Name''': The name of the storm rule. This is used in the short description of the event that will appear in the event view and also in the email subject line.
| |
| − | | |
| − | :'''Description''': This is used to describe the storm rule to other people who are editing it.
| |
| − | | |
| − | :'''Enabled''': This enables or disables the storm rule. A disabled storm rule will not detect or notify of any threshold breaches. It also will not collect any metrics for displaying in a dashboard.
| |
| − | | |
| − | :'''Event path''': This is the path that is used when counting the events each minute.
| |
| − | | |
| − | :'''Help Text''': This text appears in the event that is created and also in the body of the email.
| |
| − | | |
| − | === Information settings ===
| |
| − | The parameters in the information setting define the number of events or duplicates and the time period over which they should be received in order to create or reduce a higher level severity storm event with the status of information. It also contains the parameters for closing an open event.
| |
| − | | |
| − | '''Breach Condition
| |
| − | :'''Breach Count''': This is the minimum number of events or duplicates per '''''breach duration''''' that must be exceeded in order to create a storm event with the status of information. As long as a storm event is open and this condition is met, the storm rule will continue to be in breach with an information severity level. When this condition is not being met then the reset conditions are checked to see whether the open event can be closed.
| |
| − | | |
| − | :'''Breach Duration''': This is the time period over which the minimum number of '''''breach count''''' events or duplicates in order to trigger a storm event with the status of information.
| |
| − | '''Reset Condition
| |
| − | :'''Reset Count''': If there is no breach condition being met then this is the number that the events or duplicates must be equal to or less than during the '''''reset duration''''' before the event is closed. This will close an event. The event that is being closed can have a severity of either information, warning or exception.
| |
| − | | |
| − | :'''Reset Duration''': This is the time period over which the minimum number of '''''reset count''''' events must appear in order to close the event.
| |
| − | '''Notifications
| |
| − | :'''Notify Users''': This is a list of users who will be notified when a storm event breaches the '''''breach count''''' for the first time. If the storm event is first created with a warning or critical severity level then the users in those settings will be notified.
| |
| − | | |
| − | :'''Notify User Groups''': This is a list of user groups that will be notified when a storm event breaches the '''''breach count''''' for the first time. If the storm event is first created with a warning or critical severity event the user groups in those settings will be notified.
| |
| − | | |
| − | === Warning settings ===
| |
| − | The parameters in the warning settings define the number of events or duplicates and the time period over which they should be received in order to create a storm event with the severity of warning. It also contains the details to reduce or increase an existing open storm event to a higher or lower level severity level. If the breach conditions are no longer being met for a storm event with the severity of exception then the warning breach conditions are next checked to see whether the event can be reduced to warning. Additionally if a storm event with the severity of normal now meets the warning breach conditions the then its severity level is increased to warning.
| |
| − | | |
| − | '''Breach Condition
| |
| − | :'''Breach Count''': This is the minimum number of events or duplicates per '''''breach duration''''' that must be exceeded in order to create or change a storm event with the status of warning. While the storm event with a severity of warning is open, as long as this condition is met the storm rule will continue to be in breach. When this condition is not being met then the reset values are checked.
| |
| − | | |
| − | :'''Breach Duration''': This is the time period over which the minimum number of '''''breach count''''' events or duplicates in order to trigger a storm event with the status of warning.
| |
| − | | |
| − | :'''Reset Count''': If there is no breach condition being met then this is the number that the events or duplicates must be equal to or less than during the '''''reset duration''''' before the event can be reduced to warning.
| |
| − | '''Reset Condition
| |
| − | :'''Reset Duration''': This is the time period over which the minimum number of '''''reset count''''' events must appear in order to reduce the event to warning.
| |
| − | '''Notifications
| |
| − | :'''Notify Users''': This is a list of users who will be notified when a storm event breaches the '''''breach count''''' for the first time. If the storm event is first created with a severity level of exception then these users will not be notified when the severity is reduced to warning.
| |
| − | | |
| − | :'''Notify User Groups''': This is a list of user groups that will be notified when a storm event breaches the '''''breach count''''' for the first time. If the storm event is first created with a severity level of exception then these user groups will not be notified when the severity is reduced to warning.
| |
| − | | |
| − | === Exception settings ===
| |
| − | The parameters in the exception settings define the number of events or duplicates and the time period over which they should be received in order to create a storm event with the severity of exception. It also contains the details to increase an existing open storm event to a severity level of exception. If the breach conditions are no longer being met for a storm event with the severity of exception then the warning first followed by normal breach conditions are then checked to see the event should be changed to them.
| |
| − | | |
| − | '''Breach Condition
| |
| − | :'''Breach Count''': This is the minimum number of events or duplicates per '''''breach duration''''' that must be exceeded in order to create or change a storm event with the status of exception. While the storm event with a severity of exception is open, as long as this condition is met the storm rule will continue to be in breach. When this condition is not being met then the reset values are checked.
| |
| | | | |
| − | :'''Breach Duration''': This is the time period over which the minimum number of '''''breach count''''' events or duplicates in order to trigger a storm event with the status of exception.
| + | Storm events have increasing levels of severity that come into effect by having an increasing breach count relative to the time window (breach duration) that they are received in. |
| − | '''Reset Condition
| |
| − | :'''Reset Count''': If there is no breach condition being met then this is the number that the events or duplicates must be equal to or less than during the '''''reset duration''''' before the event may be reduced to warning or information.
| |
| | | | |
| − | :'''Reset Duration''': This is the time period over which the minimum number of '''''reset count''''' events must appear in order to reduce the event to warning or information.
| + | == Starting and ending the event storm == |
| − | '''Notifications
| + | === How event storms are raised === |
| − | :'''Notify Users''': This is a list of users who will be notified when a storm event breaches the '''''breach count''''' for the first time.
| + | For the events that match the path of the event storm rule, each minute the following is performed. It does not matter whether they are new events or [[Event#Previous events and duplicates|duplicate]] events. |
| | + | # If the number of events that that are received over the exception breach duration are added together and exceed the exception breach count then an event with the severity level of exception is raised. |
| | + | # If the number of events that that are received over the warning breach duration are added together and exceed the warning breach count then an event with the severity level of warning is raised. |
| | + | # If the number of events that that are received over the information breach duration are added together and exceed the information breach count then an event with the severity level of information is raised. |
| | | | |
| − | :'''Notify User Groups''': This is a list of user groups that will be notified when a storm event breaches the '''''breach count''''' for the first time.
| + | === Changing severity while the event storm is still active === |
| | + | Once a storm rule is breached the event storm will continue to be monitored in the same manner as above to see whether there is an increase in severity. If there is an increase in severity of the event the event is increased in severity and the relivent notifications are sent out. Storm events do not decrease in severity. |
| | | | |
| − | === Explain === | + | === How event storms are ended === |
| − | The explain tab will attempt to describe the storm rule in plain english. | + | When the current event count for the path no longer exceed any of the informational, warning or exception breach thresholds over the breach durations the reset conditions can be evaluated. The reset breach count and durations are then checked in the same manner as the initial breach condition. If none of the reset conditions are met then the storm event can be closed. If one of the reset conditions are still being met then the storm event remains open with the pre-existing severity. |
| | | | |
| − | When a change is made that does not abide by the logic of the storm rules engine then the text is marked in red indicating that it needs to be changed.
| + | == Notifications == |
| | + | Notifications will occur once with any new storm event or once with an increase of severity level. Therefore a storm event that is opened with a severity of information will initially send a notification when it is first opened. It will also send a notification if the severity is increased to warning and again if the severity is increased to exception. Whereas a storm event that is opened with a severity level of warning would not send a notification if the event decreased severity to information. However it would send a notification if the event severity increased to exception. A storm event that is opened as a exception would not notify if it reduces in severity to either warning or normal. |
| | | | |
| − | === History ===
| |
| | | | |
| − | The history tab contains a graph with the minute total count for the storm rule path for the last hour. The rule will be populated with the last hour of data if the storm rule has existed this long.
| + | [[Category:Events]] |
Storm events have increasing levels of severity that come into effect by having an increasing breach count relative to the time window (breach duration) that they are received in.
For the events that match the path of the event storm rule, each minute the following is performed. It does not matter whether they are new events or duplicate events.
Once a storm rule is breached the event storm will continue to be monitored in the same manner as above to see whether there is an increase in severity. If there is an increase in severity of the event the event is increased in severity and the relivent notifications are sent out. Storm events do not decrease in severity.
When the current event count for the path no longer exceed any of the informational, warning or exception breach thresholds over the breach durations the reset conditions can be evaluated. The reset breach count and durations are then checked in the same manner as the initial breach condition. If none of the reset conditions are met then the storm event can be closed. If one of the reset conditions are still being met then the storm event remains open with the pre-existing severity.
Notifications will occur once with any new storm event or once with an increase of severity level. Therefore a storm event that is opened with a severity of information will initially send a notification when it is first opened. It will also send a notification if the severity is increased to warning and again if the severity is increased to exception. Whereas a storm event that is opened with a severity level of warning would not send a notification if the event decreased severity to information. However it would send a notification if the event severity increased to exception. A storm event that is opened as a exception would not notify if it reduces in severity to either warning or normal.