Splunk Infrastructure Monitoring Review

[Splunk Infrastructure Monitoring (formerly SignalFx)] is being used across the entire organization for many purposes such as continuous monitoring of cloud resources and application statuses. It addresses business problems that require us to get timely notifications for events occurring within our infrastructure and address them accordingly. In my own department, our continuous monitoring tools rely on SFX reports.

My department uses this particular feature very heavily as we have a very variable workload and hence, need to continuously monitor usage so that we can respond quickly in the event that available resources are running out, and SFX is configured to alert us instantly whenever we approach certain benchmarks. It has been very helpful on numerous occasions.

Unfortunately, I have not been involved in onboarding teams to Splunk infrastructure so far, but colleagues in other teams, I have seen struggle a bit initially, and hence my recommendation for a bit of a better documentation.

While not on a daily basis, we have greatly benefited from historical anomalies and related data. Eg. When we have sometimes faced unexpected problems in our systems, we often resorted to SFX data to see if there is a pattern. I know of a case that was resolved because we were able to see the timing coincided with a similar failure a year ago during a holiday season.