Cloud-Native Apps Driving Major IT Data Crisis
A global survey of 1,303 CIOs and senior IT practitioners finds 71% of respondents believe the explosion of data produced by cloud-native technology stacks is beyond humans’ ability to manage.
Conducted by the market research firm Coleman Parkes on behalf of Dynatrace, the survey also finds more than three-quarters of respondents report that their IT environment changes once every minute or even more frequently.
More challenging still, respondents note that, on average, they only have observability across 9% of their environment. On average, the survey also finds organizations capture only 10% of observability data for querying and analytics.
A full 59% say that without a more automated approach to IT operations, their teams could soon become overloaded by the increasing complexity. Nearly half, however, say it’s too costly to manage the large volume of observability and security data using their existing tools. A total of 43% noted current approaches to observability data ingestion and storage won’t support their future needs.
Nearly two-thirds (63%) also say the costs and delays caused by reindexing and rehydration make it challenging to unlock value from the increasing amount of data they capture.
Mike Maciag, chief marketing officer for Dynatrace, says its apparent IT organizations are struggling to manage the massive amount of data that is now being collected from modern IT environments. As IT environments become more complex, IT teams are discovering they don’t have the data management tools and expertise needed to achieve observability, he adds. The need to streamline management of the data being collected is a critical first step toward achieving that goal, notes Maciag.
Overall, the survey finds IT teams use an average of 10 monitoring tools across their technology stacks. Nearly two-thirds also say it has become harder to attract and retain enough skilled IT ops and DevOps professionals to manage and maintain their cloud-native stack. A full 93% say artificial intelligence for IT operations (AIOps) will be increasingly vital to alleviating the shortage of skilled IT, development and security professionals they would otherwise require.
It’s not clear to what degree the transition to cloud-native applications will force a shift away from static monitoring tools toward observability platforms that promise to make it simpler to discover the root cause of an IT issue. The one thing that is certain is observability platforms will need to be able to automatically surface those issues using various types of machine and deep learning algorithms. Most IT staff are not going to be able to formulate the queries required to surface those issues on their own without relying on some form of AI for additional guidance.
In the meantime, with each additional cloud-native application deployed in a production environment, the observability issue will only become more acute. In fact, an inability to manage those applications at scale will inevitably slow down the rate at which cloud-native applications are deployed in production environments. The challenge is most organizations will not replace existing monitoring tools until they reach a critical mass of cloud-native applications. By then, of course, the level of stress on the IT staff will have already reached a breaking point.
