Shift to microservices and continuous software delivery puts pressure on DevOps observability

A Scalyr report, based on a survey of 155 software development practitioners, finds that organizations are shifting away from traditional, monolithic architectures, with three-quarters of survey respondents delivering at least some of their applications and more than one-third delivering most of their applications as microservices.

Practitioners are also delivering software more rapidly than ever, with 71 percent of engineers pushing code into production at least weekly and nearly one-third doing so at least once per day.

This transition has put pressure on DevOps observability. Despite having multiple visibility tools, respondents who deliver software in this modern way – delivering their applications as mostly microservices and pushing code more than once per day – say that most of their companies’ engineering time is spent troubleshooting and debugging software issues.

When it comes to log management, the capability respondents care most about is ad-hoc query speed. However, more than half of respondents who deliver software in this modern way spend the majority of their total investigation time waiting for queries to complete.

“Engineering teams have really upped their game, delivering software more quickly and efficiently than ever before,” said Steve Newman, CEO of Scalyr. “However, their modern approach puts increased pressure on monitoring and troubleshooting. Companies that are undergoing a transition to microservices and a rapid or continuous software delivery pipeline need to make sure their observability tools and processes can keep up.”

Companies are delivering software in a modern way:

• Three-quarters of respondents deliver some and more than one-third deliver most of their applications as microservices.
• 71 percent of engineers push code at least once per week, and nearly one-third push code at least once per day.

Companies rely on many tools for observability:

• Nearly half of respondents have five or more observability tools.
• 58 percent of respondents in a DevOps role have five or more such tools.

Engineering teams spend a lot of time troubleshooting:

• 40 percent of respondents say their companies’ engineers spend most of their time troubleshooting software issues.
• This percentage increases to 62 percent for respondents who deliver mostly microservices and 73 percent for those who push code at least once per day.

Ad-hoc query speed is the top log management requirement:

• 54 percent of respondents care most about ad-hoc query speed in log management.
• This number jumps to 61 percent for respondents who push code at least once per day and 68 percent for those who deliver mostly microservices.
• Other top requirements include ability to parse and operate intelligently on complex logs and alert speed.

Respondents who deliver software in a modern way spend most of their investigation time waiting:

• One-fourth of respondents spend most of their log management investigation time waiting for queries to complete.
• This number jumps to 53 percent for respondents who deliver mostly microservices and 81 percent for those who push code at least once per day.