The NHI management challenge: When employees leave

An employee is exiting your organization. Regardless of the terms of departure, an ex-staffer has the potential when they leave or change roles to impact a wide range of non-human identities, digital credentials, and other secrets. Those secrets include the credentials for accessing corporate networks, sending and receiving email, and sharing files. For each non-human identity in an enterprise, an average of 92 non-human identities (NHIs) are created. When employees exit, NHIs can become unmanaged, and in many cases, exposed to exploitation.

What is an NHI?

Non-human identities (NHIs) support machine-to-machine authentication and access across software infrastructure and applications. These digital constructs enable automated processes, services, and applications to authenticate and perform tasks securely, without direct human intervention. Access is granted to NHIs through various types of authentications, including secrets such as access keys, certificates and tokens.

For example, an application or workload might need an API key to access cloud services. Other examples include auth tokens, SSH keys, private certificates, MTM and database passwords, and container secrets. Unlike human access credentials, NHI permissions often exceed the required minimum, to facilitate autonomous and continuous operation. Such “wrong-sized” permissions make NHIs more dangerous and subject to exploitation.

Ideally, NHIs are created, classified and managed using an NHI management framework, with clear policies to govern their lifecycle. In practice, NHIs are often created ad hoc utilizing non-standard workflows esoterically created individual employees, with little or zero documentation. When one such employee exits, knowledge of those NHIs and associated workflows becomes orphaned and potentially lost forever.

Types and scope of NHI-related risk

The NHI lifecycle

Despite being used for autonomous, machine-to-machine authentication and operation, the NHI lifecycle typically involves humans and their associated risk at every step:

• Requirement – the need for secret results from architectural and integration choices made by software developers and system architects. Those choices should be thoughtful and considered
• Creation and storage – NHIs are often created and stored “on the fly” in the course of development and integration. As many as 62% of all secrets are duplicated and stored in multiple locations
• Usage – while NHIs are passed across systems by software, humans invoke their usage and update the code and configurations where they reside. However, as many as 40% of identified secrets sit idle
• Rotation – best practices dictate that secrets be updated (a.k.a. rotated) on a regular basis. When left in human hands, NHI rotation occurs on average every 627 days
• Retirement – when NHIs and other secrets are no longer needed, due to changes in software architecture and configurations and/or staff exit, they should be invalidated. However, up to 91% of employee tokens are never revoked or de-provisioned

Leakage and abandonment

The end of an employee’s tenure presents two types of risk: leakage and abandonment.

Leakage

When an employee exits, secrets can go with them. Those secrets – credentials, NHIs and associated workflows – can be exfiltrated from mental memory, recorded manually, stored in vaults and keychains, on removable media, and more. Secrets that have been exfiltrated are considered “leaked.”

Abandonment

An equally great risk is that employees, especially developers, create, deploy and manage secrets as part of software stacks and configurations, as one-time events or in regular workflows. When they exit, those secrets can become orphans, whose very existence is unknown to colleagues or to tools and frameworks. Secrets that have been “abandoned” still have access to data and are still functional. They are simply not used in any known workflows.

Secrets mismanagement

Secrets silos

While major DevOps platforms and cloud providers offer native toolsets for secrets management, these systems often don’t play well together. Diverse secrets management paradigms can create silos that hinder interoperability and consistent policy implementation. When such silos are the purview of a single employee, his or her departure turns that silo completely opaque.

Third-party access to secrets

The lifecycle of NHIs can stretch beyond the boundaries of a single organization, encompassing partners, suppliers, customers and other third parties. If the shared secrets originate in your organization, you have the opportunity to manage that lifecycle, even if the other organizations are less rigorous in their secrets management practices. However, you will have less visibility into the employment status of third-party staffers and their disposition of shared secrets than you will for your own employees.

Best practices

Best practices for managing secrets, including NHIs, focus on a few key concepts, herein the “four Cs”:

Context

A functional secrets management strategy must account for and apply across all contexts in an organization:

• Identify and analyze all existing secrets to provide full context
• Catalog the context and particulars of all new and rotated secrets
• Include development and deployment on premises, in the cloud and beyond
• Ditto for applications, infrastructure and other cybersecurity contexts
• Incorporate definition of contexts in employee on-boarding; review contexts at exit, even for non-technical employees

Comprehensiveness

No secret can remain uncatalogued and unmanaged, within the company and with third parties. Start with the following:

• Establish secrets management policy and processes; document in a playbook
• Coordinate with HR to ensure that all employees and contractors understand and accept their obligations for secrets management, during their tenure and afterwards
• Enforce role-based access control (RBAC); grant access to secrets based on user roles, ensuring only authorized personnel can access sensitive information
• Revisit employee roles and secrets access as part of off-boarding

Consistency

Secrets management policy must be applied consistently across types of secrets, different platforms, organization groups, and if possible, with third parties. Consistency is best achieved by:

• Always use secrets management tools and avoid ad hoc creation and usage
• Take advantage of platform-native tools and methods
• “Right-size” permissions granted to NHIs to match actual use cases (vs. worst cases)
• Require strong secrets (not an issue if tools are used); avoid easily guessable, exploitable common patterns
• Integrate NHI management into DevOps; integrating secrets management practices into the DevOps lifecycle helps maintain security across the software development and deployment process
• Inject NHI policy compliance into review cycles and into employee offboarding

Currency

• All secrets must have an expiration date, by which a secret must be rotated or retired.
• Rotate secrets on a regular basis
• Revisit secrets management policy and processes to include new contexts and tools
• Use lessons learned from employee and manager feedback during tenure and at time of exit to refine and update secrets management policy and processes

The NHI handoff process – When employees exit

If your company already has a comprehensive secrets management program in place, employee transitions should be transparent. Credentials get decommissioned and secrets under that employee’s purview are handed off and/or rotated.

But most organizations do not have a comprehensive secrets management program in place, or only have a partial grip on secrets inventory. So what steps are necessary?

Prepare for the inevitable (preemptive measures)

Take the opportunity of any employee changing status to bolster secrets management. If your organization has no secrets management tools or protocols in place, proactively implement them before an incident occurs.

At time of exit

Prior to exit or change of status, require the employee to catalog all credentials, other secrets and workflows they can recall. Have HR, IT, and security teams with knowledge of that employee’s activities and responsibilities check and augment that list. Cross-check the list against existing secrets management tools and enumerations and add missing entries to databases and tools. Flag volatile secrets for rotation or elimination.

Post exit checks

Repeat the checks above and ensure that all credentials and secrets have been decommissioned and/rotated and that workflows reassigned and/or retired.

Regular audits

Take the lessons learned from each change of status to augment and refine your secrets management policies and processes. Independently of actual employee departures or other changes, perform regular audits of your secrets inventory and configuration checks of the tools used to manage those secrets. Make secrets management compulsory across the entire organization.

Conclusion

Secrets in software systems are an integral part of the development and deployment landscape. Advances in software architecture, authentication, and biometrics will never obviate the need for secrets but instead multiply the number of secrets and contexts of secret management.

A change of employee status can have an emotional impact, but a team member changing roles or exiting the organization shouldn’t affect company cybersecurity posture, except in a positive fashion, from lessons learned.

Don’t wait until the employee or contractor has left the scene to take action. Forensics are expensive and seldom capture the full scope of ex-employee purview over secrets and workflows. And don’t wait until mismanaged secrets enable or worsen a breach. Instead, start managing your software secrets, including NHIs, today.