We work with customers of all different shapes and sizes, with all kinds of demands when it comes to their data, but a common area of discussion is data retention.
Data retention is an area that many organisations struggle with, and it’s an interesting area as it crosses a couple of boundaries. For infrastructure leaders, it causes a challenge in terms of cost and overall management; for governance and privacy leaders, poor retention can pose a significant risk to the business.
While both stakeholders need to deliver on retention policies, the driver for doing so differs, and sometimes we see a divide between governance/privacy and infrastructure teams regarding retention.
Privacy and governance teams create policies that are fit for purpose from a contractual or legal perspective, but Infrastructure teams often struggle to enforce these policies, as they don’t have the requisite level of visibility into the data to be able to, or the tools that they do have can’t deal with the complex nature of retention policies.
Regardless of which side of the fence you sit on, the reality is that good data retention would solve several ills from both a storage/infrastructure standpoint and a governance and privacy perspective.
In this blog, I want to take some time to consider those two stakeholder views:
Storage / Infrastructure
Data retention and management have long been an issue from an infrastructure perspective. Most companies have a data retention policy in place, but enforcing that policy proves difficult, for one (or both) of the following reasons:
- they don’t have the tools to give the requisite visibility over the data to understand it (and if you don’t understand it, you can’t control it)
- the tools that they do have can’t handle the complexity of the policies that the business is trying to enforce
- the policies in place meet regulatory/legal needs but not the needs of the users
From a storage/infrastructure perspective, this can cause several issues:
- the volume of data stored grows exponentially and is often significantly more than needed (we usually see anywhere near 5x)
- storage infrastructure bloats due to the increased data volumes, which means more infrastructure to manage, with more cost and complexity
- increased overhead in terms of data and infrastructure management
So there’s a big cost-efficiency issue here, as well as a general management issue – the bigger and more complex the infrastructure is, the more time and resource is needed to manage it.
At the same time, businesses are looking to store more data as it could be of value, so the problem is compounded.
Governance / Privacy
Governance and privacy teams are on the hook to deliver policy to support the business, whether from a legislative or regulatory perspective regarding critical data assets.
Regulatory and legislative controls are complex and wide-reaching. They have no care or context over the demands of businesses and how their users want to use their data. Nor do they understand the technical complexities of administering these policies.
The challenge this causes is that the governance and privacy teams create the right data retention policy to suit, but operationalising that complex policy can be almost impossible without the right toolsets or technical capabilities.
The other challenge is that without understanding the real-world use of your data, how can you hope to create a policy that meets governance needs but also the needs of the users?
So the big question is, how do we get it right for both sets of stakeholders?
There needs to be a synergy between technology and policy to do this successfully. Ultimately, we need the right toolsets to understand our data so that policy can fit both the business purpose and the regulatory need. Ideally, that same capability should allow us to automate the enforcement of that retention policy and help us move data beyond use.
If we can accomplish that, we solve several challenges:
- we reduce the risk surface when it comes to data we store and process
- we reduce the volumes of data that we’re dealing with
- we enable infrastructure optimisation in terms of both cost and complexity
- we make it possible to meet our governance needs
Sounds simple right? I guess if it were, everyone would be doing it! That said, there are a number of tools out there that can simplify the problem – the challenge is in ensuring that people, processes and technology are seamlessly integrated to fully utilise on these investments.
At Nephos, we specialise in simplifying complex data retention challenges. Our expertise ensures that people, processes, and technology are seamlessly integrated to fully leverage your investments. Explore how we can help you streamline your data retention strategy here: