Six Crucial Steps in Scaling Secure Universal Data Authorization

 

Modern data platforms are becoming increasingly complicated in order to fulfil the changing demands of data consumers. Data analysts and data scientists require quicker data access, but IT, security, and governance are stuck, unable to figure out how to provide data access in a simple, safe, and consistent manner across a wide range of analytic tools.

According to Gartner, just 20% of enterprises engaging in information governance would succeed in expanding their digital operations through 2022. As a result, enterprises are developing data access frameworks to address the data delivery difficulty while maintaining scalability and ensuring universal data authorizations across all partners.

Why are modern data platforms so complicated?

Data is being used by organisations of all sizes to better understand their consumers, gain a competitive edge, and enhance operational efficiency. To achieve these requirements, a corporate data platform capable of handling the complexities of data management and use is required.

One of the most difficult challenges for data platform teams today is ensuring that data is universally accessible from disparate storage systems (data lakes, data warehouses, relational databases, etc.) while meeting increasingly complex data governance and compliance requirements due to emerging privacy legislation such as GDPR, CCPA, and others.

This complexity is exacerbated by a schism between data stakeholder groups: technical data platform and data architecture teams; centralised data security and compliance; data scientists and analysts working in lines of business to generate insights; and data owners and stewards in charge of developing new data products.

The difficulty of handling customer data and personally identifiable information (PII) will dramatically hinder productivity and restrict the quantity of accessible data that can be used unless adequate data access and an authorization framework are in place to aid automate procedures.

How to Implement Cloud Data Security and Regulatory Compliance

Organizations become stuck on their data supply journey when data stakeholders are not in agreement. This is due to the fact that data consumers must be able to discover the proper dataset, understand its context, trust its quality, and access it in the tool of their choice – all while trusting the data security and governance teams to apply the appropriate data permission and governance standards.

Accelerating time-to-insight on data platforms necessitates a robust structure that not only fits the demands of all stakeholders, but also allows for scaling as systems grow.

 

When developing a solution to ensure responsible data use, it is critical to create a universal data authorisation framework that incorporates the following six fundamental capabilities:

1. Make use of attribute-based access control (ABAC)

Most businesses begin by developing access control policies based on role-based access control (RBAC). This strategy is fine for simple use cases, but because roles are created manually and are essentially static, any new use case necessitates the development of a new role with new rights assigned to that person.

As the data platform's scope and complexity increase, the outcome is an unpleasant policy environment known as "role explosion." Furthermore, each system has its own rules for creating and managing role permissions, and RBAC is frequently confined to coarse-grained access (e.g. to an entire table or file).

ABAC, on the other hand, enables businesses to build dynamic data authorisation policies by utilising attributes from different systems to make context-aware decisions on every individual request for access.

ABAC, a superset of RBAC, can accommodate the complexity of granular policy needs while also expanding data access to additional persons and use cases through three major kinds of characteristics that may be utilised to build rules (user, resource, and/or environmental).

2. Enforce Access Policies Dynamically

Most contemporary policy enforcement systems still require numerous copies of each dataset, and the expense of developing and maintaining them may soon build up. Simply using ABAC to construct policies does not eliminate the discomfort, especially when the characteristics are assessed against the access policy at the decision point. This is due to the fact that they still point to a static copy.

Once the difficult task of defining attributes and policies is completed, they should be passed down to the enforcement engine, which should dynamically filter and transform the data by redacting a column or applying data transformations such as anonymization, tokenization, masking, or even advanced techniques such as differential privacy.

Dynamic enforcement is critical for enhancing the granularity of access controls without raising the overall complexity of the data system. It is also critical to ensuring that the company is highly responsive to changing governance needs.

3. Establish a Unified Metadata Layer

If ABAC is the engine that drives scalable, secure data access, then metadata is the gasoline that powers the engine. It is essential to create attribute-based access control policies and offers visibility into the what and where of the organization's datasets. With a richer layer of information, companies may design more specific and appropriate access controls.

When designing the metadata lifecycle, four major factors must be considered:

·         Access: How can we allow smooth API access to use information for policy decisions?

·         Unification: How can a unified metadata layer be created?

·         Metadata Drift: How do we keep the metadata current?

·         How do we uncover new technical and business metadata?

The difficulty is that metadata, like data, is frequently found in many locations throughout the company and is controlled by different teams. Each analytical engine necessitates its own technical metastore, whereas governance teams save the business context and classifications in a business catalogue such as Collibra or Alation.

As a result, businesses must federate and integrate their information in order for the entire set to be available in real time for governance and access control regulations. Because it would be unfair, if not impossible, to expect metadata to be defined in a single location, this unification is done inherently through an abstract layer.

Continuously integrating metadata results in a single source of truth for data. This helps to avoid "metadata drift" or "schema drift" (aka inconsistency in data management) over time and allows effective data governance and business operations throughout the enterprise, such as data categorization or tagging. It also provides a single data taxonomy, which facilitates data discovery and access for data users.

Metadata management tools that use artificial intelligence to automate parts of the metadata lifecycle can also be beneficial because they can perform tasks such as identifying sensitive data types and applying the appropriate data classification, automating data discovery and schema inference, and automatically detecting metadata drift.

4. Make Distributed Stewardship possible.

Scaling safe data access requires more than just expanding the types of regulations and enforcement techniques. Because the types of data accessible and the business needs required to exploit it are so broad and complicated, the policy decision-making process must also be scalable.

In the same way that an improperly designed enforcement engine can be a bottleneck, a lack of an access model and user experience that allows non-technical users to administer these rules can impede an organization's ability to expand access control.

Effective data access management should attempt to embrace, rather than block, the unique demands of all constituencies. Unfortunately, many access control technologies need sophisticated change management and the creation of customised procedures and workflows in order to be effective. Early on, enterprises must consider how this access model will fit within their company.

The access system should address two essential aspects in order to enable dispersed stewardship. Delegate the maintenance of data and access policies to individuals in the lines of business (data stewards and administrators) who understand the data or governance needs, and then guarantee that change can be propagated uniformly throughout the company.

5. Make Centralized Auditing Simple

Knowing where sensitive data is stored, who is accessing it, and who has authorization to access it is crucial for making informed access decisions.

Because there is no uniform standard across the diversity of technologies in the current workplace context, editing is a continual problem for governance teams. Collecting audit logs from numerous systems so that governance teams can answer simple queries is time-consuming and unscalable.

Despite defining regulations at the highest level, the governance team has no easy method of knowing whether their policies are being implemented at the moment of data access and whether the organization's data is being secured.

Centralized auditing with a uniform schema is crucial for providing reports on data usage and can allow automatic data breach notifications via a single interface with the business SIEM. Because many log management systems are primarily focused on application logs, organisations are searching for solutions that audit log schema because they enable governance teams to address audit queries.

Another thing to think about is investing in a basic visibility mechanism early in the data platform journey to enable data stewards and governance teams analyse data consumption and demonstrate the platform's value. Once the company understands what data it has and how employees are utilising it, teams can create more effective access controls.

Finally, look for a flexible, API-driven design to guarantee that the access control framework can adapt to the demands of the data platform in the future.

6. Integrations that are Future-Proof

Integrating with an organization's larger environment is critical to any successful access control strategy, as the data platform is likely to change over time as data sources and technologies improve. Similarly, the access control architecture must be versatile and accommodate flexible data fabric connections.

One advantage of adopting ABAC for access control is that characteristics may be acquired from existing systems inside the business, as long as they can be accessed in a performant manner in order to make dynamic policy judgments.

Creating a flexible foundation also relieves the organisation of the burden of figuring out the complete architecture from the start. Instead, they may begin with a few essential technologies and use cases and expand as they learn more about how the company utilises data.

After all, policy insight is a continuum, and fascinating insights may be found at the intersection of critical questions such, "What sensitive data do we have?" Who is gaining access and why? Who should be allowed access?

Because they may adapt connectors to match their own needs, some firms prefer to focus on open source. However, it is important to note that developing and maintaining these connections may soon become a full-time job.

The data platform team should be lean and have little operational overhead in the ideal case. Investing time in developing and maintaining integrations is unlikely to give difference to the company, especially as the ecosystem already contains multiple high-quality integration solutions.

Success with Universal Data Access

When attempting to protect data access, it is critical to take a step back and use a design-to-value strategy, as with any major endeavour. This entails identifying the highest-value data domains that require sensitive data access and activating or unblocking them first, as well as attempting to gain visibility into how data is already being utilised in order to prioritise action.

Organizations are investing heavily in their data platforms in order to unleash new innovation; yet, data initiatives will continue to be stymied at the last mile in the absence of an underlying framework.

Scaling secure, universal data authorization can be a tremendous enabler of agility within an organisation, but by leveraging the six principles outlined above, organisations can ensure that they are staying ahead of the curve and designing the right underlying framework that will ensure the success of all stakeholders.