An NSW metropolitan council leveraged ML to classify data at scale, reduce a backlog of uncategorized content

Organizations have had to contend with the impact of exponential data growth for many years, but the issue becomes more pronounced with a move to remote or hybrid work. As organizations adopt more and more data sources to enable remote or hybrid working styles, it becomes more difficult to appropriately manage the data to meet regulatory obligations and ensure it is safeguarded for privacy, security, and governance.  

For one of Australia’s largest metropolitan councils, solving this challenge meant embracing RecordPoint’s advanced AI and machine learning solution.

The exponential growth in data and new data sources raises many challenges for public sector organizations    

In a recent survey, the average organization adopted two or more new data sources in the last two years, with some introducing more than five new sources of critical data. While 72% of respondents in that survey said they increased the number of data sources, only 36% of organizations were doing a good job of managing the situation.

Data sprawl raises challenges for records and information managers as the tools often lack the functionality to ensure robust record controls and policies. For unstructured data sources like file shares or collaboration tools like Slack or Microsoft Teams, this puts the burden of categorization or classification on either the user or the records and information management professional.  

For more structured environments like Microsoft SharePoint, Microsoft Exchange, Customer Relationship Management systems (CRM) or financial systems, while more consistent and standardized in form, the tools selected often lack the capabilities to allow compliant and defensible disposition.  

With a high volume of data, organizations need a more automated and scalable approach to deeply understand the data they have and then manage it appropriately.

Solving these challenges is vital. Regulators do not distinguish between tools; they are focused on whether the data is being managed appropriately and are not afraid to levy significant fines on organizations who fail to do this.  

On top of this, cyberattacks are becoming more frequent, and damaging for organizations, and there is a growing expectation from citizens and customers that their data and privacy should be protected. According to PwC, 85% of consumers said providers should disclose cyber breaches so that they can choose to use another supplier in the future.

Organizations are leaning on AI and ML to solve these problems

For one RecordPoint customer, one of New South Wales’ largest metropolitan councils, new communication systems significantly increased the volume of records and data being captured and made it more difficult to guarantee compliance.

As an existing RecordPoint customer, they were already using RecordPoint’s Connector framework to capture information from data sources like SharePoint and Microsoft Exchange without requiring effort from end-users. However, the ever-increasing volume of data was becoming a significant challenge and they had a growing backlog of uncategorized content.  

They needed a scalable way to appropriately classify records to quantify risk and ensure that the data collected was managed in line with privacy and freedom of information regulations such as the Government Information (Public Access) Act 2009 (NSW) (GIPA Act). To help solve this problem, the team turned to RecordPoint’s automated classification feature, Classification Intelligence.

What is Classification Intelligence?

Classification Intelligence (CI) enables consistent, accurate categorization for data based on an organization’s policies at scale. CI is built to help customers classify and govern information at scale, considering the content and context of the information relating to the record.  

With CI, organizations can consistently, automatically categorize their data based on their organization’s policies faster and better than any human can, ensuring greater accuracy and less dark data, and allowing teams to focus on data discovery and ensuring compliance with regulations.  

CI allows you to leverage the knowledge and experience of your information management team when training models, allowing for greater accuracy.  Rather than relying on manual effort to classify content and manage compliance, CI allows records managers and end-users to work more productively and collaboratively.  

Businesses can use the classification of records from more structured environments like SharePoint to help train the model, and “teach” it what to look for when analyzing the same types of records from less structured repositories. Misclassifications go back into the model, so it's continually improving and can accommodate data drift. This may sound like a technical process and it is, but we’ve put work into making it easy to use, even for those without a data science background.

CI streamlines data categorization and reduces risk  

For this NSW council, within weeks of adopting Classification Intelligence they were automatically classifying more than 90% of records ingested into the RecordPoint platform to a high level of accuracy.  

This has resulted in the information management team significantly streamline data categorization. CI has allowed the team to better keep up with the volume of data being generated to ensure classification and the application of retention schedules, allowing the organization to reduce its backlog of uncategorized content and reduce risk.