What does the NYC records commissioner think about AI?

The digital era has changed the way organizations, big or small, manage their records and information. The shift from tangible documents to virtual data has opened up numerous opportunities while introducing new complexities.

In a recent episode of the FILED Podcast, Pauline Toole, the commissioner of the New York City Department of Records and Information Services (DORIS), shared her insights on how the city is managing these challenges and the integration of artificial intelligence (AI) into the process.

Pauline was appointed Records Commissioner by the former mayor of New York after spending 20 years working within city and state governments. Her interest in historical records in the municipal archives ignited her curiosity about records management. Pauline realized that the city hadn't adequately preserved its historical documents. 

"There was a lot of catch-up to do clean that up and make content more available to the general public online," she says in the podcast.

Overcoming challenges in information governance

One of the critical challenges Pauline and her team at DORIS faced was the lack of centralized rules for records in New York City. As technology evolved, digital records emerged alongside hard copies, creating a complex landscape for managing records. 

The absence of standardized rules and decentralized technology structures across city agencies posed a significant hurdle. Pauline's team worked tirelessly to establish uniform rules and adapt strategies to diverse technology environments.

"We've been striving to establish them... The application of what are standard rules for managing born-digital records have never been put in place or wasn't put in place until fairly recently." 

Pauline emphasized the importance of building alliances, especially with financial divisions. In the context of government agencies, where enforcement authority might be limited, these alliances are crucial in garnering support and resources for records management initiatives. 

Pauline's team gained buy-in from agency leaders and higher-ups by showing how effective records management can lead to cost savings and improved efficiency.

The intersection of AI and records management

While Pauline admitted to not being a tech enthusiast, she acknowledged the role of AI in records management. 

"Artificial intelligence will enable us to more quickly and accurately roll out the records and data management program," she notes. 

AI's power lies in its ability to analyze vast quantities of data swiftly and accurately, facilitating the categorization, organization, and retrieval of records. This efficiency is particularly valuable in managing records across diverse agencies, each with a distinct set of technology platforms and unique data management challenges.

AI's role in records management also extends beyond categorization. It can aid in predictive analysis, anomaly detection, and even automated data retention policies. By learning from patterns and trends in data usage, AI can provide insights that inform better decision-making and strategy formulation.

Addressing privacy concerns and regulation

Ensuring the privacy of sensitive information is also of paramount importance. Pauline's perspective sheds light on the complex task of balancing the accessibility of historical records with the need to safeguard private data.

Pauline highlights the significance of the State's Freedom of Information Law in governing private data within records. This legal framework identifies specific information categories exempt from public disclosure, including personal addresses, birth dates, and sensitive security-related information.

While keeping data private is crucial, there are also challenges to managing public and private information in historical records. According to Pauline, these records contain confidential data and contribute to a comprehensive understanding of history. Striking a balance between safeguarding personal information and preserving historical context is ongoing.

Pauline highlighted the need to accommodate the changing nature of data and technology while preserving historical context. This challenge extends to the digital realm, where archives may include sensitive private data and other information.

The application of AI in records management could play a significant role in addressing this challenge. By utilizing AI algorithms, organizations can more efficiently identify and categorize sensitive information within historical records and automate redacting private data. Due to this automation, researchers and the public can access historical records without compromising individual privacy.

Pauline's advice for peers

Reflecting on her experiences, Pauline shared valuable advice for peers in the field. She emphasized the importance of patience and adaptability and stated that building alliances, particularly with financial divisions, is crucial to securing resources and support for records management projects. 

"You must build allies within your institution or government," she says. 

Securing buy-in from financial divisions is critical, as it ensures that resources are allocated appropriately to support records management projects. Collaborating with these divisions and demonstrating the economic benefits of effective records management garners support and empowers initiatives with the necessary resources.

Pauline encouraged more peer-to-peer sharing and collaboration to share lessons and insights across institutions. Pauline suggests contacting other institutions facing similar challenges to exchange insights and lessons learned. Such collaboration can lead to innovative solutions, the discovery of best practices, and the avoidance of common pitfalls.

She emphasizes that the landscape of records management is ever-changing, influenced by technological advancements and shifting organizational priorities. Being open to new approaches and willing to pivot based on emerging opportunities is essential for keeping records management practices relevant and effective.

Final thoughts

Pauline Toole's insights offer valuable guidance in the ever-changing landscape of records management and information governance. Her journey from a non-records background to leading the New York City Department of Records and Information Services underscores the transformative power of dedication, patience, and strategic alliances. 

As organizations navigate the challenges of records management and embrace the potential of AI, Pauline's experiences provide a roadmap for success in this critical domain.