Getting Started with Generative AI in the Public Sector

Jonathan Behnke, Chief Information Officer at City of San Diego

Jonathan Behnke, Chief Information Officer at City of San Diego

There has been a lot of hype surrounding generative AI and the potential to improve and streamline functions in public sector operations and services. Public sector organizations have been using different forms of artificial intelligence for years with image recognition, cyber-security tools, license plate readers, predictive analytics, and other functions embedded into SaaS solutions. Generative AI and large language models have evolved from previous generations of artificial intelligence to offer new opportunities to improve efficiencies, synthesize data, speed research, and replace or streamline mundane tasks.

Getting started with emerging technologies like generative AI can be a challenge, especially when technology staff and operational users have limited exposure and training to draw from. Generative AI has the potential for significant efficiency increases but organizations need to be mindful of the risks if they want to be successful.

Risks

The emergence of ChatGPT and other generative AI solutions has resulted in lessons learned for many organizations. Some did not realize that whatever they input into the prompt became part of the data used by the learning model. Organizations need to be aware of the risks of sharing sensitive data, PII, proprietary content, and data privacy. There are also questions about copyright infringement, inaccurate data, or algorithmic bias that can create liabilities if not managed properly. Even if the appropriate development and guardrails are put into place there are additional risks if a user blindly accepts the output of the model without first validating its accuracy. Generative AI is also creating new cyber security risks both from adversaries using it to streamline attacks and the potential increase in vulnerabilities as new solutions are implemented.

Policy

The best starting point is to develop a comprehensive policy to govern the procurement, operation, security, standards, and maintenance of a generative AI system. The policy development for AI is in the early stages for most organizations but early examples include resources from NIST, various federal agencies, state and local governments, the Gov AI Coalition, academic institutions, and commercial entities to draw from. Some organizations are integrating AI polices into existing IT governance, procurement, security, and privacy polices while others are opting to create a separate policy for AI.

There are many considerations including privacy, workforce impact, legal liability, contract language, algorithm design, roles and responsibilities, prohibited use cases, public records, and other areas that will need to be vetted by various areas of the organization. A policy builds the foundation for organizations to reduce the risks and realize the benefits of generative AI.

Getting Started with a Use Case

A small pilot project may be the best starting point to familiarize IT teams and operational users with implementing generative AI and realizing the benefits for a narrow internal use case. There are many opportunities to identify beneficial use cases across functions that require significant amounts of time to research documents, manual compilation or summarization of multiple data sources, call center knowledgebases, language translation, streamlining top document or content searches on an intranet site, or creating a generative AI workflow for multiple step processes that are currently disconnected and slow due to users having to look up information for each step.

Training Tech Teams

It will take time to get technical teams proficient in developing generative AI solutions and get familiar with best practices to train the model. Many vendor partners will assist organizations in getting off of the ground and a small pilot can build the necessary skills and experience to scale a solution once it is developed. A small pilot will provide perspective to evaluate the effectiveness, technology requirements, security, costs, ease-of-use, support needs, maintenance requirements, and scalability of a solution. One of the first things many organizations realize when they do a pilot is that their training data needs to be improved. Stale data, poorly written documents, and poor data quality can result in a generative AI solution that delivers biased, inaccurate, or poor responses.

Testing and User Training

Generative AI requires a different approach for user testing and training. Traditional IT test scripts usually have clear success criteria and defect reporting. Traditional user training usually follows workflows across specific screens and functions. Training for the use of generative AI is different and is best accomplished through best practices for prompt engineering and familiarizing users with good approaches to get the information that they are looking for. Users also need to know how to validate the information that they receive from the generative AI prompt, usually through hyperlinks to cited sources.

When testing a generative AI model there will likely be a diverse approach to how users interact with the prompt even though they are requesting the same information. A feedback mechanism is critical to capture user sentiment when the prompt returns information so that issues can be identified and corrected through iterative testing. Even with consistent user feedback on the effectiveness and performance of a solution there may be situations where multiple users receive the same response, but one will rate it as performing well and another may rate it as performing poorly.

Scaling Up and Operational Support

Like any new technology implementation, introducing a new generative AI solution will require ongoing maintenance and support resources to keep the solution current, effective, secure, and supported. An effective pilot project may generate additional demand for similar use cases from other business functions. The information from the pilot project on the effectiveness, technology requirements, security, costs, ease-of-use, support needs, and maintenance requirements will be important in scaling up the solution further with the appropriate resources and budget to support it.

It is likely that generative AI technology will continue to evolve rapidly, but one thing that we can be sure of is that it is here to stay and will likely introduce significant benefits and disruption to traditional public sector services in the future.