This executive summary lays out highlights from the report HarnessingĚýArtificial Intelligence To Measure The Well-Being Of Quebec’s Diverse Regions, written by Max Bell School Master of Public Policy students as part of the 2021 Policy Lab.
Access the summary and presentation below, and read their full report here.
Canada Economic Development for Quebec Regions (CED) tasked a student Policy Lab team from the Max Bell School of Public Policy to explore the question “How might Artificial Intelligence be harnessed in federal regional economic development policymaking to better assess and anticipate the well-being of Quebec’s diverse communities?”
Currently, delivering targeted policy interventions in Quebec regions can be challenging due to an information gap, as economic indicators alone do not provide a full picture of how a region is doing. Data that could support a broader view of regional well-being may not exist, or presents barriers around availability, accessibility,Ěýand timeliness. Artificial intelligence provides an opportunity to improve efficiency in data analysis with the intention of transforming a descriptive-analytical approach to a predictive one, leveraging the growing amount of publicly available data to measure well-being. With more holistic, place-based information, CED is developing the capacity to create more adaptive, responsive, and inclusive policy interventions.
Understanding the nature of the problem, the Policy Lab team deconstructed the challenge question into four key components
- Selecting well-being indicators;
- Using data sources to measure indicators;
- Managing data through the lifecycle;
- Turning data into meaningful insights.
These four components guided the research and structure of the report, informing the development of a conceptual framework for harnessing AI to measure well-being. While there are many separate elements included in the framework, they all connect, meaning this framework should be viewed as an entire process, because determining what questions you have and what problems you want to solve will contribute to what indicators and data sources are selected and whether you need AI. The elements at the top of the framework - well-being goals and data governance - are most important in the long-term for the agency to build effective, ethical, and sustainable data management for AI. Although to get started on implementing the conceptual framework, selecting well-being indicators, creating a data management plan, and forming an essential data management team are the most critical. There are also potential opportunities for AI use along the different stages of the process, denoted with a star. However, the use of AI in stages 2 and 3 is primarily for efficiency in data collection and management. Rather, the use of AI to “assess and anticipate” well-being as set out in the question comes at the later stage.
Starting with the first stage, establishing well-being goals will help inform which indicators are selected. Economic indicators do not provide a complete picture of how a region is doing because they don’t include important information about the health of regions and communities such as, who is doing well from growth, whether growth is environmentally sustainable, how people feel about their lives, and what factors contribute to individual and country-level success. As a result, economists have advocated for models that consider a host of socioeconomic and environmental indicators that better reflect the needs and well-being of communities and the planet.
Additionally, measuring the well-being of regions will help policymakers better understand the interconnectedness of economic issues with indicators such as health, education, sustainability, inclusiveness, and how people use their time, among other multi-dimensional impacts. Paired with a global trend towards measuring well-being, demonstrated in numerous well-being indexes, including the Government of Canada’s (GoC) recent Quality of Life Strategy (QoL Strategy), looking beyond a narrow economic view will help policymakers better understand problems and respond with interventions that meet the whole needs of communities.
In measuring well-being of Quebec's regions, the Policy Lab team recommends that CED:
- Develop an organization-specific well-being framework aligned with the agency’s strategic plan, asking what they hope to achieve and what information is needed to inform progress? CED can look to the GoC’s QoL Strategy and adapt the framework to their context by choosing indicators that align with the agency’s well-being goals, while incorporating their existing economic indicators.
- Choose well-being indicators and corresponding measures within the framework that have reliable and relevant data that can be measured over time. The QoL Strategy provides valuable factors to consider and several indicators to select. While no indicator is perfect, it’s relevant to understand the trade-offs of what type of information an indicator can provide. Health offers a useful starting place as there are many proxies, data sources, and examples of using community-level data to understand well-being with several AI projects underway that CED can learn from.
- Use a dashboard to visualize well-being indicator data across regions and to measure progress over time. Most indexes use dashboards along with the recommendation from the GoC’s QoL Strategy, as dashboards provide more granularity useful for targeted policy intervention.
The second component of the conceptual framework is using data sources to measure indicators for well-being. There are different types of traditional and innovative data sources that CED could use to measure well-being indicators and to later leverage the use of AI, including traditional sources of household and consumption data, administrative data and surveys and censuses, and other types of statistics. On the other hand, innovative sources include telecommunications data, geospatial data, social media data and web search data among others. New and innovative data sources present great potential to help fill in knowledge gaps from traditional data sources and overcome their limitations. They can also allow CED to better analyze the opportunities for economic expansion in regions and have the potential to provide more real-time data, although have greater potential if used together.
Combining data sources provides the most opportunity for useful and targeted insights, although integrating various datasets can be time consuming because of the variations in the way data is defined, recorded, and analyzed. Additionally, the lack of consistency among local, provincial, and national systems, which have varying access and availability of data, and the lack of data sharing more broadly, can make it difficult to combine sources. It can also be challenging to collect data in remote communities due to a lack of accuracy and representativity. While reliable data is needed for effective decision-making, the majority of statistics capture national averages and can fail to recognize the disparities across populations and communities from a lack of disaggregated data, which is crucial to measuring multidimensional well-being.
In selecting data sources to measure well-being, the Policy Lab team recommends that CED:
- Identify the appropriate sources, whether traditional or innovative, that satisfies and aligns with its interests and desired outcomes. When identifying well-being measures and indicators, CED should assess types of available data sources that will provide the most valuable and relevant information.
- Use disaggregated data by age, gender, race, disability status, etc. By disaggregating the data by region and listed factors, CED would be able to understand the impacts of its policies more precisely and at a place-based level.
- Use a diverse set of data that include both traditional and innovative data sources to fill in knowledge gaps. One way to improve representativity of data is to use multiple data sources and improve data integration to gain a diverse perspective on well-being measures.
The third component of the framework is managing data through the lifecycle. Data management can enhance organizational data literacy by building understanding of how data is managed within the agency. It can also help identify opportunities and gaps in the data management lifecycle, improve efficiency and standardization of processes for replicability, and increase data capability for AI adoption.
Data governance is crucial to data management, focusing on data stewardship, data quality, and total data management to ensure data is consistent and trustworthy. It seeks to administer data, reduce costs and complexity, and realize new sources of value while being clear and simple to follow. While data governance provides overarching guidance as a set of rules and policies that determine decision-making and authority for data related matters, a data architecture focuses on the technology and infrastructure design for standardizing how data is collected, transformed, distributed, stored, and used. A data architecture must also reflect the appropriate technology and design to harness AI in the long-term. WhileĚýmany aspects overlap with the components involved in data governance, a data management plan describes the data’s management, analysis and storage and what data will be acquired and generated. The research highlights that data governance, a data architecture, and a data management plan should be aligned throughout the data management lifecycle to harness AI in the long-term.
In managing data sources, the Policy Lab team recommends that CED:
- Build a data governance and a data architecture framework to understand the risks and opportunities of managing its data. CED should identify who uses the data, and under what circumstances it is used - from collecting the sources, preparing the data, and storing it.
- Develop a data management plan which can help identify the weaknesses and opportunities of its current processes and systems by clarifying what data it wants to acquire and to generate. CED should ensure that critical people within the agency understand the data management process and identify if the organizational mission and priorities align with this new process.
- Form an essential data management team that could provide a diagnosis to identify gaps in the agency’s data efficiency and effectiveness. The data management team can help guide CED towards what it has and what it needs, building offĚýbest practices from other small organizations.
- Select a database management system that fits the agency’s requirements. A database management system (DBMS) is the software interface between users and a database and provides a visible view into a dataset as a single unit that is helpful for long-term data analysis.
In the fourth component of the framework, the Policy Lab team provides an overview on the use of Artificial Intelligence (AI) to help predict and assess well-being in Quebec’s diverse regions. There are both opportunities and challenges in harnessing AI. Advanced technology tools like AI can increase efficiency of the agency and can improve its existing processes. The use of well-being indicators along with innovative data sources can also enable AI models to be effective in predicting and measuring regional well-being. AI does however, come with significant challenges, including the lack of ethical frameworks as well as the complexities required to implement its tools. In the end, achieving business value from AI is based on the notion that an efficient government must be digitally transformed and can use technology like data analytics and AI when it is available.
Prior to using AI, which uses machines capable of reviewing information, weighing options, learning from past mistakes, and decision-making, the Policy Lab team recommends CED follow six AI adoption pillars: 1) Establish Organizational Readiness For AI; 2) Responsible AI Guidelines; 3) AI Strategy; 4) Harnessing AIĚýTools; 5) Policy Principles; and 6) AI Culture. Through research and consultations with stakeholders, these pillars were identified as essential when adopting AI tools. While most of the pillars will need to be established prior to adopting the AI models, the majority of the pillars do not require extensive governance or architecture plans.
Once CED established organizational readiness and responsible AI guidelines,Ěýthe Policy Lab team recommends that CED:
- Develop an AI strategy. This strategy should include both AI Project Ideation as well as a Proof of Concept (POC), which will help the department determine which problem requires AI to solve and review past examples where AI tools and algorithm models have solved a problem in the past.
- Test the AI models to determine the best tool. For the problems that require AI, CED will need to hire an expert in data analysis or science. The POC is essentially a feasibility check, providing the department with a review of the requirements needed to harness AI models, the pros and cons of each, and valuable insight on how to solve issues and deploy the solution. Prior to investing time and money into solving problems, the AI Project Ideation and POC would ensure CED successfully deploys and harnesses AI models moving forward.
- Review responsible AI guidelines and policies to ensure correct use of AI and decrease risk. Once the expert has completed the testing phase of determining which AI models best answers the question, CED must ensure responsible AI guidelines are adhered to. The use of AI models must be governed to ensure correct use and decrease risk.
- Visualize the output on a dashboard. More specifically, CED should continue using PowerBI to leverage existing staff skills and knowledge when it comes to creating the dashboard. Dashboards with visualizations and real-time analytics will provide CED with a high level view of results from each AI tool selected.
Finally, the various elements of the conceptual framework provide a guideline to best prepare CED in implementing AI tools to assess and predict regional well-being. Using AI could offer a long-term solution to some of CED’s existing data challenges and ensure that policy interventions are more targeted. With the government’s focus on assessing well-being and understanding the ever-increasing role technology plays in our lives, these four key components of the conceptual framework should be the blueprint for CED’s success. Strong data management and governance systems will best position CED to become a leader in future policy making within this domain.
Download the full version of this report here.
This Policy Lab was presented by our MPPs on July 15, 2021. Watch the video below:
About the authors
Ellen Rowe
MPP Class of 2021
Ěý
Ěý
Rym Cheriet
MPP Class of 2021
Ěý
Ěý
Mikayla Zolis
MPP Class of 2021
Ěý
Ěý
Leonardo Lozano
MPP Class of 2021
Ěý
Ěý
