Oceanography The Official Magazine of
The Oceanography Society
Volume 37 Issue 1

View Issue TOC
Volume 37, No. 1
Pages 19 - 25

OpenAccess

Challenges and Strategies for Knowledge Co-Production in Ecosystem and Natural Hazards Research

By Sarah E. Kolesar , Peter Ruggiero, Desirée Tullos, Jenna H. Tilt, Bryan Tilt, and Brook M. Thompson 
Jump to
Article Abstract Full text Citation References Copyright & Usage
Article Abstract

The grand environmental challenges society faces cannot be solved without knowledge contributions from communities that are touched by the outcomes and impacts of research. Recognizing and valuing these sources, a research method known as co-production of knowledge involves collaboration of interested parties and knowledge owners in activities ranging from identifying research needs to study design, data collection, interpretation, and evaluation. We present a novel approach to the administrative and financial support for knowledge co-production developed by Oregon Sea Grant and provide examples from two funded project teams. We summarize barriers to these approaches, discuss points of symmetry, and propose good practices. One project emphasized equitable alternative strategies for chronic and acute coastal hazard adaptation and mitigation. The other project looked beyond dam removal in the Klamath River Basin to generate place-based and generalizable understanding of, and equitable decisions around, water quality and socio-ecological systems undergoing change. Both projects provide high-level lessons regarding opportunities and challenges encountered in the engaged research process, highlighting flexibility and adaptability as two important considerations.

Full Text

Introduction

Research does not always lead to better decision-making and management, a problem known as the knowledge-action gap (Cook et al., 2013), in part because it excludes the communities experiencing the impacts and outcomes of research and resulting management decisions. Beyond being impacted by knowledge outcomes, individuals and communities also possess knowledge that can make important contributions to both science and management from different perspectives (Bartlett et al., 2012; Berkes, 2003). Uniting rather than integrating knowledge systems can produce a richer understanding of a system and alleviate distrust and power differentials that are associated with knowledge possession (Nadasdy, 1999).

Valuing multiple ways of knowing can include Indigenous Traditional Ecological Knowledges (ITEKs) and other sources of local knowledge and experiences. Recognizing and valuing these sources, a research method known as co-production of knowledge has been described in the literature as “the contribution of multiple knowledge sources, ways of knowing, and perspectives from different user groups with the goal of co-creating knowledge and information.” (Cooke et al., 2021). The research process involves interested parties and knowledge owners (see Box 1 for definition of this term and others used in this article) in activities ranging from identifying research needs to study design, data collection, interpretation, and evaluation.

 

> High res box

 

Innovations in administrative support for co-production of knowledge are essential to its success. While the National Sea Grant network has conducted use-​inspired research that addresses community needs for over 50 years, the levels of funding and time needed to adequately address complex, community-​driven coastal issues through knowledge co-​production are not always available. Oregon Sea Grant (OSG) has recognized that research teams need tiered funding, time for building trusting relationships, and facilitated administrative support in order to address cutting-edge socioeconomic and biophysical science related to important marine and coastal issues. OSG has enabled this process through a competition called SEED-LEAF (Figure 1).

 

FIGURE 1. The Oregon Sea Grant SEED-LEAF competition funds modest, one-year SEED projects and a single, substantial LEAF project. Projects require knowledge co-production with affected parties, at least two distinct disciplinary approaches, and outreach to the broader community. Proposals, projects, and the process itself are evaluated at each stage of the six-year cycle; the upcoming 2026–2032 competition will mark the third SEED-LEAF iteration. > High res figure

 

In this article, we present a novel approach to administrative and financial support for co-production of knowledge developed by OSG, providing examples from two funded projects, acknowledging barriers to these approaches, discussing points of symmetry, and introducing lessons learned as good practices relevant to any project aiming to co-produce knowledge. The first SEED-LEAF project (2017–2023) aimed to co-produce and evaluate equitable alternative strategies for chronic and acute coastal hazard adaptation and mitigation via structured modeling workshops, semi-​structured interviews and focus groups with Oregon’s coastal Latinx community, and sustained engagement. The second SEED-LEAF project (2021–2026) looks beyond dam removal in the Klamath River Basin to generate place-based and generalizable understanding of water quality and river health in order to improve equity around water management decisions by considering perspectives from agriculture, conservation groups, the fishing industry, and the Yurok Tribe, for whom salmon have always been an important source of livelihood and cultural identity.

 

The Oregon Coastal Futures Project

Coastal communities in Oregon increasingly face challenges related to chronic (storm frequency, sea level rise) and acute (earthquakes, tsunamis) coastal natural hazards. Planning for both types of coastal hazards has historically faced barriers such as limited financial resources, lack of capacity at local levels, and slow-to-change policy and regulations. The Oregon Coastal Futures Project (OCF) co-developed equitable adaptation pathways and hazard resilient mitigation measures by quantitatively examining resilience decisions and associated trade-offs across a range of scales (communities to counties to the entire state). We developed, applied, and assessed a transdisciplinary framework to increase coastal community adaptation and resilience to chronic and acute coastal hazards. This framework combined community engagement, an alternative futures model, and robust evaluation of policy and coastal hazard scenarios to inform decision-​making. Our goal was to apply an adaptable framework that combined expertise in natural sciences (geomorphology, coastal hazards), social sciences (environmental and resource economics, land use planning), engineering (civil infrastructure), and computer science (mathematical modeling) to increase adaptation and resilience within Oregon’s coastal communities. Two aspects of the OCF project engagement strategy are discussed relative to knowledge co-development: working with an advisory council and collaborating with the coastal Latinx community.

 

Engagement Strategy

The OCF project design—collaborative production of knowledge and adaptive capacity through alternative scenarios assessment—relied on the success of our engagement plan. A significant element of this plan was to work with a group of advisors identified through previous projects and the initial SEED project. Advisors included federal and state agency employees; county commissioners, planners, and emergency managers; city planners and emergency managers; and nongovernmental organization representatives, each of whom provided a depth of expertise and experience to guide policy scenario development and help parameterize resilience performance metrics. To expand the project’s reach, we actively chose advisors who worked with large and non-overlapping communities with whom they regularly interacted regarding coastal hazards issues.

Ongoing interactions with the Advisory Council were a key objective of the project and the full complement of advisors was critical in achieving project results. Detailed agendas and “read ahead” materials were sent prior to each touch point. At each meeting, advisors responded to proposed adaptation policy scenarios and shared their perceptions of environmental, economic, and social costs and benefits and whether trade-offs were accurately portrayed in scenarios. Specific feedback and discussion followed advisors’ reactions to the scenario results by soliciting what people found surprising, and why, and if the results and the visualizations motivated new thinking about policies. Presentations and discussions were primarily led by graduate student researchers funded by the project. For example, the model framework was used to facilitate actionable research on hazard planning for Cascadia Subduction Zone earthquakes and tsunamis, with a focus on the transportation sector (Sanderson et al., 2022). Graduate students often collaborated with individual Advisory Council members to request more information, data, or to follow up on key points. These professional-student interactions allowed for informal mentoring and increased the applicability of the research being produced. Within two weeks following each Advisory Council meeting, we distributed a summary and planned next steps. As an indication of individual commitment, most of the original advisors (~20 individuals) remained engaged with the project throughout its duration, and five individuals were invited to participate on the Community Advisory Council for a continuing project, the $19M National Science Foundation-funded Cascadia Coastlines and Peoples Hazards Research Hub.

One dimension of the OCF project was to identify how specific policies may decrease sensitivity to hazards for marginalized populations. While it is widely understood that vulnerable and underrepresented populations (e.g., low-​income, persons with disabilities, minorities) are more sensitive to the impacts of disasters (Cutter et al., 2003), there is little information regarding how specific community actions can mitigate this sensitivity. In Oregon, the largest minority populations in coastal communities, the Latinx (about 10% of the total population), are particularly vulnerable to coastal hazards because of their employment in industries highly exposed to coastal hazards (e.g., fisheries, accommodation, food services). They also have a lower adaptive capacity (e.g., low-income, English as a second language, systemic racism) to prepare for and respond to hazards.

To establish connections with the coastal Latinx population, we partnered with Oregon State University (OSU) Extension, an interested party, through cooking classes and other events specifically directed to this population. After attending several events, we co-developed interview questions with our Extension advisors and a few Latinx coastal residents. Over a nine-month period, we conducted focus group interviews before each cooking class, then participated in cooking, eating, and cleaning up with participants. Recruitment for all focus groups—​completed by our Extension advisor to keep contact information confidential—​was done mostly through word-of-mouth and by community flyers distributed in Spanish by Extension advisors. Activities for children attending the meeting with their parents were provided, and all participants were compensated with a grocery store gift card. In total, 34 Latinx residents from two coastal communities participated in this initial study as knowledge owners.

Our findings revealed a considerable gap in the places Latinx residents felt comfortable going to in times of emergency, primarily churches and community-​based organizations (CBOs), while disaster preparedness plans emphasized protection of public critical facilities such as fire and police stations (Stanton and Tilt, 2023). Key reasons for this disconnect given by Latinx residents were feelings of not belonging in traditionally designated spaces, as well as concerns regarding deportation. With this knowledge, we altered the OCF alternative futures model to evaluate the hazard risk to these community assets, in addition to critical facilities, and developed potential targeted adaptation strategies that protect the most valuable and exposed assets (Figure 2).

 

FIGURE 2. Project process of deep community engagement, community asset identification, risk quantification (for both chronic and acute hazards), and assessment of mitigation and risk reduction measures via modeling equitable dynamic adaptation pathways. > High res figure

 

Since then, we have expanded to new communities and co-developed research questions and protocols with additional CBOs, recruiting more than 60 additional Latinx study participants (e.g., knowledge owners). These research protocols consider how the lived experience of COVID influenced social capital and trust between Latinx residents and their communities. Preliminary findings show that while Latinx residents are now more cognizant and somewhat better prepared for disasters due to their experiences with COVID, they rely on only a very few CBOs for resources and information (Blockstein and Tilt, 2023).

We continue to refine community engagement practices initiated by the OCF project, including: (1) co-developing research protocols and trainings with CBO advisors, (2) inviting the CBOs to suggest logistics (e.g., time, date, place, childcare, and recruitment) that align with residents’ needs and to allow for confidentiality, (3) providing meals to all participants and their families to facilitate informal discussion and to build trusting relationships, and (4) providing direct or in-kind compensation to the CBOs for their time and expertise, and gift cards for all participants. The co-production of knowledge gained by implementing these engagement practices provided essential information to Oregon’s emergency managers, coastal planners, and others to inform equitable hazard adaptation decision-​making and community preparedness practices.

 

Multiple Ways of Knowing About Water Quality and Socio-ecological Systems Undergoing Change

The Klamath dam removals have attracted scientists, media, and financial resources to the Klamath basin. However, both journalism and research life cycles do not typically lend themselves to establishing long-term relationships with uncertain outcomes (Cooke et al., 2021). OSG funded this knowledge co-production project based on trust in the ability of the partners to navigate complex relationships from which transformative outcomes could emerge. The project emphasizes water quality in the Klamath basin, a topic central to human and ecological health. Despite its importance, water quality is often subordinate to water quantity in the Klamath basin, given long-standing tensions over allocating scarce water resources, as well as the complex interactions between human and ecosystem components that influence water quality.

In addition to applying transdisciplinary approaches of Western science, the project approached scientific questions about drivers of and responses to water quality changes from a framework that emphasized place-based knowledge and multiple ways of knowing. This goes far beyond applying Indigenous knowledge in the study design (i.e., utilizing knowledge of tribal fisherpeople to establish sampling sites, sample timing, and hypotheses). A core partnership between academic institutions and a sovereign tribe was central to our approach. In part because of the highly transdisciplinary and cross-cultural nature of the project, the team navigated challenges uncommon in typical research and engagement as we aimed to put a decolonizing approach into practice (Smith, 2013). Below, we summarize some of the administrative and scientific hurdles of cross-cultural research and report on strategies we applied to overcome them (Figure 3).

 

FIGURE 3. Activities designed to support co-development of knowledge and equitable decisions around water quality management in the Klamath River. The project coupled multiple ways of knowing with structured decision-making to combine insights from ecological and social science to evaluate water quality management for the uses and benefits people value (Conroy and Peterson, 2013). > High res figure

 

First, cross-cultural research requires aligning administrations from different philosophies and processes. This work demanded researchers’ and administrators’ flexibility, humility, and patience, and required educating administrators, particularly at Western institutions. Administrators from different cultures and institutions operated on dissimilar timescales and expected varying levels of administrative detail, which led to conflicts in processing agreements and contracts. For example, it took OSU’s legal team nearly eight months to negotiate revisions to a data-sharing agreement (DSA), after project researchers spent four months drafting and approving it. The final agreement addressed ownership of data and intellectual property, a process for classifying data, and details of data security and management, among other things. For the human participant research, the DSA created two separate (but linked) projects: one led by OSU focusing on non-tribal knowledge owners in the basin, and one led by the Yurok Tribe focusing on the perspectives of tribal knowledge owners. Regarding different timescales of institutions, we found that the researchers needed to constantly monitor and nudge administrators at both Western and tribal institutions to communicate (i.e., respond to emails, copy to all parties), review and approve documents, pay invoices, and execute other time-consuming efforts to move the research past administrative delays. In addition, engagement with OSG as the sponsor greatly facilitated some dialogue that accelerated administrative activity. Project planning should anticipate that many administrative offices are understaffed relative to their demands and that cross-cultural collaboration generates a greater demand on administrative offices than traditional disciplinary projects.

The most robust aspect of this agreement involved a process for the tribe’s Cultural Committee to review and classify data into three categories with different permissions for dissemination: protected (for tribal use only), confidential (for project use), and public. This process protected intellectual property and culturally vulnerable understandings and locations and helped avoid some well-documented harmful outcomes of research with American Indian and Alaskan Native communities (Cochran et al., 2007; Christopher, 2005; Beauvais and Trimble, 1992). Building this capacity and trust in project partners took time and resources to ensure objectives were met. One key mechanism for building capacity and trust was the availability of SEED funding to provide the time and justification to clarify the project’s values, objectives, and outcomes, as well as to seek tribal leadership approval prior to LEAF proposal submission. Our time investment will contribute to long-term institutional capacity as the DSA not only governs this project but also will serve as a template for future collaborations between tribal and academic institutions.

Second, cross-cultural and transdisciplinary work can require finding new (and sometimes uncomfortable) ways of communicating and documenting knowledge. How do we represent knowledge and uncertainty across a diverse spectrum of “data”? This project coupled (1) results of qualitative and quantitative interviews with tribal members and other knowledge owners who have multi-​generational experience with the river (i.e., conservation groups, recreational guides, agricultural water users, and fisherpeople) with (2) field data and output from numerical and statistical ecological models. Rather than attempting to integrate multiple ways of knowing and confronting the problems that integration generates (Nadasny, 1999), we explored approaches characterized by Two-Eyed Seeing in the Indigenous methodologies research (Reid et al., 2020) and multi-model comparison in Western science (Lewis et al., 2021).

An example of uniting knowledge systems around water quality is how to define river productivity. The concept of productivity generally refers to how much living material (e.g., algae, plants, insects, fish) a river produces over time. Western science may rely on numerical thresholds (e.g., chlorophyll a, turbidity), which sometimes lack direct ecological relevance (Jakus et al., 2017) and/or a historical point of reference. In contrast, tribal members may define a river’s productivity based on how many hours they spend cleaning algae from their nets and may provide historical context for when a shift in productivity occurred. Research can also connect landmarks between different knowledge systems, such as comparing quantitative measures of algal biomass from Western science to times when tribal members feel that algae impeded their ability to fish or conduct cultural activities.

Key challenges to Two-Eyed Seeing and multi-model comparisons include visualizations and uncertainty. We explored visualizations to present different knowledge systems without reducing context and resolution. This work was inherently biased towards Western frameworks that rely heavily on graphs, but collaboration with artists, tribal leadership, and other invested parties have helped to identify and revise visualizations. We found that geographical storytelling and visually based products (e.g., art installations, photo series, story maps, physical materials for non-internet users) can be effective as unifying platforms. Uniting or bridging knowledge systems includes and goes beyond policy and technical decisions, and also aims to develop understanding of the knowledge systems themselves.

This project demonstrates how bringing together multiple knowledge frameworks can address major gaps in Western science, and that doing so requires overcoming strong institutional barriers, entrenched scientific traditions, and unrecognized power differentials. This work requires the slow and patient effort of critical listening and building trust among researchers and institutions. Our experience is that this process requires sustained funding and relentless commitment to inefficient collaboration, but that it ultimately enriches all parties’ experiences and perspectives.

 

Discussion

By explicitly incorporating contributions from local communities experiencing the outcomes and impacts of research, co-production of knowledge is an effective strategy for addressing some of society’s greatest challenges. We recognize that all communities are unique and suggest strategies for overcoming challenges as starting points in knowledge co-production. These strategies, and others, should be tailored and developed to meet the specific needs and circumstances of partners, advisors, idea generators, knowledge owners, and interested parties (Box 1).

First and foremost, the challenge in co-producing knowledge is sustaining community engagement and relationship building with individuals and communities. Community engagement must have a solid foundation of mutual trust and reciprocity. Past experiences with researchers and others that did not take community concerns into account, or follow-​up on study findings, can erode trust. To develop sustained community engagement, built on a strong foundation of trust, we recommend several strategies:

1. Collaborate with partners and advisors. CBOs and local advisory boards offer valuable place-based knowledge and community access. These interested parties can identify locally relevant issues, help recruit research participants, and facilitate communication and outreach. Developing and sustaining partnerships begins well before project initiation and extends long beyond its conclusion. We recommend articulating shared goals and expectations with knowledge holders, including timelines and rules of engagement. These “living documents” should be broadly accessible and periodically reviewed to create opportunities for continuing equitable representation of all voices. Additional good practices include acknowledging partners’ or advisors’ contributions in publications and providing compensation when possible.

2. Create opportunities for shared experiences. Opportunities to get to know one another in informal settings such as sharing meals, field trips to places of community significance, and other activities that allow for a relaxed setting can help shed the “researcher/researchee” vibe.

3. Provide innovative funding opportunities. Creative grant support, such as the OSG SEED-LEAF opportunity, provides researchers additional time and flexibility to do many of the above to co-produce research projects that provide direct community benefits.

Another significant challenge to building cross-cultural collaborations is overcoming institutional and bureaucratic differences when initiating community engagement and research. For example, in the Klamath project, data sharing agreements that met tribal best interests did not fit within the existing university protocols. In the OCF project, additional institutional approvals were required to provide gift cards, childcare, and meals in order to creatively address systemic barriers for study participants. We recommend that research teams reach out early and communicate regularly with administrators about timelines, regulatory flexibility in participant compensation, ownership of intellectual property, and other potential conflict points.

Co-producing new knowledge requires a mutual understanding of existing knowledge. Researchers should be open to different epistemologies that explain natural and human systems and processes, as well as how knowledge is shared in communities, including through stories and artwork. Strategies to facilitate the incorporation and comprehension of diverse knowledge include:

1. The research team can iteratively produce informational products in collaboration with partners, advisors, knowledge holders, and interested parties to increase comprehension. Innovative visualization and modeling techniques need to be based on community preferences for and comprehension of models and information.

2. Communicating project outcomes by coupling technical results with an interactive panel of experts and community members who can discuss local implications can increase understanding and project engagement.

Bolstered by a novel approach to administrative and financial support for knowledge co-production developed by Oregon Sea Grant, projects that focused on complex societal issues, including Oregon’s chronic and acute coastal hazards and water quality in the Klamath River Basin, provide lessons regarding opportunities and challenges of a locally engaged research process. By incorporating a wide range of knowledge co-production practices into these efforts, both projects demonstrate how sustained and meaningful community engagement can result in policy-relevant research that enables equitable community resilience.

 

Acknowledgments

This contribution was prepared by Oregon Sea Grant under award numbers NA18OAR4170072 and NA22OAR4170102 from the National Oceanic and Atmospheric Administration’s National Sea Grant College Program, US Department of Commerce, and by appropriations made by the Oregon State Legislature. The statements, findings, conclusions, and recommendations are those of the authors and do not necessarily reflect the views of these funders. Comments from two reviewers improved this submission.
Citation

Kolesar, S.E., P. Ruggiero, D. Tullos, J.H. Tilt, B. Tilt, and B.M. Thompson. 2024. Challenges and strategies for knowledge co-production in ecosystem and natural hazards research. Oceanography 37(1):19–25, https://doi.org/10.5670/oceanog.2024.225.

References

Bartlett, C., M. Marshall, and A. Marshall. 2012. Two-Eyed Seeing and other lessons learned within a co-learning journey of bringing together indigenous and mainstream knowledges and ways of knowing. Journal of Environmental Studies and Sciences 2:331–340, https://doi.org/10.1007/s13412-012-0086-8.

Beauvais, F., and J.E. Trimble. 1992. The role of the researcher in evaluating American Indian alcohol and other drug abuse prevention programs. Pp. 173–202 in Cultural Competence for Evaluators. M. Orlandi, ed., National Clearinghouse for Alcohol and Drug Information, Washington, DC.

Berkes, F. 2003. Alternatives to conventional management: Lessons from small-scale fisheries. Environments 31(1):5–20.

Blockstein, J., and J.H. Tilt. 2023. Exploring the relationship between community assets and social capital in a disaster-resilience context for Latinx communities along the Oregon coast. Paper presented at American Association of Geographers Annual Conference, March 23–27, 2023, Denver, CO.

Christopher, S. 2005. Recommendations for conducting successful research with Native Americans. Journal of Cancer Education 20(suppl.):47–51.

Cochran, P., C. Marshall., C. Garcia-Downing, E. Kendall, D. Cook, L. McCubbin, and R.M.S. Gover. 2007. Indigenous ways of knowing: Implications for participatory research and community. American Journal of Public Health 98:22–27, https://doi.org/​10.2105/AJPH.2006.093641.

Conroy, M.J., and J.T. Peterson. 2013. Decision Making in Natural Resource Management. Wiley-Blackwell, New York, NY, 480 pp.

Cook, C.N., M.B. Mascia, M.W. Schwartz, H.P. Possingham, and R.A. Fuller. 2013. Achieving conservation science that bridges the knowledge-action boundary. Conservation Biology 27:669–678, https://doi.org/10.1111/cobi.12050.

Cooke, S.J., V.M. Nguyen, J.M. Chapman, A.J. Reid, S.J. Landsman, N. Young, S.G. Hinch, S. Schott, N.E. Mandrak, and C.A.D. Semeniuk. 2021. Knowledge co-production: A pathway to effective fisheries management, conservation, and governance. Fisheries 46(2):89–97, https://doi.org/​10.1002/​fsh.10512.

Cutter, S.L., B.J. Boruff, and W.L. Shirley. 2003. Social vulnerability to environmental hazards. Social Science Quarterly 84(2):242–261, https://doi.org/​10.1111/1540-6237.8402002.

Jakus, P.M., N. Nelson, and J. Ostermiller. 2017. Using survey data to determine a numeric criterion for nutrient pollution. Water Resources Research 53(12):10,188–10,200, https://doi.org/​10.1002/2017WR021527.

Lewis, K.A., K.A. Rose, K. de Mutsert, S. Sable, C. Ainsworth, D.C. Brady, and H. Townsend. 2021. Using multiple ecological models to inform environmental decision-making. Frontiers in Marine Science 8:625790, https://doi.org/10.3389/fmars.2021.625790.

Nadasdy, P. 1999. The politics of TEK: Power and the ‘integration’ of knowledge. Arctic Anthropology 36(1–2):1–18.

Reid, A.J., L.E. Eckert, J.-F. Lane, N. Young, S.G. Hinch, C.T. Darimont, S.J. Cooke, N.C. Bain, and A. Marshall. 2020. “Two-Eyed Seeing”: An Indigenous framework to transform fisheries research and management. Fish and Fisheries 22(2):243–261, https://doi.org/10.1111/faf.12516.

Sanderson, D., D. Cox, A. Barbosa, and J. Bolte. 2022. Modeling regional and local resilience of infrastructure networks following disruptions from natural hazards. ASCE Journal of Infrastructure Systems 28(3), https://doi.org/10.1061/(ASCE)IS.​1943-555X.0000694.

Smith, L. 2013. Decolonizing Methodologies: Research and Indigenous Peoples. University of Otago Press, Dunedin, New Zealand, 256 pp.

Stanton, K.A., and J.H. Tilt. 2023. Building resilient Oregon coastal communities: Reimagining critical facilities through Latinx sense of place. International Journal of Disaster Risk Reduction 87:103600, https://doi.org/10.1016/​j.ijdrr.2023.103600.

Copyright & Usage

This is an open access article made available under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution, and reproduction in any medium or format as long as users cite the materials appropriately, provide a link to the Creative Commons license, and indicate the changes that were made to the original content. Images, animations, videos, or other third-party material used in articles are included in the Creative Commons license unless indicated otherwise in a credit line to the material. If the material is not included in the article’s Creative Commons license, users will need to obtain permission directly from the license holder to reproduce the material.