PDFFull Text
Introduction
The strengthening and enhancement of capacity are stated goals of several international ocean law and policy instruments that focus on biodiversity conservation, fisheries management, sustainable development, pollution, mineral extraction, traditional knowledge, and ocean science. Yet, achieving these goals has proved elusive, as illustrated by persisting divides in capacity and technology (Amon et al., 2022a; Bell et al., 2023). This has led to the emergence of new terminology, such as capacity sharing, which reflects the need for evolving practices away from unidirectional approaches and toward equitable partnerships (Harden-Davies et al., 2022) that recognize the value of existing knowledge (scientific or not), as well as practices and values held by a community (Spalding et al., 2023).
Existing metrics for “capacity-building” effectiveness tend to be output-related, such as the number of people trained, workshops held, or publications produced. These output metrics are easy to quantify but often fail to reflect meaningful success through lasting benefits for those involved. If “success” of capacity initiatives is defined solely in terms of quantity, without considering the actual impact and benefits for individuals taking part in it, there is a risk of further encouraging approaches or initiatives that perpetuate inequities and cause harm (Amon et al., 2022). Further, these types of metrics do not consider that capacity initiatives are bidirectional (Harden-Davies et al., 2022), based on mutual learning, and include local and Indigenous knowledge in addition to science.
The term “capacity initiatives” adopted in this article is used to avoid any one term such as capacity building/development/sharing, recognizing that different terms are preferred in different contexts. It reflects that all countries have installed capacities in different levels, and they should be considered as locally relevant knowledge systems and capabilities. This is particularly important in the understanding and management of connected ocean contexts, where entwined environmental and societal issues require diverse skills, knowledge forms, and cooperation both locally and globally (de Vos et al., 2023; Spalding et al., 2023).
For this short article, we convened scientists and ocean-policy specialists from multiple countries and backgrounds to discuss the importance of incorporating qualitative metrics in measuring, achieving, and communicating positive outcomes from capacity initiatives. Based on their experiences, the authors suggest preliminary metrics that can contribute toward effectively assessing outcomes of ongoing and future initiatives, and call for a more critical conversation about how the success of capacity initiatives is measured.
Metrics for Meaningful Outcomes
Here, we introduce a series of questions aimed at inspiring those who undertake the design, implementation, and monitoring of capacity initiatives through the consideration of desirable outcomes. We recognize that this is an area requiring further study and deliberation, and thus there are not yet definitive answers regarding what metrics are the most appropriate in each case.
1. How involved do participants feel in the design of the initiative?
One of the characteristics of a successful capacity initiative is that it is co-designed from the beginning, incorporating the needs, priorities, and aspirations of all partners (Singh et al., 2021; Harden-Davies et al., 2022)—partners should feel that there is an equal space for everyone to share their thoughts and make proposals.
Some of the metrics used to measure participation in the design of the initiative include how and whether the participants reflect the diversity of voices in a given context, the number and role of local participants and organizations, the extent and duration of their participation, and the degree to which each partner’s stated priorities were included in the initiative.
The use of qualitative metrics helps to provide deeper insights and further inform outcomes. This information can be gathered through surveys or interviews, which could include questions about participant perceptions of their level of inclusion, the degree to which their needs and expectations were met, or the level of trust that was established within the initiative. These surveys can be anonymized, if needed, and administered to participants at the midpoint and at the end of their involvement.
2. How inclusive was the initiative for participants?
Feelings of inclusiveness and safety are essential for meaningful engagement of partners in a capacity-sharing initiative (Amon et al., 2022b). In practice, participants should feel comfortable in openly sharing how they perceive and feel about the implementation of an initiative. A potential pathway to inclusiveness between partners is through the creation of a foundational agreement based on shared values (e.g., trust and non-judgment) and understanding of what a collective working environment should look like. Such conversations avoid uncomfortable power dynamics that inhibit participants from voicing their thoughts and fully contributing to the process, including regarding the improvement of the initiative.
A valuable avenue for gathering participants’ feelings and thoughts could be through regular check-ins and confidential conversations where participants can express their concerns and difficulties, report on their progress, and discuss desired next steps. A truly inclusive capacity initiative is an important enabler for strengthening and maintaining local self-sustained leadership in the long-term and contributes to knowledge co-production.
3. What are the main benefits and impacts resulting from the initiative as perceived by participants? What mechanisms for long-term support are available?
Benefits and positive impacts on participants in the long term are among the most desirable outcomes of a capacity initiative. The number of participants who conclude a training program, summer course, or workshop, for instance, will provide information about how many people attended such events, but not about their perceived impacts on supporting their careers, applicable skills, and well-being in the long term. Positive long-term impacts of benefits can be defined, although not exhaustively, as perceived strengthening of technical skills (e.g., knowledge about a topic) and soft skills (e.g., oral and written communication, development of scientific thinking, publications and presentations, organization of field trips/research cruises, application of methodologies, advocacy, outreach, leadership, project management), and the development of a support network.
Qualitative information gathered through posing open-ended questions, semi-structured interviews, or conversations dedicated to gathering feedback can reveal richer and more nuanced information that can help partners to better understand the direct impact of the initiative on the participants. Based on participants’ feedback and active participation, partners can create mechanisms and long-term initiatives to ensure that capacity outcomes continue to benefit participants beyond the immediate duration of the initiative. Long-term qualitative data can also capture indirect impacts of capacity initiatives, for instance, development of confidence; creation of support networks and other connections that can potentially connect participants to future partnerships, jobs opportunities, grant proposals, and other capacity initiatives; and engagement in policy processes.
Final Reflections
Here, we have put forward descriptions of metrics for several desired outcomes of capacity initiatives that have often been marginalized or ignored in their design, implementation, and monitoring. Questions suggested can be used as starting points for both designing better initiatives and measuring whether they produce meaningful outcomes. If such measures and metrics are applied throughout and beyond the duration of the initiative, adjustments can be made so that those involved can evolve to reflect lessons learned.
We suggest that a combination of quantitative and qualitative metrics is desirable—particularly when capacity initiatives concern individuals. Such an approach can meet the demands of funding agencies without leaving behind the evaluation of meaningful outcomes that capacity initiatives can have on individuals and communities locally, nationally, regionally, and globally. However, we acknowledge that proposing qualitative metrics is resource and time intensive and that establishing such metrics can be a challenging task, requiring consultations with experts working, for instance, in disciplines and practices in qualitative approaches. Nevertheless, we urge that starting these conversations is imperative to ensure co-created, fit-for-purpose, impactful, and long-term capacity.
We also acknowledge the importance of accompanying such conversations with awareness raising, including of funders, on the importance of supporting initiatives that prioritize quality over quantity. This is essential to incentivize the recalibration of priorities and inclusion of qualitative metrics as part of capacity-building/development/sharing processes. Finally, we suggest that holding knowledge exchange sessions that are focused on incorporating qualitative metrics into designing, monitoring, and evaluating capacity initiatives is an important starting point toward change.
Acknowledgments
MG, MV, HHD, SC, CG, VL, and GS gratefully acknowledge the Nippon Foundation for supporting this research through the Nippon Foundation-University of Edinburgh Ocean Voices Programme. DA received funding from UC Santa Barbara’s Benioff Ocean Science Laboratory.