PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of frontpubhealthLink to Publisher's site
 
Front Public Health. 2017; 5: 182.
Published online 2017 July 28. doi:  10.3389/fpubh.2017.00182
PMCID: PMC5532392

Framework to Define Structure and Boundaries of Complex Health Intervention Systems: The ALERT Project

Abstract

Health intervention systems are complex and subject to multiple variables in different phases of implementation. This constitutes a concrete challenge for the application of translational science in real life. Complex systems as health-oriented interventions call for interdisciplinary approaches with carefully defined system boundaries. Exploring individual components of such systems from different viewpoints gives a wide overview and helps to understand the elements and the relationships that drive actions and consequences within the system. In this study, we present an application and assessment of a framework with focus on systems and system boundaries of interdisciplinary projects. As an example on how to apply our framework, we analyzed ALERT [an integrated sensors and biosensors’ system (BEST) aimed at monitoring the quality, health, and traceability of the chain of the bovine milk], a multidisciplinary and interdisciplinary project based on the application of measurable biomarkers at strategic points of the milk chain for improved food security (including safety), human, and ecosystem health (1). In fact, the European food safety framework calls for science-based support to the primary producers’ mandate for legal, scientific, and ethical responsibility in food supply. Because of its multidisciplinary and interdisciplinary approach involving human, animal, and ecosystem health, ALERT can be considered as a One Health project. Within the ALERT context, we identified the need to take into account the main actors, interactions, and relationships of stakeholders to depict a simplified skeleton of the system. The framework can provide elements to highlight how and where to improve the project development when project evaluations are required.

Keywords: food safety, food security, primary production, food chain, dairy chain, interdisciplinary, transdisciplinary, One Health

Introduction

Recent financial, economic, social, environmental, and health crises have led to the renewed recognition that collaborative approaches between disciplines are urgently needed to tackle such global challenges (2, 3). Consequently, the approach to emerging pandemics, as well as climate change, drug resistance, food and water security, and safety, has shifted from an interdisciplinary approach of experts, whereby experts collaborate across disciplinary boundaries, to a transdisciplinary approach that integrates society and science by including all potentially affected or otherwise relevant stakeholders (46). This transcends traditional boundaries and integrates knowledge and perspectives from scientific and non-scientific sources (2, 7). Many health communities have proposed transdisciplinary and systemic approaches with different focus points, such as EcoHealth, Global Health, Planetary Health, or Health in scaled Social–Ecological Systems (5, 6). The end goal is to have an additional instrument to improve the effectiveness of health intervention/care projects, thereby ensuring safety for humans, animals, and the environment alike (8).

The One Health approach, and ALERT as example, employed in a health intervention project that will be used in our manuscript, aims at simultaneously considering human and ecosystem health (9). Integration of multiple disciplines, sectors, stakeholders, living and inanimate elements yields highly complex constructs with varying dynamics at different scales. Although there is considerable literature describing the integrated approaches to health (10, 11), to the best of our knowledge, there are no recognized guidelines on how to evaluate to what extent the underlying integration as a principle contributes to address especially complex health problems, such as antibiotic resistance, outbreaks of highly infectious or non-communicable diseases, or ecotoxicology (12). There is a clear need for methods to represent and analyze such initiatives for management and evaluation purposes using a systemic and integrated approach (13).

The aim of the Network for Evaluation of One Health,1 a Transdomain action of the European Cooperation for Science and Technology, is to enable appropriate evaluations of One Health activities and hence comparison of initiatives as well as informed decision-making and resource allocation. Its conceptual framework includes the definition of the (One Health) system in which the initiative is implemented and the definition of the scale and boundaries of the system under evaluation. Human relationships, cultural behaviors, languages expressions, governance organizations, and constructive collaboration within interdisciplinary groups are all elements to be potentially included in a system. A possible way to visualize interactions and connections in and among different systems can be the system network approach.

We propose to employ a complex systems’ perspective to overcome the shortcomings of the traditional reductionist approaches (1416). We consider “system thinking” as the process described by Whitehead et al. (17). These authors describe the “baseline lexicon of systems thinking” as being: descriptive scenarios, system boundaries, system stakeholders, scope of the analysis, type of system (state of system and life cycle of system), metrics, axiological components, observer effects, normative scenarios, indices of performances, and development alternatives (out-scoping, evaluating and ranking alternatives, interactions, iterating analysis, and leverage points). Occasionally, more appropriate elements can be added or other lexical components can be used.

In health, systemic techniques have been applied to work on problems such as obesity and epidemic diseases. WHO recommends some main techniques to identify points of interventions (leverage points) in complex health-related systems (3). The guideline differentiates building blocks of a health system such as health services, health workforce, medical technologies, financing, governance, and the main system goals like improved health, responsiveness, and improved efficiency.

The framework advocates a systemic overview of those building blocks to visualize synergies and the dynamic architecture. The main goal of the approach is to consider the effect of an intervention across as many major subsystems of the health system as possible. This process is initiated with a “stakeholders analysis,” where the interconnections and perspectives of each stakeholder are inventoried. This is overlaid with an open and transparent network of interventions and their possible consequences.

In a more generic manner, systems’ thinking has been applied in project evaluation to inform policy makers and executives for best resource allocation (18). The systemic approach is intended to design programs and policies that are aware of and prepared for possible unintended consequences and that integrate multiple stakeholder perspectives. The resulting framework or model should describe and predict the various ways in which a system might react to change.

For the evaluation of One Health interventions as an example, we have adapted the system thinking techniques to visualize the main elements, stakeholders, constrains, and internal dynamics. Furthermore, defining the system boundaries allows exploring the needs and gaps of a system. Once these elements are established, the system can be studied in a prospective and retrospective way, to optimize it or maximize benefits from interventions on it.

In this study, we propose a framework to describe and delimit One Health initiatives using ALERT as an example project as a first step toward evaluating them as complex adaptive systems.

Our framework will be a further instrument to be used alone or combined and used in synergies with other already existing frameworks. The right choice of a specific framework or a combination of different frameworks for analyzing different types of complex systems will have to be addressed in a separate effort and it will require specific expertise (in particular, socioeconomic background) that will collaborate with us in the future work.

Materials and Methods

Preliminary Considerations about Systems and Their Boundaries

Whitehead et al. (17) and Gibson et al. (19) define a system as “a set of elements so interconnected as to aid in driving toward a defined goal.” A system might include subsystems or a collection of systems. In other words, abstractions about systems and their constituent components can go to very high and very low levels of detail [intending level as a position in a scale or rank (20)] depending on one’s perspective and the purpose of the abstractions. The lack of specificity in defining what is a system vs. a subsystem or system component or element is one reason why all relevant stakeholders should be involved in defining the structure of a system. In public health, “systems are dynamic architectures of interactions and synergies,” where the elements of the system are also coming from social science (3). When working with such complex systems, the meaning of multiple perspectives, interactions, and boundaries must be understood, because each element can be potentially essential for identifying successful interventions. While the perspectives are determined by the stakeholders, these are at the same time players in the system and may become agents of change (3, 21).

Any observation, intervention or evaluation, faces the dilemma between focus and comprehensiveness, and to become operational, the system of interest must have an operational space that defines its limitations (21). Nevertheless, the environment of a system is important and should be well described to generate awareness of the wider context and to avoid missing potential external interactions. Real systems are dynamic and even geophysical boundaries change over time (22). We consider the dynamic of the system adopting an iterative process. If there are modifications in the inner scenario, we consider the modified scenario as new inputs and iteratively integrated it to the new analysis, as described by Gibson et al. (19).

Elements of the System Definition Framework

In the next section, we define the elements contributing directly or indirectly to the system and system boundaries, i.e., the network of connected interactions that temporally close it, represent limits, and contribute to the overall structure of the system. We have then combined these elements to create the framework for defining and analyzing the structure and boundaries of a system and applied this framework on a health intervention project in the Section “Results.”

The Aim of a System

The aim should provide an answer to the question “Why are we looking at this system? Which are the problems, questions to solve?” The aim should help to investigate the way a system is used to solve a problem.

In the framework, we differentiate among the declared aim by the system and the observed, enacted, as well as the perceived aims. Each stakeholder may have a different perception of the declared aim and again each of them can have a different way to interpret how the system is performing in relation to its aim (23).

The System Space and Time, and Scale of Analysis

Space

This element identifies how the system extends geospatially, what is the geophysical environment, how large it is, and which ethno-political entities are involved (region, state, and nation). It also defines the scale of analysis that is of primary interest, individuals, households, groups or populations, etc., and finally how the different stakeholders are influenced by the spatial conditions.

Time

This element defines at which primary time scale is the system being observed, such as seconds, days, weeks, months, years, etc., and how the stakeholders are influenced by this time scale.

Interactions with Space and Time

This element defines the involvement of iterations and pathways along space and time dimensions.

Stakeholders and Actors

Stakeholders are entities affecting or affected by the system; they can be entities of different size between individuals, families, institutions, government agencies, etc. (24). More specifically, we can define primary actors as stakeholders who act on the system and secondary actors that can still partially interact with and modify the system. Examples of stakeholders of a health intervention system are farmers at the beginning of the milk chain, veterinarians checking the animals, dairies, food industry, toxicologists, chemists, veterinarians, biologists, and agronomists part of a research institute or a control institution.

In the framework, the information how the actors and stakeholders influence or are influenced by the system is specifically required.

The Systems Restrictions/Conditions—Boundaries

Which are the restrictions, conditions, and boundaries associated with a system? For example, a limited production due to regulations (e.g., the old milk quotas system in Europe), a closed market for a food due to regulations (e.g., raw milk consumption), or a cultural behavior that will limit a certain system to a group of individuals or animals (e.g., bovine milk consumption in certain Asian communities), relations of control among stakeholders and actors and relevant legal requirements, or constraints imposed by daily food production and market (e.g., quality systems must comply with production time), financial capability of primary food producers (e.g., calling for public incentives) or by sustainability aspects [e.g., the impact of the global milk production on the environment at the global level, i.e., in relation to the planetary boundaries (22)]. Such elements need to be potentially considered when defining the boundaries of a system in line with the system aim and goals. As defined by Senge (25), the description of system boundaries “considers what is improved, affected, or replaced by the system and, conversely, what affects the system under study, as the system changes and is changed by the environment.” Being able to well define the system space and its limitations helps in the definition of the system boundaries.

In the framework, constrains and boundaries are also related to the aim of the system. Understand how they interact with the system aims is important information to understand possible leverage points.

The Consequences (of Different Degree)

Consequences are the results of interactions in the system. They follow the path of interactions and stop at the system boundaries. Once the system and the system boundaries are defined, the consequences for that system are determined subsequently inside the system space. The “boundaries,” “externalities,” and “constraints” to the system should also be considered as surrounding and limiting the consequences to a certain degree (26). As an example, foods and food products are impacting directly human or animal health and indirectly (e.g., through the food chain or animal or human waste) ecosystem health.

In the framework, the consequences are related to the boundaries/constrains of the system, as described above.

The System Evolution

Following the work of Forrester (27): “there is not a single method, but an approach that uses a set of tools, to understand the behavior of complex systems over time designed to solve the problem of simultaneity (mutual causation).” Every real system is dynamically evolving, and this is why it is important to periodically reevaluate the system (interacting actors and stakeholders, restrictions, and consequences) and redefine it and its boundaries in an iterative way. The various processes of definition of the system and the problems that should be solved, the definition of a way to act/interact with a specific population/culture/disease/habit should be redefined periodically, because the system itself is constantly under change.

The ALERT Project

The ALERT project2 has been used as an example on how to apply the proposed framework. ALERT is funded by the Italian Ministry for Economic Development and is based on the transfer of technical innovation and technological know-how, which emerged from public research in the field, to the actors in daily food production (primary food producers and food industry). ALERT is coordinated by the Italian National Institute of Health (ISS) and develops a new risk management framework based on recent technological advances to manage the bovine milk chain for improved product safety and quality. It exploits new biomarkers for (i) early detection of production anomalies, (ii) monitoring (and assessment) of effects of corrective actions undertaken as risk management measure, and (iii) for assessment of production improvements (e.g., feed changes).

ALERT involves the human health affected by the environment, animal health and ecosystem health, sectors, as well as the producing farmer community and their web of interactions, and other actors of the food chain up to the food industry and the consumers. It is transdisciplinary and multisectoral to provide space for innovation and harvest the benefits of such integrated approaches and can thus be seen as a One Health project. It further focuses the responsibility of primary producers in the European food safety framework (28). Milk is a food particularly interesting for One Health: it is an animal product, highly susceptible to toxic contaminants (29, 30), highly consumed by vulnerable consumers, and suited as sentinel matrix for environmental monitoring purposes (30). The description of the project system may also be of interest to primary productions in economically developing countries, where environmental conditions and restricted resources amplify both risks of contaminations and challenges for their prevention (29).

In Table Table1,1, the ALERT response to the primary producers’ mandate for legal, scientific, and ethical responsibility in the EU food safety framework is described.

Table 1
ALERT project response to develop and translate invention (BEST) into practical innovation for One Health-related needs.

ALERT Institutional Framework

ALERT requires the establishment of an institutional setting matching different silos like public vs. private bodies; public health vs. fundamental research; food industry vs. high-tech industry; risk analysis vs. marketable technologies; scientists vs. food producers; and scientists vs. citizens/consumers.

ALERT relies on the integration between three clusters or pool of actors:

  • (a)
    A “Risk Management Cluster” including:
    • Istituto Superiore di Sanità (ISS), i.e., the Italian National Institute of Health, is the leading technical-scientific body of the Italian National Health Service, with top-level expertise in risk analysis (from risk assessment to formulation of scientific options for risk management) of food chains (Department of Food Safety, Nutrition and Veterinary Public Health), public health (Department of Environment and Health), prevention of non-communicable diseases and relevant technologies development (Department of Cardiovascular, Dysmetabolic and Aging-Associated Diseases). Disciplines/expertise deployed: anthropologists, biologists, chemists, engineers, statisticians, toxicologists, and veterinarians.
    • Istituto Zooprofilattico Sperimentale of regions Lazio and Toscana, i.e., public institute with top-level expertise and governmental commitment in the protection of food chain wholesomeness and animal health and welfare in the network of IZS regional institutes. Disciplines/expertise deployed: agronomists, biologists, chemists, and zootechnicians.
    • Lattepiù, i.e., leading enterprises in milk production, including the Pascolini Elio dairy farm, and milk transport and storage system. Disciplines/expertise deployed: farmers and livestock staff.
    • Centrale del Latte di Roma (CLR), i.e., leading regional milk industry. Disciplines/expertise deployed: biologists and chemists.
    Lattepiù and CLR are also the final end users of the ALERT products.
  • (b)
    A “Technology Cluster” including:
    • Consiglio Nazionale delle Ricerche (CNR, Institute on nano-structured materials), i.e., public institute with top-level expertise/commitment in technological innovation, research and technology transfer. Disciplines/expertise deployed: biologists, chemists, and biotechnologists.
    • Amel, Biosensor, Nutriservice, i.e., enterprises with complementary expertise in the setting, optimization, miniaturization, and robotization of (bio) probes systems as well as the development of management software and electronic systems. Disciplines/expertise deployed: biotechnologists, electronic engineers, and software developers.
  • (c)
    A “Marketing Cluster” involving different expertise in marketing strategies, strategic partnering at industrial level, as well as dissemination of pre-industrial research deliverables of the Leonardo Business Consulting. Disciplines/expertise deployed: economists, marketing managers, and business developers.

Indeed, the primary producers’ role in the EU food safety frame calls for transdisciplinary work as it is shown in Figure Figure22.

Figure 2
Interdisciplinary team in ALERT.
Figure 1
Institutional framework governing the ALERT project.

Results

The proposed framework (Table (Table2)2) provides an overview on elements and relationships of the system in which ALERT operates (Table (Table3).3). By employing a structured process to define system elements and boundaries, the system representation is developed from the various stakeholders’ perspectives and sets the system boundaries. In analyzing the three aspects of contextualization, relationships and evolution, one gains an understanding of static and dynamic properties of the system.

Table 2
Framework for identifying a system and its boundaries.
Table 3
The system and system boundaries applied to the ALERT project as an example.

System Identification Framework

A framework of the main definition steps to identify a system and its boundaries is summarized in Table Table22 (below).

The framework is divided into three questions. Main questions are the ones that help to contextualize the system; secondary questions help to define the relationships among actions and finally the tertiary questions concern the evolution of the system.

These questions are made for each of the element described in the Section “Materials and Methods,” namely: (1) aim of the system, (2) system dimension (space and time), (3) actors–stakeholders, (4) restrictions/conditions/boundaries, and (5) consequences.

Application of System and System Boundaries Framework to ALERT

The system and system boundaries framework is applied to the example ALERT to identify its gaps and weaknesses and relevant possible solutions (Table (Table33).

Using the framework, we can evidence some main arguments, highlighted in bold in Table Table33 and summarized in Box 1. For clarity, we have associated a letter to each highlighted point in Table Table33 and reported in Box 1.

Box 1

Main results after the application of system and system boundaries framework to ALERT.

  • The aim of the system is not interpreted in the same way by different stakeholders (a, d)
  • Stakeholders from different backgrounds and disciplines miss a harmonized collaboration to fulfill the aims of the project in the most productive way (b, d)
  • It is shown from the space and dimension elements that the system has potential to be applied at a larger scale (e.g., other regions and nationwide) (c)
  • Along the restrictions/constrictions/system boundaries, two main aspects (market-driven or daily tasks vs. importance of acquiring new knowledge through non-traditional approaches, and missing policies facilitating proactiveness at the farm) lead to the evolution scenario of weaknesses in synergizing activities between partners (d)
  • The consequences follow and detail the fact that the project aims to give more responsibility to the primary food producer and the different stakeholders approach this task with limited synergies (e)
  • The evolution of the system depicts needs and constrains in the future: namely, short-term/long-term effects along the awareness of toxicological risk/One Health approach in the food chain (g)

Discussion and Conclusion

Using a structured framework that defines the system, its stakeholders, its boundaries, and its evolution helps in showing the situation as it is conceived, the stakeholder roles, the relationship, and recurrence of the system components. Furthermore, in line with a system thinking approach, we can observe the “leverage” points, described in the Section “Results,” that can modify the system evolution, for example, improving its performances, along a time line.

This type of project system and project system boundary analyses helps to have an overview on the project, without missing possible important connections within aim, expertise, business, and development factors (36).

To be able to improve the performance of a project, it is often required to not get “unwanted surprises” about its behavior, so to be able to follow almost predictable results, in line to the aim of the project. Considering that the system properties and behaviors are per se unpredictable along the project evolution, it is important to be able to understand why there may be a divergence from the wanted aims and the actual proceeding of the project.

Having such a project system description framework helping to connect the various elements of the project, also the little ramifications of a network of interactions among activities/performances/roles/results, gives the possibility to interpret the aim of the project in its real evolution and dynamic, as shown in Table Table3.3. The strategy to defragment a project into such a framework may lead to the possible uses of the framework reassumed in Box 2.

Box 2

Summary of possible use of the framework.

  • To find possible representative information from the network of connections among aims, causes, consequences, and results of a project.
  • To prove the role of the different actors along the time line of the project, causes and consequences of their behaviors, and their points of view/background. This will help in getting a meta-perspective to evaluate a project. A further development of this basic framework can be to ask different stakeholders to apply the framework to the same project. Because their perspectives differ, such an analysis would further highlight synergies and antagonisms allowing for improvement.
  • To monitor progress along the project implementation phase (e.g., first and second year).
  • To compare and evaluate projects’ impacts and to measure their progress and compliance with the aim.

Different points are evidenced in the system and system boundaries analyses for ALERT example, in particular from Table Table33 and Figure Figure22.

The aim of the project “to support primary producers” is maintained along the description of the system in the framework but many elements are competing with the main aim (see Table Table33).

It is worth mentioning how the core aspect of the ALERT project, i.e., the emerging role of toxicological risk in the onset of diseases in the context of One Health (33, 34), is both the key and the “problematic” aspect of ALERT as showed in Box 1.

The increasing movement worldwide on the need of primary prevention measures to protect communities from non-communicable diseases as well as of sustainable food safety policies for primary prevention of transgenerational risks in the food chains (33, 35) is expected to modify the system in the medium term.

Indeed, the increasing consumers’ awareness of food safety long-term impact on health implies a growing demand of safer and safer products, along with the protection of the environment. In this context, policies facilitating primary food producers in their proactive roles are crucial (37).

The integrated analysis of the system and its boundaries including stakeholders, etc. highlights the importance of the collaboration within different disciplines, stakeholders, methodologies, expertise, etc. To allow the smooth project implementation taking into consideration the evidenced needed for interaction/collaboration/links among stakeholders and disciplines, governance mechanisms should be defined. However, integrating health aspects of humans, animals, and the environment is often not sufficient to identify relevant trade-offs, potential burden shifting, and undesired consequences in a system change, e.g., via an intervention. To address this issue, a more comprehensive approach is required where in addition to health- and risk management-related aspects also sustainability related aspects are considered along the entire system life cycle. Combining risk and sustainability aspects in a consistent manner to provide a more reliable decision support of health intervention and various other systems is proposed by the Global Decision Support Initiative (GDSI3).

In conclusion, a system view of a complex project and the definition of its boundaries help in understanding the way how to structure and optimize a system and actions (e.g., health interventions) within that system: how to integrate different expertise, meet different needs, mission and vision, and improve communications. This framework is a good way to have a concise, common overview of the most important elements that run a system now and in the future, considering the several externalities and impacts generated, but also to identify current system limitations.

Author Contributions

All the authors contributed to the design of the work, to the acquisition and interpretation of data for the work, and to revise it critically and all the authors approved the final version to be published.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This article is based upon work from the European Cooperation in Science and Technology (COST) Action TD1404 “Network for Evaluation of One Health”, supported by COST. Authors also acknowledge the ALERT project (http://www.alert2015.it).

Funding. The publishing fees were covered by the COST Action TD 1404.

References

2. Schelling E, Zinsstag J. Transdisciplinary research and One Health. In: Zinsstag J, Schelling E, Waltner-Toews D, Whittaker M, Tanner M, editors. , editors. One Health: The Theory and Practice of Integrated Health Approaches. Oxfordshire: CABI; (2015). p. 366–73. Available from: www.cabi.org
3. WHO Anonymous. In: de Savigny D, Taghreed A, editors. , editors. Systems Thinking For Health Systems Strengthening. Geneva: (2009). Available from: http://apps.who.int/iris/bitstream/10665/44204/1/9789241563895_eng.pdf
4. Ledford BYH. Team science. Nature (2015) 525:308–11.10.1038/525308a [PubMed] [Cross Ref]
5. Whitmee S, Haines A, Beyrer C, Boltz F, Capon AG, de Souza Dias BF, et al. Safeguarding human health in the Anthropocene epoch: report of The Rockefeller Foundation–Lancet Commission on planetary health. Lancet (2015) 6736:1–56.10.1016/S0140-6736(15)60901-1 [PubMed] [Cross Ref]
6. Zinsstag J. Convergence of ecohealth and one health. Ecohealth (2012) 9:371–3.10.1007/s10393-013-0812-z [PMC free article] [PubMed] [Cross Ref]
7. Choi BCK, Pak AWP. Multidisciplinarity, interdisciplinarity and transdisciplinarity in health research and policy: 1. Definitions, objectives, and evidence of effectiveness. Clin Investig Med (2006) 29:351–64.10.1002/eji.201090065 [PubMed] [Cross Ref]
8. Kahn LH. The one-health way. Nat Perspect (2017) 543:S47.10.1038/543S47a [PubMed] [Cross Ref]
9. Rüegg SR, McMahon BJ, Häsler B, Esposito R, Rosenbaum Nielsen L, Ifejika Speranza C, et al. A blueprint to evaluate One Health. Front Public Health (2017) 5:20.10.3389/fpubh.2017.00020 [PMC free article] [PubMed] [Cross Ref]
10. Briggs DJ.. A framework for integrated environmental health impact assessment of systemic risks. Environ Health (2008) 7:61.10.1186/1476-069X-7-61 [PMC free article] [PubMed] [Cross Ref]
11. Bresalier M, Cassidy A, Woods A. One Health in history. In: Zinsstag J, Schelling E, Waltner-Toews D, Whittaker M, Tanner M, editors. , editors. One Health: The Theory and Practice of Integrated Health Approaches. Oxfordshire: CABI; (2015). p. 1–15.
12. Baum SE, Machalaba C, Daszak P, Salerno RH, Karesh WB.. Evaluating one health: are we demonstrating effectiveness? One Health (2016) 3:5–10.10.1016/j.onehlt.2016.10.004 [PMC free article] [PubMed] [Cross Ref]
13. Sukhdev P, May P, Müller A. Fix food metrics. Nature (2016) 540:33–4.10.1038/540033a [PubMed] [Cross Ref]
14. Kernick D.. The demise of linearity in managing health services: a call for post normal health care. J Health Serv Res Policy (2002) 7(2):121–4.10.1258/1355819021927782 [PubMed] [Cross Ref]
15. Resnicow K, Page SE.. Embracing chaos and complexity: a quantum change for public health. Am J Public Health (2008) 98(8):1382–9.10.2105/AJPH.2007.129460 [PubMed] [Cross Ref]
16. Thygeson M, Morrissey L, Ulstad V.. Adaptive leadership and the practice of medicine: a complexity-based approach to reframing the doctor-patient relationship. J Eval Clin Pract (2010) 16(5):1009–15.10.1111/j.1365-2753.2010.01533.x [PubMed] [Cross Ref]
17. Whitehead NP, Scherer WT, Smith MC. Systems thinking about systems thinking. IEEE Syst J (2015) 9:1117–28.10.1109/JSYST.2014.2332494 [Cross Ref]
18. Williams B. Using Systems Concepts in Evaluation Design: A Workbook. 1st ed Kyoto: (2016). Available from: http://www.bobwilliams.co.nz
19. Gibson JE, Scherer WT, Gibson WF. Introduction. In: How to Do Systems Analysis. Hoboken, NJ, USA: John Wiley & Sons, Inc. (2007). p. 1–28.
20. Merriam Webster Dictionary. Available from: www.Merriam-Webster.com
21. O’Conner J, McDermott I. The Art of Systems Institute. Thinking: Essential Skills for Creativity and Problem-Solving. San Francisco, CA: Thorsons Publishing; (1997).
22. Steffen W, Richardson K, Rockström J, Cornell SE, Fetzer I, Bennett EM, et al. Sustainability. Planetary boundaries: guiding human development on a changing planet. Science (2015) 347:736–46.10.1126/science.1259855 [PubMed] [Cross Ref]
23. Anderson V, Johnson L. Systems Thinking Basics: From Concepts to Causal Loops. Acton, USA: Leverage Networks Inc; (1997). 131 p.
24. Ostrom E. A general framework for analyzing sustainability of social-ecological systems. Science (2009) 325:419–22.10.1126/science.1172133 [PubMed] [Cross Ref]
25. Senge P. The Fifth Discipline. NY: Currency/Doubleday; (1990).
26. Peters DH. The application of systems thinking in health: why use systems thinking? Health Res Policy Syst (2014) 12:51.10.1186/1478-4505-12-51 [PMC free article] [PubMed] [Cross Ref]
27. Forrester JW. Principles of Systems. 2nd ed Portland, Oregon: Productivity Press; (1968).
28. Regulation (EC) No 852/2004. Regulation (EC) No 852/2004 of the European Parliament and of the Council of 29 April 2004 on the Hygiene of Foodstuffs (2004). Available from: http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2004:139:0001:0054:en:PDF
29. Cheng R, Mantovani A, Frazzoli C. Analysis of food safety and security challenges in emerging African food producing areas through a One Health lens: the dairy chains in Mali. J Food Prot (2017) 80:57–67.10.4315/0362-028X.JFP-15-561 [PubMed] [Cross Ref]
30. Frazzoli C, Petrini C, Mantovani A. Sustainable development and next generation’s health: a long-term perspective about the consequences of today’s activities for food safety. Annali dell’Istituto Superiore di Sanità (2009) 45:65–75. [PubMed]
31. Frazzoli C, Mantovani A, Dragone R. Il brevetto “BEST” per la gestione precoce del rischio in sanità pubblica veterinaria e sicurezza alimentare: l’incontro della ricerca pubblica con l’impresa. Notiziario dell’Istituto Superiore di Sanità (2012) 25(1):3–5.
32. Commission of the European Communities (CEC). White Paper on Food Safety, Brussels, 12 January 2000, COM (1999) 719 Final. (2000).
33. Frazzoli C, Mantovani A, Esposito R. Sustainable food safety and toxicant zoonoses: new prevention challenges in global health governance. Quaderni della Società Italiana di Medicina Tropicale e Salute Globale (2016) 1:117–27.
34. Frazzoli C, Mantovani A. Toxicants exposures as novel zoonoses: reflections on sustainable development, food safety and veterinary public health. Zoonoses Public Health (2010) 57:e136–42.10.1111/j.1863-2378.2009.01309.x [PubMed] [Cross Ref]
35. Frazzoli C, Mantovani A, Campanella L, Dragone R. Technological Integrated Bioelectronic System and Relevant Control Charting for Early Intervention On Food Chain and the Environment [BEST]. Roma: Owner: Istituto Superiore di Sanità (2009). PCT/2009/000293.
37. Frazzoli C, Mantovani A, Dragone R. Local role of food producers’ communities for a Global One-Health framework: the experience of translational research in an Italian dairy chain. J Agric Chem Environ (2014) 3(2B):14–9.10.4236/jacen.2014.32B003 [Cross Ref]

Articles from Frontiers in Public Health are provided here courtesy of Frontiers Media SA