PMCC PMCC

Search tips
Search criteria

Advanced
Results 1-13 (13)
 

Clipboard (0)
None

Select a Filter Below

Journals
Year of Publication
Document Types
1.  The global stock of research evidence relevant to health systems policymaking 
Background
Policymakers and stakeholders need immediate access to many types of research evidence to make informed decisions about the full range of questions that may arise regarding health systems.
Methods
We examined all types of research evidence about governance, financial and delivery arrangements, and implementation strategies within health systems contained in Health Systems Evidence (HSE) (http://www.healthsystemsevidence.org). The research evidence types include evidence briefs for policy, overviews of systematic reviews, systematic reviews of effects, systematic reviews addressing other questions, systematic reviews in progress, systematic reviews being planned, economic evaluations, and health reform and health system descriptions. Specifically, we describe their distribution across health system topics and domains, trends in their production over time, availability of supplemental content in various languages, and the extent to which they focus on low- and middle-income countries (LMICs), as well as (for systematic reviews) their methodological quality and the availability of user-friendly summaries.
Results
As of July 2013, HSE contained 2,629 systematic reviews of effects (of which 501 are Cochrane reviews), 614 systematic reviews addressing other questions, 283 systematic reviews in progress, 186 systematic reviews being planned, 140 review-derived products (evidence briefs and overviews of systematic reviews), 1,669 economic evaluations, 1,092 health reform descriptions, and 209 health system descriptions. Most systematic reviews address topics related to delivery arrangements (n = 2,663) or implementation strategies (n = 1,653) with far fewer addressing financial (n = 241) or governance arrangements (n = 231). In addition, 2,928 systematic reviews have been quality appraised with moderate AMSTAR ratings found for reviews addressing governance (5.6/11), financial (5.9/11), and delivery (6.3/11) arrangements and implementation strategies (6.5/11); 1,075 systematic reviews have no independently produced user-friendly summary and only 737 systematic reviews have an LMIC focus. Literature searches for half of the systematic reviews (n = 1,584, 49%) were conducted within the last five years.
Conclusions
Greater effort needs to focus on assessing whether the current distribution of systematic reviews corresponds to policymakers’ and stakeholders’ priorities, updating systematic reviews, increasing the quality of systematic reviews, and focusing on LMICs.
doi:10.1186/1478-4505-11-32
PMCID: PMC3847996  PMID: 24007378
2.  Community-based organizations in the health sector: A scoping review 
Community-based organizations are important health system stakeholders as they provide numerous, often highly valued programs and services to the members of their community. However, community-based organizations are described using diverse terminology and concepts from across a range of disciplines. To better understand the literature related to community-based organizations in the health sector (i.e., those working in health systems or more broadly to address population or public health issues), we conducted a scoping review by using an iterative process to identify existing literature, conceptually map it, and identify gaps and areas for future inquiry.
We searched 18 databases and conducted citation searches using 15 articles to identify relevant literature. All search results were reviewed in duplicate and were included if they addressed the key characteristics of community-based organizations or networks of community-based organizations. We then coded all included articles based on the country focus, type of literature, source of literature, academic discipline, disease sector, terminology used to describe organizations and topics discussed.
We identified 186 articles addressing topics related to the key characteristics of community-based organizations and/or networks of community-based organizations. The literature is largely focused on high-income countries and on mental health and addictions, HIV/AIDS or general/unspecified populations. A large number of different terms have been used in the literature to describe community-based organizations and the literature addresses a range of topics about them (mandate, structure, revenue sources and type and skills or skill mix of staff), the involvement of community members in organizations, how organizations contribute to community organizing and development and how they function in networks with each other and with government (e.g., in policy networks).
Given the range of terms used to describe community-based organizations, this scoping review can be used to further map their meanings/definitions to develop a more comprehensive typology and understanding of community-based organizations. This information can be used in further investigations about the ways in which community-based organizations can be engaged in health system decision-making and the mechanisms available for facilitating or supporting their engagement.
doi:10.1186/1478-4505-10-36
PMCID: PMC3511187  PMID: 23171160
3.  Treatment, care and support for people co-infected with HIV and hepatitis C: a scoping review 
Open Medicine  2009;3(4):e184-e195.
Background
Providing care for people who are co-infected with both HIV and hepatitis C virus (HCV) is becoming increasingly complex and requires integrated prevention, screening, support and programming efforts. We undertook a scoping review to provide a summary of the existing evidence base and to identify and assess the quality of treatment guidelines and systematic reviews related to 3 domains of interest: treatment; epidemiology; and care, support, programming and prevention.
Methods
We searched 7 databases, hand-searched 8 journals and contacted key informants to identify relevant literature. We included all primary research (including systematic reviews and meta-analyses) or treatment guidelines that assessed pegylated interferon and ribavirin for HCV or highly active antiretroviral therapy for HIV treatment, or both. In the epidemiology domain, we included all primary research (including systematic reviews and meta-analyses). Studies that included only people with hemophilia and those conducted in developing countries were excluded. In the care, support, programming and prevention domain, we included all studies and reports that focused on co-infection. Two reviewers independently applied coding criteria and assessed the quality of the treatment guidelines and systematic reviews using the Appraisal of Guidelines Research and Evaluation and A MeaSurement Tool to Assess Reviews instruments.
Results
Our search strategy yielded 1633 unique references. Of these, 227 references met the final inclusion criteria: 114 addressed treatment, 52 epidemiology and 79 care, support, programming or prevention. The references included 9 treatment guidelines: 4 were assessed as “strongly recommend,” 3 as “recommend (with provisos or alterations)” and 1 as “would not recommend” (1 could not be located). Of 10 systematic reviews that were located, 7 were assessed as being high quality, 2 as medium quality and 1 as low quality.
Conclusion
This quality-assessed inventory of treatment guidelines and systematic reviews can be used by physicians and service providers to rapidly locate research about HIV–HCV co-infection. However, many treatment guidelines and reviews often indicate that treatment of current injection drug users and/or people with mental health issues should proceed on a “case-by-case basis.” Therefore, much of the evidence (particularly in the treatment literature) is limited in its scope and applicability to important populations that are vulnerable to HIV or HCV infection or co-infection.
PMCID: PMC3090111  PMID: 21688755
4.  Counselling, Case Management and Health Promotion for People Living with HIV/AIDS: An Overview of Systematic Reviews 
AIDS and Behavior  2012;17(5):1612-1625.
Our objective was to identify all existing systematic reviews related to counselling, case management and health promotion for people living with HIV/AIDS. For the reviews identified, we assessed the quality and local applicability to support evidence-informed policy and practice. We searched 12 electronic databases and two reviewers independently assessed the 5,398 references retrieved from our searches and included 18 systematic reviews. Each review was categorized according to the topic(s) addressed, quality appraised and summarized by extracting key messages, the year searches were last completed and the countries in which included studies were conducted. Twelve reviews address topics related to counselling and case management (mean quality score of 6.5/11). Eight reviews (mean quality score of 6/11) address topics related to health promotion (two address both domains). The findings from this overview of systematic reviews provide a useful resource for supporting the development and delivery of evidence-informed support services in community settings.
Electronic supplementary material
The online version of this article (doi:10.1007/s10461-012-0283-1) contains supplementary material, which is available to authorized users.
doi:10.1007/s10461-012-0283-1
PMCID: PMC3663251  PMID: 22961581
HIV; Systematic review; Case management; Health promotion; Counselling
5.  Autologous Bone Marrow Mononuclear Cell Therapy is Safe and Promotes Amputation Free Survival in Patients with Critical Limb Ischemia 
Objective
The purpose of this phase I open label non-randomized trial was to assess the safety and efficacy of autologous bone marrow mononuclear cell (ABMNC) therapy in promoting amputation free survival (AFS) in patients with critical limb ischemia (CLI).
Methods
Between September 2005 and March 2009 twenty-nine patients (30 limbs), with a median age of 66 (range 23–84) (14 male,15 female) with CLI were enrolled . Twentyone limbs presented with rest pain (RP), six with RP and ulceration, and three with ulcer only. All patients were not candidates for surgical bypass due to absence of a patent artery below the knee and/or endovascular approaches to improving perfusion was not possible as determined by an independent vascular surgeon. Patients were treated with an average dose of 1.7 ± 0.7 × 109 ABMNC injected intramuscularly in the index limb distal to the anterior tibial tuberosity. The primary safety endpoint was accumulation of serious adverse events and the primary efficacy endpoint was AFS at one year. Secondary endpoints at 12 weeks post-treatment were changes in first toe pressure (FTP), toe-brachial index (TBI), ankle-brachial index (ABI), and transcutaneous oxygen measurements (TcPO2). Perfusion of the index limb was measured with PET-CT with intra-arterial infusion of H2O15. Rest pain (RP), using a 10-cm visual analog scale, quality of life using the VascuQuol questionnaire, and ulcer healing were assessed at each follow-up interval. Subpopulations of endothelial progenitor cells were quantified prior to ABMNC administration using immunocytochemistry and fluorescent activated cell sorting.
Results
There were two serious adverse events however there no procedure related deaths. Amputation-free survival at one-year was 86.3%. There was a significant increase in FTP (10.2+ 6.2 mmHg, P=.02) and TBI (0.10± 0.05, P=.02) and a trend in improvement in ABI (0.08±0.04, P=.73). Perfusion Index by PET-CT H2O15 increased by 19.3 ± 3.1 and RP decreased significantly by 2.2 ± 0.6 cm. (P=.02). The VascuQol questionnaire demonstrated significant improvement in QOL and three of 9 ulcers (33%) healed completely. KDR+ but not CD34+ or CD133+ subpopulations of ABMNC were associated with improvement in limb perfusion.
Conclusion
This phase I study has demonstrated safety and the AFS rates suggest efficacy of ABMNC in promoting limb salvage in “no option” CLI. Based on these results we plan to test the concept that ABMNCs improve AFS at one year in a phase III randomized, double-blinded multicenter trial.
doi:10.1016/j.jvs.2011.01.074
PMCID: PMC3102148  PMID: 21514773
6.  Evaluation of an ambulatory system for the quantification of cough frequency in patients with chronic obstructive pulmonary disease 
Background
To date, methods used to assess cough have been primarily subjective, and only broadly reflect the impact of chronic cough and/or chronic cough therapies on quality of life. Objective assessment of cough has been attempted, but early techniques were neither ambulatory nor feasible for long-term data collection. We evaluated a novel ambulatory cardio-respiratory monitoring system with an integrated unidirectional, contact microphone, and report here the results from a study of patients with COPD who were videotaped in a quasi-controlled environment for 24 continuous hours while wearing the ambulatory system.
Methods
Eight patients with a documented history of COPD with ten or more years of smoking (6 women; age 57.4 ± 11.8 yrs.; percent predicted FEV1 49.6 ± 13.7%) who complained of cough were evaluated in a clinical research unit equipped with video and sound recording capabilities. All patients wore the LifeShirt® system (LS) while undergoing simultaneous video (with sound) surveillance. Video data were visually inspected and annotated to indicate all cough events. Raw physiologic data records were visually inspected by technicians who remained blinded to the video data. Cough events from LS were analyzed quantitatively with a specialized software algorithm to identify cough. The output of the software algorithm was compared to video records on a per event basis in order to determine the validity of the LS system to detect cough in COPD patients.
Results
Video surveillance identified a total of 3,645 coughs, while LS identified 3,363 coughs during the same period. The median cough rate per patient was 21.3 coughs·hr-1 (Range: 10.1 cghs·hr-1 – 59.9 cghs·hr-1). The overall accuracy of the LS system was 99.0%. Overall sensitivity and specificity of LS, when compared to video surveillance, were 0.781 and 0.996, respectively, while positive- and negative-predictive values were 0.846 and 0.994. There was very good agreement between the LS system and video (kappa = 0.807).
Conclusion
The LS system demonstrated a high level of accuracy and agreement when compared to video surveillance for the measurement of cough in patients with COPD.
doi:10.1186/1745-9974-1-3
PMCID: PMC1277005  PMID: 16270923
7.  Community capacity to acquire, assess, adapt, and apply research evidence: a survey of Ontario's HIV/AIDS sector 
Background
Community-based organizations (CBOs) are important stakeholders in health systems and are increasingly called upon to use research evidence to inform their advocacy, program planning, and service delivery. To better support CBOs to find and use research evidence, we sought to assess the capacity of CBOs in the HIV/AIDS sector to acquire, assess, adapt, and apply research evidence in their work.
Methods
We invited executive directors of HIV/AIDS CBOs in Ontario, Canada (n = 51) to complete the Canadian Health Services Research Foundation's "Is Research Working for You?" survey.
Findings
Based on responses from 25 organizations that collectively provide services to approximately 32,000 clients per year with 290 full-time equivalent staff, we found organizational capacity to acquire, assess, adapt, and apply research evidence to be low. CBO strengths include supporting a culture that rewards flexibility and quality improvement, exchanging information within their organization, and ensuring that their decision-making processes have a place for research. However, CBO Executive Directors indicated that they lacked the skills, time, resources, incentives, and links with experts to acquire research, assess its quality and reliability, and summarize it in a user-friendly way.
Conclusion
Given the limited capacity to find and use research evidence, we recommend a capacity-building strategy for HIV/AIDS CBOs that focuses on providing the tools, resources, and skills needed to more consistently acquire, assess, adapt, and apply research evidence. Such a strategy may be appropriate in other sectors and jurisdictions as well given that CBO Executive Directors in the HIV/AIDS sector in Ontario report low capacity despite being in the enviable position of having stable government infrastructure in place to support them, benefiting from long-standing investment in capacity building, and being part of an active provincial network. CBOs in other sectors and jurisdictions that have fewer supports may have comparable or lower capacity. Future research should examine a larger sample of CBO Executive Directors from a range of sectors and jurisdictions.
doi:10.1186/1748-5908-6-54
PMCID: PMC3123230  PMID: 21619682
8.  Effects of an evidence service on health-system policy makers' use of research evidence: A protocol for a randomised controlled trial 
Background
Health-system policy makers need timely access to synthesised research evidence to inform the policy-making process. No efforts to address this need have been evaluated using an experimental quantitative design. We developed an evidence service that draws inputs from Health Systems Evidence, which is a database of policy-relevant systematic reviews. The reviews have been (a) categorised by topic and type of review; (b) coded by the last year searches for studies were conducted and by the countries in which included studies were conducted; (c) rated for quality; and (d) linked to available user-friendly summaries, scientific abstracts, and full-text reports. Our goal is to evaluate whether a "full-serve" evidence service increases the use of synthesized research evidence by policy analysts and advisors in the Ontario Ministry of Health and Long-Term Care (MOHLTC) as compared to a "self-serve" evidence service.
Methods/design
We will conduct a two-arm randomized controlled trial (RCT), along with a follow-up qualitative process study in order to explore the findings in greater depth. For the RCT, all policy analysts and policy advisors (n = 168) in a single division of the MOHLTC will be invited to participate. Using a stratified randomized design, participants will be randomized to receive either the "full-serve" evidence service (database access, monthly e-mail alerts, and full-text article availability) or the "self-serve" evidence service (database access only). The trial duration will be ten months (two-month baseline period, six-month intervention period, and two month cross-over period). The primary outcome will be the mean number of site visits/month/user between baseline and the end of the intervention period. The secondary outcome will be participants' intention to use research evidence. For the qualitative study, 15 participants from each trial arm (n = 30) will be purposively sampled. One-on-one semi-structured interviews will be conducted by telephone on their views about and their experiences with the evidence service they received, how helpful it was in their work, why it was helpful (or not helpful), what aspects were most and least helpful and why, and recommendations for next steps.
Discussion
To our knowledge, this will be the first RCT to evaluate the effects of an evidence service specifically designed to support health-system policy makers in finding and using research evidence.
Trial registration
ClinicalTrials.gov: NCT01307228
doi:10.1186/1748-5908-6-51
PMCID: PMC3123565  PMID: 21619621
9.  Effects of an evidence service on community-based AIDS service organizations' use of research evidence: A protocol for a randomized controlled trial 
Background
To support the use of research evidence by community-based organizations (CBOs) we have developed 'Synthesized HIV/AIDS Research Evidence' (SHARE), which is an evidence service for those working in the HIV sector. SHARE consists of several components: an online searchable database of HIV-relevant systematic reviews (retrievable based on a taxonomy of topics related to HIV/AIDS and open text search); periodic email updates; access to user-friendly summaries; and peer relevance assessments. Our objective is to evaluate whether this 'full serve' evidence service increases the use of research evidence by CBOs as compared to a 'self-serve' evidence service.
Methods/design
We will conduct a two-arm randomized controlled trial (RCT), along with a follow-up qualitative process study to explore the findings in greater depth. All CBOs affiliated with Canadian AIDS Society (n = 120) will be invited to participate and will be randomized to receive either the 'full-serve' version of SHARE or the 'self-serve' version (a listing of relevant systematic reviews with links to records on PubMed and worksheets that help CBOs find and use research evidence) using a simple randomized design. All management and staff from each organization will be provided access to the version of SHARE that their organization is allocated to. The trial duration will be 10 months (two-month baseline period, six-month intervention period, and two month crossover period), the primary outcome measure will be the mean number of logins/month/organization (averaged across the number of users from each organization) between baseline and the end of the intervention period. The secondary outcome will be intention to use research evidence as measured by a survey administered to one key decision maker from each organization. For the qualitative study, one key organizational decision maker from 15 organizations in each trial arm (n = 30) will be purposively sampled. One-on-one semi-structured interviews will be conducted by telephone on their views about and their experiences with the evidence service they received, how helpful it was in their work, why it was helpful (or not helpful), what aspects were most and least helpful and why, and recommendations for next steps.
Discussion
To our knowledge, this will be the first RCT to evaluate the effects of an evidence service specifically designed to support CBOs in finding and using research evidence.
Trial registration
ClinicalTrials.gov: NCT01257724
doi:10.1186/1748-5908-6-52
PMCID: PMC3127774  PMID: 21619622
10.  Community-based knowledge transfer and exchange: Helping community-based organizations link research to action 
Background
Community-based organizations (CBOs) are important stakeholders in health systems and are increasingly called upon to use research evidence to inform their advocacy, program planning, and service delivery efforts. CBOs increasingly turn to community-based research (CBR) given its participatory focus and emphasis on linking research to action. In order to further facilitate the use of research evidence by CBOs, we have developed a strategy for community-based knowledge transfer and exchange (KTE) that helps CBOs more effectively link research evidence to action. We developed the strategy by: outlining the primary characteristics of CBOs and why they are important stakeholders in health systems; describing the concepts and methods for CBR and for KTE; comparing the efforts of CBR to link research evidence to action to those discussed in the KTE literature; and using the comparison to develop a framework for community-based KTE that builds on both the strengths of CBR and existing KTE frameworks.
Discussion
We find that CBR is particularly effective at fostering a climate for using research evidence and producing research evidence relevant to CBOs through community participation. However, CBOs are not always as engaged in activities to link research evidence to action on a larger scale or to evaluate these efforts. Therefore, our strategy for community-based KTE focuses on: an expanded model of 'linkage and exchange' (i.e., producers and users of researchers engaging in a process of asking and answering questions together); a greater emphasis on both producing and disseminating systematic reviews that address topics of interest to CBOs; developing a large-scale evidence service consisting of both 'push' efforts and efforts to facilitate 'pull' that highlight actionable messages from community relevant systematic reviews in a user-friendly way; and rigorous evaluations of efforts for linking research evidence to action.
Summary
Through this type of strategy, use of research evidence for CBO advocacy, program planning, and service delivery efforts can be better facilitated and continually refined through ongoing evaluations of its impact.
doi:10.1186/1748-5908-5-33
PMCID: PMC2873302  PMID: 20423486
11.  SUPPORT Tools for evidence-informed health Policymaking (STP) 4: Using research evidence to clarify a problem 
This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers.
Policymakers and those supporting them often find themselves in situations that spur them on to work out how best to define a problem. These situations may range from being asked an awkward or challenging question in the legislature, through to finding a problem highlighted on the front page of a newspaper. The motivations for policymakers wanting to clarify a problem are diverse. These may range from deciding whether to pay serious attention to a particular problem that others claim is important, through to wondering how to convince others to agree that a problem is important. Debates and struggles over how to define a problem are a critically important part of the policymaking process. The outcome of these debates and struggles will influence whether and, in part, how policymakers take action to address a problem. Efforts at problem clarification that are informed by an appreciation of concurrent developments are more likely to generate actions. These concurrent developments can relate to policy and programme options (e.g. the publication of a report demonstrating the effectiveness of a particular option) or to political events (e.g. the appointment of a new Minister of Health with a personal interest in a particular issue). In this article, we suggest questions that can be used to guide those involved in identifying a problem and characterising its features. These are: 1. What is the problem? 2. How did the problem come to attention and has this process influenced the prospect of it being addressed? 3. What indicators can be used, or collected, to establish the magnitude of the problem and to measure progress in addressing it? 4. What comparisons can be made to establish the magnitude of the problem and to measure progress in addressing it? 5. How can the problem be framed (or described) in a way that will motivate different groups?
doi:10.1186/1478-4505-7-S1-S4
PMCID: PMC3271831  PMID: 20018111
12.  SUPPORT Tools for evidence-informed health Policymaking (STP) 5: Using research evidence to frame options to address a problem 
This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers.
Policymakers and those supporting them may find themselves in one or more of the following three situations that will require them to characterise the costs and consequences of options to address a problem. These are: 1. A decision has already been taken and their role is to maximise the benefits of an option, minimise its harms, optimise the impacts achieved for the money spent, and (if there is substantial uncertainty about the likely costs and consequences of the option) to design a monitoring and evaluation plan, 2. A policymaking process is already underway and their role is to assess the options presented to them, or 3. A policymaking process has not yet begun and their role is therefore to identify options, characterise the costs and consequences of these options, and look for windows of opportunity in which to act. In situations like these, research evidence, particularly about benefits, harms, and costs, can help to inform whether an option can be considered viable. In this article, we suggest six questions that can be used to guide those involved in identifying policy and programme options to address a high-priority problem, and to characterise the costs and consequences of these options. These are: 1. Has an appropriate set of options been identified to address a problem? 2. What benefits are important to those who will be affected and which benefits are likely to be achieved with each option? 3. What harms are important to those who will be affected and which harms are likely to arise with each option? 4. What are the local costs of each option and is there local evidence about their cost-effectiveness? 5. What adaptations might be made to any given option and could they alter its benefits, harms and costs? 6. Which stakeholder views and experiences might influence an option's acceptability and its benefits, harms, and costs?
doi:10.1186/1478-4505-7-S1-S5
PMCID: PMC3271832  PMID: 20018112
13.  Social determinants of health associated with hepatitis C co-infection among people living with HIV: results from the Positive Spaces, Healthy Places study 
Open Medicine  2011;5(3):e132-133.
Background
Social determinants of health (SDOH) may influence the probability of people living with HIV also being infected with hepatitis C virus (HCV). We compared the SDOH of adults co-infected with HCV/HIV with that of HIV mono-infected adults to identify factors independently associated with HCV infection.
Methods
In this cross-sectional study, face-to-face interviews were conducted with 509 HIV-infected adults affiliated with or receiving services from community-based AIDS service organizations (CBAOs). The primary outcome measure was self-reported HCV infection status. Chi-square, Student’s t tests, and Wilcoxon rank-sum tests were performed to compare SDOH of HCV/HIV co-infected participants with that of HIV mono-infected participants. Multivariable hierarchical logistic regression was used to identify factors independently associated with HCV co-infection.
Results
Data on 482 (95 HCV/HIV co-infected and 387 HIV mono-infected) adults were analyzed. Compared with participants infected with HIV only, those who were co-infected with HIV and HCV were more likely to be heterosexual, Aboriginal, less educated and unemployed. They were more likely to have a low income, to not be receiving antiretroviral treatment, to live outside the Greater Toronto Area (GTA), to use/abuse substances, experience significant depression, and utilize addiction counselling and needle-exchange services. They also were more likely to report a history of homelessness and perceived housing-related discrimination and to have moved twice or more in the previous 12 months. Factors independently associated with HCV/HIV co-infection were history of incarceration (odds ratio [OR] 8.81, 95% CI 4.43–17.54), history of homelessness (OR 3.15, 95% CI 1.59–6.26), living outside of the GTA (OR 3.13, 95% CI 1.59–6.15), and using/abusing substances in the past 12 months (OR 2.05, 95% CI 1.07–3.91).
Conclusion
Differences in SDOH exist between HIV/HCV co-infected and HIV mono-infected adults. History of incarceration, history of homelessness, substance use, and living outside the GTA were independently associated with HCV/HIV co-infection. Interventions that reduce homelessness and incarceration may help prevent HCV infection in people living with HIV.
PMCID: PMC3205830  PMID: 22046224

Results 1-13 (13)