A range of methods will be utilized during the data collection phase. Document reviews, time series, surveys, in-depth interviews, exit interviews, focus group discussions, case record reviews, and structured observations will be used. These are described in greater detail in the section below.
Data collection methods
1. Provincial (policy) level:
At this level we will collect secondary data on program related inputs, outputs and outcomes in each province. A time series analysis, consultations with key stakeholders, and review of relevant program documents will allow us to study program trends (objective i) and comment on the uptake of the programs since inception. A complete time series will be derived from maternal health data with regard to uptake (proportion of eligible women who participate), institutional delivery (total number of women delivering in a public or private facility), district level ante-natal care (ANC) coverage (proportion of women completing >3 ANC visits), and maternal and neonatal deaths. The data will be obtained from multiple sources including the District Level Household Survey (DLHS), Annual Health Survey, Sample Registration System (SRS), Vital Registration and health department sources for the purpose of data triangulation and the development of robust estimates. A major challenge in the analysis is the lack of complete time series for the maternal mortality data. Instead of relying on a single source for information, statistical data fusion techniques will be applied to collate data from various sources. The data fusion techniques will borrow strengths across time and space from the existing data to derive a complete time series estimates for the variables of interest.
A second study at the policy level will focus on trends in private sector participation (objective ii) in each of the programs over time. The role of private providers varies in each program. Private providers played a small part in the JSY program; however they are a large resource in Gujarat and thus the central focus in CY. Private facility characteristics will be studied through a cross-sectional facility review (see objective v). In MP, we will explore with NRHM program managers the experiences of collaborating with private providers and the perception of the role of the private sector in JSY, justification for criteria used for selection, and rationale for their inclusion in and subsequent exclusion from the program through in-depth interviews. Implementation issues and the overall benefits or disadvantages of participating in the program will also be explored with a selected group of private providers.
In Gujarat, we will use qualitative methods to explore participation of the for-profit health sector in CY and understand the factors influencing private provider participation. The qualitative research will be conducted using an interpretive approach, and will seek to understand private providers motives for participation/non participation, appropriateness of payment amounts under the program, procedures for reimbursement and interfacing with government officials. In addition, the study will explore how the program affects/affected patient turnover, what proportion of their patients are/were CY, influence on human resources, changes made to the facility to incorporate the program and suggestions on how to improve the functionality of the program. An experienced researcher will interview a purposive sample of private providers who (i) are currently participating in the program, (ii) were part of the program but no longer participate and (iii) never participated. Thematic analysis using the framework approach will be applied.
Difficulties accessing transportation to receive emergency obstetric care is one of the numerous barriers experienced by many rural women [11
], and we plan to explore the emergency transportation alternatives for obstetric care (objective iii) in each of the programs using quantitative and qualitative methods. The two programs use different emergency transportation models both of which are public private partnerships initiated by their respective governments. The ‘108’ emergency transport scheme in Gujarat was launched in 2004. ‘108’ is a centralized, free, fully integrated emergency service ambulance that is equipped with trained staff and equipment to administer lifesaving care en route. It is a common transportation means used for all medical emergencies and disaster situations. Janani Express Yojana (JEY or Maternal Express Program), piloted in 2006 and scaled up in 2008, is utilized in Madhya Pradesh. It is also a free ambulance service; however it is used exclusively to transport obstetric cases to public facilities. It is a more basic form of transportation, and does not provide amenities to treat patients en route to the hospital. JEY is a decentralized service; vehicles are hired on a contractual basis locally at the district level and utilize a decentralized call center.
A quantitative retrospective data analysis will be completed with data ascertained from call centers to determine utilization patterns and uptake of the respective transport systems. Socio-demographic characteristics of the beneficiaries and other outcomes will be collected in a cross-sectional questionnaire to study equity in use of the service.
NRHM program officials and District Chief Medical & Health Officers (CMHOs) will be interviewed to gain insight into the structure under each model and the benefits and limitations of the respective decentralized and centralized emergency transportation services. The discussion will include perceptions on efficiency, geographical accessibility, functionality, sustainability, technical capacity, constraints and motivations of the human resources, and procedural limitations. In addition, costs associated with implementation and ongoing usage will be elicited.
All interviews will be conducted in Hindi by a research assistant, transcribed and then translated into English. In-depth interview data will be analyzed thematically, using the framework approach. The analysis will draw out differences between the two schemes in terms of implementation, stakeholder views of the schemes and experiences of utilization. A cost effectiveness analysis between the two programs is also planned.
A cost effectiveness of the two programs (objective iv) will be performed through two different methods. The first approach will be to compare the costs and outcomes of the two programs for matched groups of participants. The primary outcome will be the proportion of institutional delivery. Other outcomes to be analyzed will be the proportion of cesarean deliveries, antenatal check-ups and delivery outcomes (live births, stillbirths, miscarriages). The costs will primarily be from the payer perspective, the state, but also costs to the patients will be collected and analyzed. The second approach will be to analyze each system separately from an equity perspective, i.e. how costs and access to services are distributed according to categories of socio-economic status. In view of the complexity and multi-dimensionality of both the interventions and the outcomes, we will not aim to comprise the analysis in a single index number.
2. Facility Level:
At the facility level we will collect primary data on the types of facilities that are administering obstetric care under the programs (objective v), and the quality of care received (objective vi). These facilities will be visited to study maternal health outcomes. Both quantitative and qualitative methods will be utilized to achieve these objectives.
An initial list of all public and private facilities in the study districts will be obtained from government records. The list will be verified by site visits. Snowballing will then be utilized to identify any missing private facilities where obstetric care is provided. A modified Averting Maternal Death and Disability (AMDD) survey [12
] will be administered to each facility by a member of the research team and the performance of EmOC functions will be ascertained by interviewing key staff. The facilities will be classified with reference to standards based on the UN and WHO definitions [13
] as non-EmOC, Basic Emergency Obstetric Care (BEmOC) or Comprehensive Emergency Obstetric Care (CEmOC). The availability level of EmOC services will be ascertained with reference to the population; the number of CEmOC, BEmOC and non-EmOC facilities in the public and private sector and in the rural and urban areas will be calculated.
A cross-sectional survey will provide additional information on how the facilities function under the respective programs as well as aspects of quality of care provided. The assessment will include variables on infrastructure, accessibility, human resources, functional equipment, accessibility to drugs and supplies, referral communications and processes, infection control procedures and service statistics. This survey will provide the current operating state of the facilities. Descriptive statistics will be calculated for the areas mentioned above. These will be compared between public and private facilities and between the two programs.
Data on delivery outcomes for women admitted in the facility will also be collected over a five day period at each facility that had more than 10 deliveries in a month. A research assistant, assigned to the facility, will administer a short questionnaire to mothers who deliver within the defined time frame. Basic socio-demographic characteristics, pregnancy and delivery details, and indication for referral if appropriate will be elicited. More specific details of the same delivery will be obtained from the nurse in the labor ward. These will include type of delivery, presence of known pregnancy or delivery related complications, maternal and neonatal outcomes. This will provide information on the proportion of these events among program mothers. In the case of CY, it will allow a comparison with non-program mothers who deliver at the same time in the program hospitals. This study will provide a cross sectional snap shot of outcomes for program mothers. The proportion of program mothers who have complications, who undergo instrumental delivery, referred, who have adverse neonatal outcomes will be reported as will the number of maternal deaths during this period.
The Hulton et al. Quality of Care framework will be used to assess the quality of care (objective vi) from both the provision of care (facility perspective) and the experience of care (user perspective) [14
]. Observations of normal deliveries, case record and register reviews, competency rating of service care provider, mother’s experience and the referral process will all be used to ascertain the quality of care being administered to obstetric patients.
Non-participant observation of normal deliveries will be performed on a sample of mothers in selected facilities. The observations will describe the technical care given during labor and delivery, infection control practices and the interaction between the staff and mother. A retrospective review of pre-defined indicators from a sample of case records will be performed in all study facilities to assess the quality of records maintained. The registers will also be assessed and the number of deliveries and procedures performed will be recorded. The competence of labor ward nurses providing obstetric care will be assessed utilizing case vignettes. The case vignettes will focus on normal labor and delivery. A short structured exit interview will be administered to all post-partum mothers enrolled in the study before discharge to determine which key procedures they underwent. The procedures included in the questionnaire will be based on common practices that may harm or benefit the mother during or after delivery, as described in the WHO evidence based practices [14
]. A sub-set of women who participate in the exit interview will be selected for an in-depth interview that will take place in their homes within four weeks of delivery. The in-depth interview will explore her experience of delivery care under the program and explore potential aspects of care that may inhibit her utilization of the facility for a future delivery. Indications for referral and referral processes for women referred to facilities at different levels will also be studied. Mothers who have been referred will be interviewed to obtain details of indications for referral in, distance travelled, time taken, transport and costs.
For the observations, content analysis will be used to conceptualize and classify events, behaviors and actions to obtain a rich understanding of quality of care and explanation of how women experience care at facilities [16
]. Basic descriptive statistics will be calculated for key indicators from the record and register review. With regard to provider competence, descriptive analysis will identify competence areas that require strengthening. The data will be analyzed using inferential statistics to compare scores between levels of facilities and qualification and experience of providers. The data from exit interviews will be analyzed to describe use of evidence based practices by level of facility. Qualitative data derived from the in-depth interviews with mothers will be analyzed using the thematic Framework approach [17
]. The obstetric referrals, indications for referral and details of referral-transport, costs, distance, and time will be described by the level of facility using mean, standard deviation and tests of significance. The referral pathways of the maternal deaths will be mapped on GIS and buffer analyzed. The buffer analysis will be used to study referring facility locations with regard to transfer time to CEmOC facilities.
3. Community level (mothers in the study villages):
At the community level we will collect primary data from recently delivered mothers to compare outcomes between eligible beneficiaries and non-beneficiaries (objective vii) for each of the programs. A cross-sectional study will be conducted with women residing in the selected study villages who delivered in the last year. JSY program participation is defined as an institutional delivery in any public health facility. Program non participation is defined as a home or private facility delivery. CY program participation is defined as a below poverty line (BPL) or tribal mother who delivers in a CY approved facility. Program non participation is defined as BPL or tribal mother who delivers outside of a CY approved facility.
Stratified systematic random sample will be used to select the study villages in each district. The districts will first be stratified by blocks (administrative units) and then the villages in each block will be classified into two strata based on the size of the population. Villages will be randomly selected from each strata based on the population proportion. Given the different design of each program, the sample sizes will vary in each province.
There will be three points of data collection; (i) when the woman delivers, (ii) 15 days and (iii) 28 days post-delivery. The ASHA will administer a simple symptom based checklist to all mothers regardless of place of delivery to ascertain maternal morbidity, complications associated with the pregnancy and delivery, at the three time points. To ensure quality data collection, a physician will administer a second checklist to 5% of all mothers which will be compared to the data collected by the ASHA.
Socio-demographic details, history of previous pregnancies, receipt of comprehensive maternal health care (antenatal, intrapartum and post natal care), place of delivery, delivery type, birth weight, immunization of the child, indirect and direct costs associated with delivery, program participation and maternal and neonatal outcomes will be elicited post-delivery in the mother’s home by a trained research assistant. In addition, reasons for participation/non-participation in the program, reasons for their choice of facility and barriers (if any) to access of care will be collected. Data will also be collected on household amenities and assets. This will help create an indicator to assist in quantifying the degree of income related inequalities and how it influences the inequities in access to care. In the event of a maternal death, the head of the household will be requested to provide the information.
Propensity score matching will be used to select a comparison group from the sample of non-participants closest in terms of observable characteristics to a sample of program participants. Comparisons of the outcome variables described above between the two groups (users and non-users) will be studied. Reasons for participation/non participation will be described. In a separate hierarchical model, the predictors of non-participation will be studied in each program. Costs incurred will be compared by place of delivery.
The facilities identified in objective v will be mapped onto a Geographic Information System (GIS) (objective viii). Digitized GIS maps of the selected districts in both provinces will be created. The map will have basic layers of district and block boundaries, road and rail networks, and settlements. Global Positioning System (GPS) co-ordinates will be taken for all facilities and added to the map as a separate layer. The location of the mothers who participated or did not participate in the program (objective vii) will be added as an additional layer. Distance from the mothers’ village to the selected health center will be calculated from the GIS. The association between distance from a facility and program use will be studied. Level of care offered at the facility will also be used as a predictor in the model for choice of place of delivery. Suitable GIS software along with standard statistical software will be used. A regression model will be developed using the background predictors of place of delivery (maternal characteristics, socioeconomic situation) as well as distance to the nearest facility.
The last objective in this level will focus on the community health workers’ participation in the program (objective ix). The ASHA is a female resident of the village who is incentivized by the government to motivate women to deliver at facilities under the programs. The government has identified the ASHA as a pivotal role in the community success of the program [18
]. A purposive sample of ASHAs and mothers from their villages will be identified and invited to participate in separate focus group discussions and individual in-depth interviews. The following areas will be explored within the two groups: their practical role in the program, experience recruiting and retaining mothers in the program (cooperation received and barriers encountered), influence on the decision making process of the mothers on where to deliver, types of services provided to the mother, the interaction with other ground level health workers, perception of how their activism is received within the community (voluntary, enforced or incentivized activism), and long term career aspirations. In addition, a sample of TBAs will be identified and interviewed about their role in the program and how they interact with the mothers, ASHAs and other community level health workers. Among NRHM program officials at state level, the risk and implications of overlapping roles in the community will be explored through in-depth interviews. Data from all interviews and group discussions will be analyzed using thematic Framework analysis [17