West Bengal Outperformed States That Implemented PM's Health Insurance Scheme, On Key Indicators

The PMJAY scheme aims to provide Rs 5 lakh health insurance coverage per year to nearly 110 million economically vulnerable families, or 500 million people. PMJAY is jointly implemented by the central and state governments in 32 states and UTs. West Bengal, Telangana, Odisha and Delhi have not implemented PMJAY. The Telangana government has announced in December 2020 that it intends to participate in PMJAY.

As PMJAY implementation began in 2018, data from the fourth (2015-16) and fifth (2019-20) rounds of the National Family Health Survey (NFHS), India's largest official household health survey used for framing health policy, would adequately capture pre- and post-implementation health outcomes, the Economic Survey said. This would enable an assessment of the impact of PMJAY.

The Economic Survey concluded that of 22 states and UTs covered in Phase 1 of NFHS-5 (2019-20), PMJAY states were showing greater improvements than non-PMJAY states on several indicators. West Bengal had been outperformed by PMJAY-implementing neighbours on many of these indicators, the Centre's survey said.

"Most of the indicators used in the report have nothing to do with PMJAY. For example, PMJAY does not offer vaccination or ante-natal care visits. PMJAY is restricted to hospitalisations. So the Economic Survey's assessment of PMJAY improving various such health parameters is flawed to begin with," Sulakshana Nandi, Chhattisgarh convenor of the Public Health Resource Network, and a researcher on state-funded insurance schemes, told IndiaSpend.

The Survey, all the same, has made a larger conclusion--that multiple health indicators have improved in states which have implemented the scheme. IndiaSpend analysed the same data used in the Economic Survey to verify the claims, and found that the picture is more complex. We also analysed other indicators to see if the premise of the Economic Survey stands, and found that West Bengal was often performing higher than the states it has been compared with.

While the claims in the Economic Survey are technically correct, West Bengal's health data have been compared to the average of its three neighbours' after adding their health data together. It is unclear why data of one state should be compared to average aggregated data of three other states.

Being West Bengal's neighbours, Bihar, Assam and Sikkim are therefore "similar on socio-economic dimensions," the Economic Survey has claimed. None of these states, however, share a border with each other, and their socio-economic dissimilarities are underscored by the same NFHS data the Economic Survey has used.

Further, should health data of Bihar, India's poorest and third-most populous state, situated in the Gangetic plain, be grouped with that of India's second-wealthiest and least populous state Sikkim, a former Himalayan kingdom, to make a comparison with a third state? That would be akin to comparing upper middle-income country Mexico's gross domestic product (GDP) to the combined GDP of its northern and southern neighbours, high income United States and upper-middle income Guatemala, respectively. Mexico's GDP would inevitably be lesser.

In fact, West Bengal outperformed its neighbours and other large states that implemented PMJAY, on several health indicators mentioned by the Economic Survey, including infant and child mortality, child vaccinations and use of family planning methods, per the same NFHS data disaggregated by individual state, our analysis shows. Bihar's performance was poor on several indicators, but this was not evident from the aggregated outcomes. Many states have shown worsening infant and child mortality, despite implementing PMJAY.

Further, the Survey used "difference-in-difference" calculations--that is, relative difference in improvement between states. In such analyses, states that have more ground to cover to begin with would seem to be performing better than those that were better off in the first place--a fact that the Survey alludes to when it mentions a "higher base" or a "lower base".

Moreover, health insurance coverage is only one way to assess access to health; affordability is another, health researchers tell us. For instance, increased coverage under PMJAY should reflect both higher institutional newborn deliveries and a decrease in out-of-pocket expenditure (OOPE) for the same, they say, but PMJAY has not had a positive bearing on either. On these indicators, West Bengal has outperformed both its neighbours and all large states that have implemented PMJAY, NFHS data show. In 12 of 20 PMJAY states, OOPE on delivery increased by between 3% and 130%.

Claim: PMJAY implementation enhanced health insurance coverage in states adjacent to West Bengal by 89%, while coverage fell 12% in the latter

PMJAY enhanced health insurance coverage by 54% in states and UTs included in Phase 1 of NFHS-5 (2019-20), in the four years since NHFS-4 (2015-16), said the Economic Survey, while falling by nearly 10% in non-PMJAY West Bengal and Telangana. In West Bengal's neighbours Bihar, Assam and Sikkim, the proportion of households with health insurance coverage increased by 89%, while in West Bengal, the same metric declined by 12%, said the Economic Survey.

Disaggregating the four states on individual performance, health insurance coverage increased by 2.3 and 49.6 percentage points in Bihar and Assam, respectively. In Sikkim, coverage decreased by 4.6 and in West Bengal by 4.1 percentage points, our analysis shows.

It would not be appropriate to compare tiny Sikkim's performance on health, however, with West Bengal, which has 150 times more people. Sikkim's health indicators are better compared with all other North-Eastern states' included in the NFHS-5.

Looking at the individual performance of large PMJAY states that have over 10 million people, like West Bengal, Andhra Pradesh has also seen a decrease in households with health insurance cover and Karnataka has seen no change, despite both having implemented PMJAY.

Overall, the proportion of households with health insurance cover has increased 10 percentage points, on average, in large states that implemented PMJAY.

Out-of-pocket expenditure on newborn delivery higher in PMJAY states

Measuring access to health insurance is only one way to assess affordability of healthcare for people. Another is to look at the metric of out-of-pocket expenditure (OOPE), i.e. how much people are paying from their own pockets for healthcare. It makes sense to look at OOPE as the main indicator to show the performance of a cashless scheme such as PMJAY, said Nandi.

"Any insurance scheme that claims to be cashless must lead to zero OOPE," she said.

Average OOPE per newborn delivery in a public health institution is the only OOPE metric publicly available in NFHS-5 factsheets. On this particular metric, the same data set shows West Bengal to be the best performer among all states, by a substantial degree.

People in West Bengal paid on average Rs 7,919 per delivery in a public health facility in 2015-16, per NFHS-4. By NFHS-5 (2019-20), they paid Rs 2,683, a 66% reduction in cost. The state went from being the second-worst on this metric in 2015-16 to fourth-best in 2019-20, among the 22 states/UTs, large and small, covered in Phase 1 of NFHS-5. OOPE per delivery rose in 12 out of 20 PMJAY states and UTs, by between 3 and 130%. Among large states, only Gujarat has a lower OOPE per delivery in a public health facility than West Bengal.

PMJAY partly covers institutional deliveries of babies, including in public hospitals, so enrollment in PMJAY should reflect higher utilisation of health facilities for institutional delivery, and a decrease in OOPE for the same. "However, the data does not show that this has happened," said Nandi, based on her own analysis of NFHS data. PMJAY has not had a positive bearing either on the levels of institutional deliveries or on the amount of money women spend on it, she concluded.

"The biggest failure of PMJAY and other insurance schemes in India has been their inability to decrease OOPE. This has been proven time and again from numerous studies. A study from Chhattisgarh that analysed [National Sample Survey data] on utilisation of hospital-care pre- and post-PMJAY implementation found that enrollment under PMJAY or other schemes did not increase this in Chhattisgarh. The study also found that OOPE and incidence of catastrophic health expenditure did not decrease with enrollment under PMJAY," said Nandi.

Claim: West Bengal outperformed its neighbours on proportion of institutional births; increased C-section deliveries show people accessed public healthcare more in PMJAY states

The Economic Survey noted that while the percentage of institutional births increased in all four states, West Bengal's performance saw a larger increase at 22%, compared to its PMJAY neighbours Bihar, Assam and Sikkim, at 11%. In terms of births in public health institutions, West Bengal's performance (a 28% increase) was even better compared to its neighbours (10%), per the Economic Survey. This remains correct even when comparing West Bengal only with other large states. Non-PMJAY Telangana too has outperformed large PMJAY states Andhra Pradesh, Maharashtra and Karnataka on this metric. Thus, a link between PMJAY and increased institutional deliveries is not made clear by the data.

An increase in births via caesarean section (C-section) in public healthcare institutions has been higher in Bihar, Assam and Sikkim (46%), versus in West Bengal, (21%), said the Economic Survey, while noting that West Bengal began from a higher base. The survey concluded that PMJAY thus "seems to have enabled" people in Bihar, Assam and Sikkim to make greater use of public health facilities.

Increased use of C-sections may not, however, be the best metric to assess improved public health access, given this health ministry guidance from July 2019, quoting a World Health Organization finding that "at population level, caesarean section rates higher than 10% are not associated with reductions in maternal and new-born mortality rates". The ministry ordered a curb of unnecessary C-sections in public health institutions. Such births have increased in public health institutions in all large states, PMJAY and otherwise, in the four years to NFHS-5 (2019-20). Bihar (3.6%) is the only large state where C-section births are less than 10% of all deliveries.

Claim: West Bengal's neighbours performed better in reducing infant and under-five mortality

The Economic Survey said that in the four years between the two NFHS surveys, infant mortality had declined by 20% for West Bengal, but by 28% for its three neighbouring states. On under-five mortality as well, West Bengal had seen an improvement of 20% whereas its three neighbouring states improved by 27%. Lastly, the neonatal mortality rate declined by 22% in PMJAY states compared to 16% in non-PMJAY states, said the Economic Survey.

Once again, West Bengal's standalone performance was judged, while infant and child mortality rates for its three neighbouring states were aggregated to provide their combined performance. When we look at individual performance, the Economic Survey's claim is only partly correct.

In the four years to 2019-2020, West Bengal improved infant mortality by 20%. In Sikkim, infant mortality improved by 62% and in Assam, by 33%. In Bihar, infant mortality improved by just 2.7%. On under-five mortality, West Bengal improved by 20.1%, Sikkim by 65.2% and Assam by 30.8%. In Bihar, under-five mortality dropped by just 2.9%. By 2019-2020, West Bengal improved neonatal mortality by 29.5%, while Sikkim and Assam did so by 76% and 31.4%. But it didn't drop as much in Bihar (6%, from 36.7 deaths per 1,000 live births to 34.5).

Bihar's actual performance was obscured behind the aggregated figures of all three states. It continues to be the worst-performer on neonatal, infant and under-five mortality among all the 22 states and UTs covered in NFHS-5 Phase 1.

What about other large states that have implemented PMJAY but were not included in the Economic Survey's analysis? West Bengal has seen the third biggest improvement in neonatal, infant and under-five mortality compared to all other large PMJAY states. Only Assam and Jammu & Kashmir have performed better than West Bengal on these metrics. Non-PMJAY Telangana too has seen higher improvement than Maharashtra and Kerala.

Claim: Adjacent PMJAY states saw a higher increase in use of family planning methods than West Bengal

West Bengal's three neighbouring states had performed better on use of family planning methods as well, the Economic Survey said. Bihar, Sikkim and Assam saw a 51% increase in use of family planning methods, while West Bengal improved by only 5%, said the survey. "[S]imilar to what we observed in the case of child mortality, the increase has been higher in states that have adopted PM-JAY," concluded the survey.

Disaggregating NFHS-4 (2015-16) data by individual state, 24.1%, 46.7%, 52.4% and 70.9% of married women aged 15-49 years in Bihar, Sikkim, Assam and West Bengal, respectively, had used at least one family planning method. This has improved in all four states by NFHS-5 (2019-20), by 31.7, 22.4, 8.4, and 3.5 percentage points, respectively.

It appears that West Bengal has seen the least improvement on this important metric. But the base for West Bengal's performance is much higher to begin with, as the data above show and the Economic Survey also notes. It was and remains the best performer on uptake of family planning methods among all 22 states and UTs, large and small, covered in NFHS-5 Phase 1, with 74.4% of married women in West Bengal accessing family planning methods, without PMJAY.

Claim: The proportion of children aged 12-23 months who have received the BCG vaccine increased 5% in PMJAY states, but decreased by 1% in non-PMJAY states

The Economic Survey said the number of children aged 12 to 23 months immunised with BCG vaccine (for tuberculosis and meningitis) increased in PMJAY states compared to non-PMJAY states. However, BCG is only one of five types of immunisation given to children assessed in the Economic Survey.

On fully vaccinating children, West Bengal has outperformed all the large PMJAY states. While Assam, Maharashtra and Karnataka have seen a higher level of increase than West Bengal, they are catching up to West Bengal's higher base level of child vaccinations. Telangana too is ahead of all the large PMJAY states, except J&K and Karnataka.

"The Economic Survey's attribution of health indicators to PMJAY is faulty and unscientific as it does not use adequate statistical methods for such an analysis. States which have done well in terms of health indicators and universal healthcare must be recognised and we should learn lessons from them," said Nandi. "We should also emphasise strengthening the government health system, rather than schemes such as PMJAY that commercialise healthcare further. This will ensure more equitable and free healthcare access," she added.

We welcome feedback. Please write to respond@indiaspend.org. We reserve the right to edit responses for language and grammar.