While there are signs that the rental market is softening at the high end, it appears to be “settling into a new normal” with nearly half of all U.S. renter households considered cost burdened, according to the Joint Center for Housing Studies (JCHS) of Harvard University’s biennial America’s Rental Housing 2017 report.

“Since the start of the decade we are seeing a broad-based surge in demand for rental housing and with that a strong supply response, but both of which are now beginning to slow to a more measured pace,” says Chris Herbert, JCHS managing director. “But the new normal that is emerging after the shifts of the past decade include two troubling elements. The first is that the market has struggled to be able to supply moderate-cost rentals, and the second is that even with recent improvements the number and share of renters paying excessive shares of their income for housing remains not far from record levels.”

With over 45 million rental households in the United States, that number declined slightly in 2016. Construction starts on new rental units leveled off in 2016 and have fallen by 9% so far this year. The national vacancy rate also rose to 7.2% in the third quarter from 6.9% in the third quarter of 2016. However, rents still continue to rise faster than inflation, says Herbert.

For a second consecutive year, the number of cost-burdened renters, those paying over 30% of income for housing, has declined. According to the report, that number fell from 21.3 million in 2014 to 20.8 million in 2016.

For severely cost-burdened renters, those paying over 50% of income for housing, the number decreased from 11.4 million in 2014 to 11 million in 2016.

“Yes, we’re seeing improvement, and the market is providing a lot of housing, and we’re starting to see vacancies rise and rent growth slow,” Herbert says. “The market is working as it should, except it really struggles to provide housing below that top-tier level. We have to be innovative on the private-sector side and what we can do to help folks who can’t afford private-sector housing to close that gap.”

Despite the declines over the past two years, the number of cost-burdened renters is well above the 2001 levels. At the average rate of improvement from 2014 to 2016, it will take 24 long years for the number of cost-burdened renters to return to the 2001 figures.

In addition, more middle-income renters are spending more of their income for housing. The share of renters earning between $30,000 and $45,000 with cost burdens increased from 37% in 2001 to 50% in 2016, and those earning between $45,000 and $75,000 almost doubled from 12% in 2001 to 23% in 2016.

The lowest-income households, those earning less than $15,000, remained at a steep 83%.

With the shift of building toward higher-income households, there also is a shortage of adequate and available rentals for extremely low-income and very low-income households. For every 100 extremely low-income households, there are only 66 units anywhere in the country that are affordable. However, half of those units are occupied by renters at higher incomes, says senior research associate Jonathan Spader.

As low-cost housing units are disappearing, it’s also becoming increasingly difficult for very low-income households to access rental assistance. Only 25% of very low-income households received rental assistance in 2015, down from 28% in 2001.

“If there’s an industry that needs to be transformed, it’s the housing industry in general, but the affordable housing industry in particular. We need a lot more innovation in building code, in building construction, in the types of housing we’re producing, and how to reuse existing housing spaces,” says Shekar Narasimhan, managing partner at Beekman Advisors. “If we really spend a lot of time and attention there instead of just thinking about capital structures and subsidy systems, I think we would do ourselves a great service in the next couple of years.”