A New York state judge approved most of an $8.5 billion settlement between Bank of America and investors over soured mortgage-backed securities.

But in denying approval of one portion of the deal, the decision raises questions over whether Bank of America will proceed with the pact if it remains open to future claims by investors.