Ultimately the only thing you can do is to either pay the loan off in full, or refinance the loan into a new mortgage at another institution.

When you got your loan you borrowed money from a bank. The loan paperwork you signed stipulated that the loan could be sold, and that that right to sell belonged to the bank. If they decided to sell the loan, even if you don't like the new lender you don't have any recourse. The simplest way to put this is because you have borrowed their money. If it were a checking or savings account you could certainly close your account, but being that you owe this bank the money due on the loan, you can't simply tell them you don't want to pay them. The only way to move the account from the new bank would be to refinance that loan with a new loan, at a new institution that you were more comfortable with, or to pay the mortgage loan off in full. As long as you owe that bank money you cannot close the account.