First in time, first in place. That means that whichever lien is recorded first, has first lien preference. Unless a lienholder consents otherwise. If you simply refinanced the first mortgage and no contact was made with the second mortgage holder, the second mortgage would have a priority lien, which would violate one of the sections of the new first mortgage.

This does not mean, however, that you MUST incorporate a second mortgage balance into your refinanced loan. What is DOES mean is that you will need to get a subordination agreement from the lender holder your second mortgage and have that recorded. The subordination agreement simply states that the second mortgage holder agrees to stay in second place. Sometimes there is a fee from that lender to obtain one. But, if the first mortgage balance is increasing only by the closing costs AND you are improving your ability to pay by lowering the rate and payment on the first mortgage, then most lenders will issue a subordination.

If you are looking at a 'cash out' refinance, though, it is very unlikely that the holder of the HELOC will agree to subordinate their lien.

A home refinance does necessarily require the payoff of both of the existing liens.  However, there are several variables that will come into play as to whether someone will be able to do so. 

One question is whether the holder of the second lien will agree to resubordinate its lien, as discussed above.  The other question is whether a new lender will give you a loan for the first knowing that there is going to be a second mortgage, regardless of whether you otherwise qualify. 

Let's say you bought the house for $100,000 with an $80,000 first mortgage and a $20,000 second mortgage.  When you decide to refinance you discover that your home's value has not changed.  Some loan programs place limits on the combined LTV (amount of new first mortgage plus amount of resubordinated second mortgage divided by appraised value) for any new loans.  Thus, in this example, unless the new lender permits a combined LTV of 100%, you will not be able to refinance.