$ 25,645

$ 27,298

$ 69,074

$ 116,934

14,500

$ 33,000

$ 42,700

6,900

5,300

19,900

18,300

Loans Held for Sale - Narrative (Details) - USD ($) $ in Thousands		3 Months Ended			9 Months Ended
	May 31, 2015	Sep. 30, 2016	Sep. 30, 2015	Mar. 31, 2015	Sep. 30, 2016	Sep. 30, 2015
Accounts, Notes, Loans and Financing Receivable [Line Items]
Gain on loans held for sale, net		$ 25,645	$ 27,298		$ 69,074	$ 116,934
Lending [Member] \| Line of Credit [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Collateral		288,400			288,400
Servicing [Member] \| Line of Credit [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Collateral		14,500			14,500
Unrelated Party [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Gain on loans held for sale, net	$ 7,200			$ 12,900
Unpaid principal balance	$ 33,000			$ 42,700
MSRs Retained [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Gain on loans held for sale, net		9,800	9,500		25,300	27,800
Repurchased Ginnie Mae Loans [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Gain on loans held for sale, net		6,900	5,300		19,900	18,300
Ginnie Mae Loans [Member]
Accounts, Notes, Loans and Financing Receivable [Line Items]
Gain on sales of reverse mortgage loans		$ 37,800	$ 30,100		$ 90,300	$ 91,100