Residential Whole Loans - Components of Interest Income (Details) - Residential whole loans - USD ($) $ in Thousands	12 Months Ended
Dec. 31, 2021	Dec. 31, 2020	Dec. 31, 2019
Accounts, Notes, Loans and Financing Receivable [Line Items]
Interest income	$ 303,468	$ 332,212	$ 358,161
Purchased Credit Deteriorated Loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Interest income	$ 40,130	$ 36,238	$ 43,346

The aggregate interest and fee income generated by: (1) loans the Entity has positive intent and ability to hold for the foreseeable future, or until maturity or payoff, including commercial and consumer loans, whether domestic or foreign, which may consist of: (a) industrial and agricultural; (b) real estate; and (c) real estate construction loans; (d) trade financing; (e) lease financing; (f) home equity lines-of-credit; (g) automobile and other vehicle loans; and (h) credit card and other revolving-type loans and (2) loans and leases held-for-sale which may include mortgage loans, direct financing, and sales-type leases.

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 942
-SubTopic 220
-Section S99
-Paragraph 1
-Subparagraph (SX 210.9-04.1)
-URI http://asc.fasb.org/extlink&oid=120399700&loc=SL114874048-224260

Name:	us-gaap_InterestAndFeeIncomeLoansAndLeases
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	duration