Schedule IV - Mortgage Loans on Real Estate (Details) $ in Thousands	12 Months Ended
	Dec. 31, 2021 USD ($) loan counterparty	Dec. 31, 2020 USD ($)	Dec. 31, 2019 USD ($)
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Carrying Value	$ 3,575,601	$ 3,049,166	$ 20,780,548
Principal Amount of Loans Subject to Delinquent Principal or Interest	238,887
Reconciliation of Balance Sheet Reported Amounts of Mortgage Loans on Real Estate
Beginning balance	3,049,166	20,780,548	12,707,625
Additions during period:
Purchases	1,581,979	569,557	8,762,553
Accretion of purchase discount	4,154	5,265	11,234
Change in realized and unrealized gains	44,564	101,957	638,557
Deductions during period:
Repayments of principal	(1,018,176)	(674,337)	(1,052,812)
Collection of interest	0	0	(11,429)
Transfer to investment securities available for sale	0	(237,297)	0
Transfer to REO	(4,133)	(8,509)	(6,105)
Cost of loans sold	(77,127)	(17,478,478)	(213,871)
Provision for loan loss	0	0	2,780
Amortization of premium	(4,826)	(15,352)	(57,984)
Balance at end of period	3,575,601	3,049,166	20,780,548
Cumulative Effect, Period of Adoption, Adjustment
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Carrying Value		0	5,812
Reconciliation of Balance Sheet Reported Amounts of Mortgage Loans on Real Estate
Beginning balance	$ 0	5,812	0
Deductions during period:
Balance at end of period		$ 0	$ 5,812
Mortgage Loans Under $100,000 \| First lien loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	1,120
Carrying Value	$ 57,475
Principal Amount of Loans Subject to Delinquent Principal or Interest	1,030
Deductions during period:
Balance at end of period	$ 57,475
Mortgage Loans Under $100,000 \| First lien loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	0.00%
Mortgage Loans Under $100,000 \| First lien loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	14.99%
Mortgage Loans Under $100,000 \| First lien loans \| Residential loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	1,178
Carrying Value	$ 64,755
Principal Amount of Loans Subject to Delinquent Principal or Interest	8,968
Deductions during period:
Balance at end of period	$ 64,755
Mortgage Loans Under $100,000 \| First lien loans \| Residential loans held in securitization trusts \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	1.38%
Mortgage Loans Under $100,000 \| First lien loans \| Residential loans held in securitization trusts \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	12.13%
Mortgage Loans Under $100,000 \| Second lien loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	245
Carrying Value	$ 10,126
Principal Amount of Loans Subject to Delinquent Principal or Interest	576
Deductions during period:
Balance at end of period	$ 10,126
Mortgage Loans Under $100,000 \| Second lien loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	5.75%
Mortgage Loans Under $100,000 \| Second lien loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	8.88%
Mortgage Loans Under $100,000 \| Business purpose loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	452
Carrying Value	$ 56,598
Principal Amount of Loans Subject to Delinquent Principal or Interest	1,818
Deductions during period:
Balance at end of period	$ 56,598
Mortgage Loans Under $100,000 \| Business purpose loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	4.25%
Mortgage Loans Under $100,000 \| Business purpose loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	13.99%
Mortgage Loans Under $100,000 \| Business purpose loans \| Residential loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| counterparty	60
Carrying Value	$ 6,797
Principal Amount of Loans Subject to Delinquent Principal or Interest	0
Deductions during period:
Balance at end of period	$ 6,797
Mortgage Loans Under $100,000 \| Business purpose loans \| Residential loans held in securitization trusts \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	7.25%
Mortgage Loans Under $100,000 \| Business purpose loans \| Residential loans held in securitization trusts \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	13.50%
Mortgage Loans Between $100,000 - $199,999 \| First lien loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	1,341
Carrying Value	$ 173,951
Principal Amount of Loans Subject to Delinquent Principal or Interest	2,001
Deductions during period:
Balance at end of period	$ 173,951
Mortgage Loans Between $100,000 - $199,999 \| First lien loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	2.00%
Mortgage Loans Between $100,000 - $199,999 \| First lien loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	11.84%
Mortgage Loans Between $100,000 - $199,999 \| First lien loans \| Residential loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	1,390
Carrying Value	$ 162,907
Principal Amount of Loans Subject to Delinquent Principal or Interest	19,276
Deductions during period:
Balance at end of period	$ 162,907
Mortgage Loans Between $100,000 - $199,999 \| First lien loans \| Residential loans held in securitization trusts \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	0.00%
Mortgage Loans Between $100,000 - $199,999 \| First lien loans \| Residential loans held in securitization trusts \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	11.85%
Mortgage Loans Between $100,000 - $199,999 \| Second lien loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	19
Carrying Value	$ 2,291
Principal Amount of Loans Subject to Delinquent Principal or Interest	0
Deductions during period:
Balance at end of period	$ 2,291
Mortgage Loans Between $100,000 - $199,999 \| Second lien loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	6.25%
Mortgage Loans Between $100,000 - $199,999 \| Second lien loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	8.63%
Mortgage Loans Between $100,000 - $199,999 \| Business purpose loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	453
Carrying Value	$ 80,298
Principal Amount of Loans Subject to Delinquent Principal or Interest	1,437
Deductions during period:
Balance at end of period	$ 80,298
Mortgage Loans Between $100,000 - $199,999 \| Business purpose loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	3.75%
Mortgage Loans Between $100,000 - $199,999 \| Business purpose loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	13.49%
Mortgage Loans Between $100,000 - $199,999 \| Business purpose loans \| Residential loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| counterparty	81
Carrying Value	$ 13,680
Principal Amount of Loans Subject to Delinquent Principal or Interest	169
Deductions during period:
Balance at end of period	$ 13,680
Mortgage Loans Between $100,000 - $199,999 \| Business purpose loans \| Residential loans held in securitization trusts \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	7.50%
Mortgage Loans Between $100,000 - $199,999 \| Business purpose loans \| Residential loans held in securitization trusts \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	12.99%
Mortgage Loans Between $200,000 - $299,999 \| First lien loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	763
Carrying Value	$ 170,794
Principal Amount of Loans Subject to Delinquent Principal or Interest	3,378
Deductions during period:
Balance at end of period	$ 170,794
Mortgage Loans Between $200,000 - $299,999 \| First lien loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	0.00%
Mortgage Loans Between $200,000 - $299,999 \| First lien loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	10.86%
Mortgage Loans Between $200,000 - $299,999 \| First lien loans \| Residential loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	619
Carrying Value	$ 123,285
Principal Amount of Loans Subject to Delinquent Principal or Interest	16,348
Deductions during period:
Balance at end of period	$ 123,285
Mortgage Loans Between $200,000 - $299,999 \| First lien loans \| Residential loans held in securitization trusts \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	1.75%
Mortgage Loans Between $200,000 - $299,999 \| First lien loans \| Residential loans held in securitization trusts \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	11.90%
Mortgage Loans Between $200,000 - $299,999 \| Second lien loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	6
Carrying Value	$ 1,333
Principal Amount of Loans Subject to Delinquent Principal or Interest	0
Deductions during period:
Balance at end of period	$ 1,333
Mortgage Loans Between $200,000 - $299,999 \| Second lien loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	6.75%
Mortgage Loans Between $200,000 - $299,999 \| Second lien loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	8.13%
Mortgage Loans Between $200,000 - $299,999 \| Business purpose loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	309
Carrying Value	$ 87,576
Principal Amount of Loans Subject to Delinquent Principal or Interest	1,755
Deductions during period:
Balance at end of period	$ 87,576
Mortgage Loans Between $200,000 - $299,999 \| Business purpose loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	3.50%
Mortgage Loans Between $200,000 - $299,999 \| Business purpose loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	12.50%
Mortgage Loans Between $200,000 - $299,999 \| Business purpose loans \| Residential loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| counterparty	73
Carrying Value	$ 19,006
Principal Amount of Loans Subject to Delinquent Principal or Interest	200
Deductions during period:
Balance at end of period	$ 19,006
Mortgage Loans Between $200,000 - $299,999 \| Business purpose loans \| Residential loans held in securitization trusts \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	6.50%
Mortgage Loans Between $200,000 - $299,999 \| Business purpose loans \| Residential loans held in securitization trusts \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	12.00%
Mortgage Loans Over $299,999 \| First lien loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	968
Carrying Value	$ 417,639
Principal Amount of Loans Subject to Delinquent Principal or Interest	9,205
Deductions during period:
Balance at end of period	$ 417,639
Mortgage Loans Over $299,999 \| First lien loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	1.88%
Mortgage Loans Over $299,999 \| First lien loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	9.13%
Mortgage Loans Over $299,999 \| First lien loans \| Residential loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	673
Carrying Value	$ 241,055
Principal Amount of Loans Subject to Delinquent Principal or Interest	30,946
Deductions during period:
Balance at end of period	$ 241,055
Mortgage Loans Over $299,999 \| First lien loans \| Residential loans held in securitization trusts \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	1.38%
Mortgage Loans Over $299,999 \| First lien loans \| Residential loans held in securitization trusts \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	9.75%
Mortgage Loans Over $299,999 \| Business purpose loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	688
Carrying Value	$ 645,209
Principal Amount of Loans Subject to Delinquent Principal or Interest	5,874
Deductions during period:
Balance at end of period	$ 645,209
Mortgage Loans Over $299,999 \| Business purpose loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	3.50%
Mortgage Loans Over $299,999 \| Business purpose loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	12.00%
Mortgage Loans Over $299,999 \| Business purpose loans \| Residential loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| counterparty	178
Carrying Value	$ 169,944
Principal Amount of Loans Subject to Delinquent Principal or Interest	0
Deductions during period:
Balance at end of period	$ 169,944
Mortgage Loans Over $299,999 \| Business purpose loans \| Residential loans held in securitization trusts \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	6.50%
Mortgage Loans Over $299,999 \| Business purpose loans \| Residential loans held in securitization trusts \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	11.00%
First lien loans \| Consolidated SLST
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	6,802
Carrying Value	$ 1,070,882
Principal Amount of Loans Subject to Delinquent Principal or Interest	135,906
Deductions during period:
Balance at end of period	$ 1,070,882
First lien loans \| Consolidated SLST \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	1.38%
First lien loans \| Consolidated SLST \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	10.50%