Schedule IV - Mortgage Loans on Real Estate (Details) $ in Thousands	12 Months Ended
	Dec. 31, 2022 USD ($) loan counterparty	Dec. 31, 2021 USD ($)	Dec. 31, 2020 USD ($)
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Fair Value of Loans Pledged	$ 3,525,080	$ 3,575,601	$ 3,049,166
Principal Amount of Loans Subject to Delinquent Principal or Interest	303,157
Reconciliation of Balance Sheet Reported Amounts of Mortgage Loans on Real Estate
Beginning balance	3,575,601	3,049,166	20,780,548
Additions during period:
Purchases	1,733,265	1,581,979	569,557
Accretion of purchase discount	5,292	4,154	5,265
Change in realized and unrealized gains	(404,524)	44,564	101,957
Deductions during period:
Repayments of principal	(1,362,294)	(1,018,176)	(674,337)
Collection of interest	0	0	0
Transfer to investment securities available for sale	0	0	(237,297)
Transfer to REO	(18,858)	(4,133)	(8,509)
Cost of loans sold	0	(77,127)	(17,478,478)
Provision for loan loss	0	0	0
Amortization of premium	(3,402)	(4,826)	(15,352)
Balance at end of period	3,525,080	3,575,601	3,049,166
Cumulative Effect, Period of Adoption, Adjustment
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Fair Value of Loans Pledged		0	0
Reconciliation of Balance Sheet Reported Amounts of Mortgage Loans on Real Estate
Beginning balance	$ 0	0	5,812
Deductions during period:
Balance at end of period		$ 0	$ 0
Mortgage Loans Under $100,000 \| First lien loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	333
Fair Value of Loans Pledged	$ 9,272
Principal Amount of Loans Subject to Delinquent Principal or Interest	462
Deductions during period:
Balance at end of period	$ 9,272
Mortgage Loans Under $100,000 \| First lien loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	2.00%
Mortgage Loans Under $100,000 \| First lien loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	14.99%
Mortgage Loans Under $100,000 \| First lien loans \| Residential loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	1,776
Fair Value of Loans Pledged	$ 86,794
Principal Amount of Loans Subject to Delinquent Principal or Interest	8,923
Deductions during period:
Balance at end of period	$ 86,794
Mortgage Loans Under $100,000 \| First lien loans \| Residential loans held in securitization trusts \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	1.38%
Mortgage Loans Under $100,000 \| First lien loans \| Residential loans held in securitization trusts \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	14.29%
Mortgage Loans Under $100,000 \| Second lien loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	187
Fair Value of Loans Pledged	$ 6,281
Principal Amount of Loans Subject to Delinquent Principal or Interest	450
Deductions during period:
Balance at end of period	$ 6,281
Mortgage Loans Under $100,000 \| Second lien loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	5.75%
Mortgage Loans Under $100,000 \| Second lien loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	8.88%
Mortgage Loans Under $100,000 \| Business purpose loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	276
Fair Value of Loans Pledged	$ 45,808
Principal Amount of Loans Subject to Delinquent Principal or Interest	1,853
Deductions during period:
Balance at end of period	$ 45,808
Mortgage Loans Under $100,000 \| Business purpose loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	4.25%
Mortgage Loans Under $100,000 \| Business purpose loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	13.00%
Mortgage Loans Under $100,000 \| Business purpose loans \| Residential loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| counterparty	380
Fair Value of Loans Pledged	$ 53,978
Principal Amount of Loans Subject to Delinquent Principal or Interest	1,425
Deductions during period:
Balance at end of period	$ 53,978
Mortgage Loans Under $100,000 \| Business purpose loans \| Residential loans held in securitization trusts \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	4.00%
Mortgage Loans Under $100,000 \| Business purpose loans \| Residential loans held in securitization trusts \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	13.00%
Mortgage Loans Between $100,000 - $199,999 \| First lien loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	333
Fair Value of Loans Pledged	$ 37,214
Principal Amount of Loans Subject to Delinquent Principal or Interest	405
Deductions during period:
Balance at end of period	$ 37,214
Mortgage Loans Between $100,000 - $199,999 \| First lien loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	2.13%
Mortgage Loans Between $100,000 - $199,999 \| First lien loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	11.72%
Mortgage Loans Between $100,000 - $199,999 \| First lien loans \| Residential loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	2,236
Fair Value of Loans Pledged	$ 227,081
Principal Amount of Loans Subject to Delinquent Principal or Interest	20,136
Deductions during period:
Balance at end of period	$ 227,081
Mortgage Loans Between $100,000 - $199,999 \| First lien loans \| Residential loans held in securitization trusts \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	0.00%
Mortgage Loans Between $100,000 - $199,999 \| First lien loans \| Residential loans held in securitization trusts \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	11.85%
Mortgage Loans Between $100,000 - $199,999 \| Second lien loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	12
Fair Value of Loans Pledged	$ 1,259
Principal Amount of Loans Subject to Delinquent Principal or Interest	0
Deductions during period:
Balance at end of period	$ 1,259
Mortgage Loans Between $100,000 - $199,999 \| Second lien loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	6.25%
Mortgage Loans Between $100,000 - $199,999 \| Second lien loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	8.63%
Mortgage Loans Between $100,000 - $199,999 \| Business purpose loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	255
Fair Value of Loans Pledged	$ 50,122
Principal Amount of Loans Subject to Delinquent Principal or Interest	5,023
Deductions during period:
Balance at end of period	$ 50,122
Mortgage Loans Between $100,000 - $199,999 \| Business purpose loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	4.25%
Mortgage Loans Between $100,000 - $199,999 \| Business purpose loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	13.00%
Mortgage Loans Between $100,000 - $199,999 \| Business purpose loans \| Residential loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| counterparty	527
Fair Value of Loans Pledged	$ 74,963
Principal Amount of Loans Subject to Delinquent Principal or Interest	510
Deductions during period:
Balance at end of period	$ 74,963
Mortgage Loans Between $100,000 - $199,999 \| Business purpose loans \| Residential loans held in securitization trusts \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	3.75%
Mortgage Loans Between $100,000 - $199,999 \| Business purpose loans \| Residential loans held in securitization trusts \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	12.25%
Mortgage Loans Between $200,000 - $299,999 \| First lien loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	270
Fair Value of Loans Pledged	$ 49,466
Principal Amount of Loans Subject to Delinquent Principal or Interest	866
Deductions during period:
Balance at end of period	$ 49,466
Mortgage Loans Between $200,000 - $299,999 \| First lien loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	1.88%
Mortgage Loans Between $200,000 - $299,999 \| First lien loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	7.50%
Mortgage Loans Between $200,000 - $299,999 \| First lien loans \| Residential loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	1,092
Fair Value of Loans Pledged	$ 188,444
Principal Amount of Loans Subject to Delinquent Principal or Interest	15,714
Deductions during period:
Balance at end of period	$ 188,444
Mortgage Loans Between $200,000 - $299,999 \| First lien loans \| Residential loans held in securitization trusts \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	0.00%
Mortgage Loans Between $200,000 - $299,999 \| First lien loans \| Residential loans held in securitization trusts \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	11.90%
Mortgage Loans Between $200,000 - $299,999 \| Second lien loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	4
Fair Value of Loans Pledged	$ 714
Principal Amount of Loans Subject to Delinquent Principal or Interest	0
Deductions during period:
Balance at end of period	$ 714
Mortgage Loans Between $200,000 - $299,999 \| Second lien loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	6.75%
Mortgage Loans Between $200,000 - $299,999 \| Second lien loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	7.75%
Mortgage Loans Between $200,000 - $299,999 \| Business purpose loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	171
Fair Value of Loans Pledged	$ 50,510
Principal Amount of Loans Subject to Delinquent Principal or Interest	6,240
Deductions during period:
Balance at end of period	$ 50,510
Mortgage Loans Between $200,000 - $299,999 \| Business purpose loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	5.63%
Mortgage Loans Between $200,000 - $299,999 \| Business purpose loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	12.49%
Mortgage Loans Between $200,000 - $299,999 \| Business purpose loans \| Residential loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| counterparty	281
Fair Value of Loans Pledged	$ 66,168
Principal Amount of Loans Subject to Delinquent Principal or Interest	1,070
Deductions during period:
Balance at end of period	$ 66,168
Mortgage Loans Between $200,000 - $299,999 \| Business purpose loans \| Residential loans held in securitization trusts \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	3.50%
Mortgage Loans Between $200,000 - $299,999 \| Business purpose loans \| Residential loans held in securitization trusts \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	12.00%
Mortgage Loans Over $299,999 \| First lien loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	390
Fair Value of Loans Pledged	$ 141,546
Principal Amount of Loans Subject to Delinquent Principal or Interest	648
Deductions during period:
Balance at end of period	$ 141,546
Mortgage Loans Over $299,999 \| First lien loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	1.99%
Mortgage Loans Over $299,999 \| First lien loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	8.50%
Mortgage Loans Over $299,999 \| First lien loans \| Residential loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	1,305
Fair Value of Loans Pledged	$ 420,189
Principal Amount of Loans Subject to Delinquent Principal or Interest	31,884
Deductions during period:
Balance at end of period	$ 420,189
Mortgage Loans Over $299,999 \| First lien loans \| Residential loans held in securitization trusts \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	1.88%
Mortgage Loans Over $299,999 \| First lien loans \| Residential loans held in securitization trusts \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	9.75%
Mortgage Loans Over $299,999 \| Business purpose loans \| Residential loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	651
Fair Value of Loans Pledged	$ 689,192
Principal Amount of Loans Subject to Delinquent Principal or Interest	54,065
Deductions during period:
Balance at end of period	$ 689,192
Mortgage Loans Over $299,999 \| Business purpose loans \| Residential loans \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	4.00%
Mortgage Loans Over $299,999 \| Business purpose loans \| Residential loans \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	13.00%
Mortgage Loans Over $299,999 \| Business purpose loans \| Residential loans held in securitization trusts
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| counterparty	586
Fair Value of Loans Pledged	$ 498,497
Principal Amount of Loans Subject to Delinquent Principal or Interest	10,307
Deductions during period:
Balance at end of period	$ 498,497
Mortgage Loans Over $299,999 \| Business purpose loans \| Residential loans held in securitization trusts \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	3.49%
Mortgage Loans Over $299,999 \| Business purpose loans \| Residential loans held in securitization trusts \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	12.25%
First lien loans \| Consolidated SLST
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Number of Loans \| loan	6,160
Fair Value of Loans Pledged	$ 827,582
Principal Amount of Loans Subject to Delinquent Principal or Interest	143,176
Deductions during period:
Balance at end of period	$ 827,582
First lien loans \| Consolidated SLST \| Minimum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	1.38%
First lien loans \| Consolidated SLST \| Maximum
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Interest Rate	10.50%