Fair Value of Financial Instruments - Quantitative Information about Significant Unobservable Inputs Used in Valuation of Level 3 Assets and Liabilities Measured at Fair Value (Details)
$ in Thousands

Fair Value of Financial Instruments - Quantitative Information about Significant Unobservable Inputs Used in Valuation of Level 3 Assets and Liabilities Measured at Fair Value (Details) $ in Thousands	9 Months Ended
	Sep. 30, 2022 USD ($) $ / loan	Dec. 31, 2021 USD ($)
Assets
ABS issued, net, At fair value \| $	$ 7,564,312	$ 8,843,147
Variable Interest Entity, Primary Beneficiary \| Collateralized Financing Entities \| HEI
Liabilities
Fair value of securities owned \| $	14,000
ABS Issued
Liabilities
ABS issued \| $	$ 3,210,603
ABS Issued \| Prepayment rate (annual CPR) \| Minimum
Liabilities
ABS issued, measurement input	0.05
ABS Issued \| Prepayment rate (annual CPR) \| Maximum
Liabilities
ABS issued, measurement input	0.24
ABS Issued \| Prepayment rate (annual CPR) \| Weighted Average
Liabilities
ABS issued, measurement input	0.13
ABS Issued \| Whole loan discount rate \| Minimum
Liabilities
ABS issued, measurement input	0.04
ABS Issued \| Whole loan discount rate \| Maximum
Liabilities
ABS issued, measurement input	0.18
ABS Issued \| Whole loan discount rate \| Weighted Average
Liabilities
ABS issued, measurement input	0.06
ABS Issued \| Default rate \| Minimum
Liabilities
ABS issued, measurement input	0
ABS Issued \| Default rate \| Maximum
Liabilities
ABS issued, measurement input	0.33
ABS Issued \| Default rate \| Weighted Average
Liabilities
ABS issued, measurement input	0.01
ABS Issued \| Loss severity \| Minimum
Liabilities
ABS issued, measurement input	0.25
ABS Issued \| Loss severity \| Maximum
Liabilities
ABS issued, measurement input	0.50
ABS Issued \| Loss severity \| Weighted Average
Liabilities
ABS issued, measurement input	0.32
CAFL
Liabilities
ABS issued \| $	$ 2,703,223
CAFL \| Prepayment rate (annual CPR) \| Minimum
Liabilities
ABS issued, measurement input	0
CAFL \| Prepayment rate (annual CPR) \| Maximum
Liabilities
ABS issued, measurement input	0.03
CAFL \| Prepayment rate (annual CPR) \| Weighted Average
Liabilities
ABS issued, measurement input	0.002
CAFL \| Whole loan discount rate \| Minimum
Liabilities
ABS issued, measurement input	0.003
CAFL \| Whole loan discount rate \| Maximum
Liabilities
ABS issued, measurement input	0.16
CAFL \| Whole loan discount rate \| Weighted Average
Liabilities
ABS issued, measurement input	0.06
CAFL \| Default rate \| Minimum
Liabilities
ABS issued, measurement input	0.04
CAFL \| Default rate \| Maximum
Liabilities
ABS issued, measurement input	0.21
CAFL \| Default rate \| Weighted Average
Liabilities
ABS issued, measurement input	0.06
CAFL \| Loss severity \| Minimum
Liabilities
ABS issued, measurement input	0.30
CAFL \| Loss severity \| Maximum
Liabilities
ABS issued, measurement input	0.40
CAFL \| Loss severity \| Weighted Average
Liabilities
ABS issued, measurement input	0.30
Freddie Mac SLST
Liabilities
ABS issued \| $	$ 1,150,323
Freddie Mac SLST \| Prepayment rate (annual CPR) \| Minimum
Liabilities
ABS issued, measurement input	0.06
Freddie Mac SLST \| Prepayment rate (annual CPR) \| Maximum
Liabilities
ABS issued, measurement input	0.08
Freddie Mac SLST \| Prepayment rate (annual CPR) \| Weighted Average
Liabilities
ABS issued, measurement input	0.06
Freddie Mac SLST \| Whole loan discount rate \| Minimum
Liabilities
ABS issued, measurement input	0.05
Freddie Mac SLST \| Whole loan discount rate \| Maximum
Liabilities
ABS issued, measurement input	0.08
Freddie Mac SLST \| Whole loan discount rate \| Weighted Average
Liabilities
ABS issued, measurement input	0.06
Freddie Mac SLST \| Default rate \| Minimum
Liabilities
ABS issued, measurement input	0.13
Freddie Mac SLST \| Default rate \| Maximum
Liabilities
ABS issued, measurement input	0.14
Freddie Mac SLST \| Default rate \| Weighted Average
Liabilities
ABS issued, measurement input	0.14
Freddie Mac SLST \| Loss severity \| Minimum
Liabilities
ABS issued, measurement input	0.35
Freddie Mac SLST \| Loss severity \| Maximum
Liabilities
ABS issued, measurement input	0.35
Freddie Mac SLST \| Loss severity \| Weighted Average
Liabilities
ABS issued, measurement input	0.35
Freddie Mac K-Series
Liabilities
ABS issued \| $	$ 395,411
Freddie Mac K-Series \| Whole loan discount rate \| Minimum
Liabilities
ABS issued, measurement input	0.03
Freddie Mac K-Series \| Whole loan discount rate \| Maximum
Liabilities
ABS issued, measurement input	0.09
Freddie Mac K-Series \| Whole loan discount rate \| Weighted Average
Liabilities
ABS issued, measurement input	0.05
Consolidated Point Entities
Liabilities
ABS issued \| $	$ 104,751
Consolidated Point Entities \| Prepayment rate (annual CPR) \| Minimum
Liabilities
ABS issued, measurement input	0.20
Consolidated Point Entities \| Prepayment rate (annual CPR) \| Maximum
Liabilities
ABS issued, measurement input	0.20
Consolidated Point Entities \| Prepayment rate (annual CPR) \| Weighted Average
Liabilities
ABS issued, measurement input	0.20
Consolidated Point Entities \| Whole loan discount rate \| Minimum
Liabilities
ABS issued, measurement input	0.09
Consolidated Point Entities \| Whole loan discount rate \| Maximum
Liabilities
ABS issued, measurement input	0.15
Consolidated Point Entities \| Whole loan discount rate \| Weighted Average
Liabilities
ABS issued, measurement input	0.10
Consolidated Point Entities \| Home price appreciation \| Minimum
Liabilities
ABS issued, measurement input	(0.07)
Consolidated Point Entities \| Home price appreciation \| Maximum
Liabilities
ABS issued, measurement input	0.04
Consolidated Point Entities \| Home price appreciation \| Weighted Average
Liabilities
ABS issued, measurement input	0.03
Jumbo fixed-rate loans
Assets
ABS issued, net, At fair value \| $	$ 718,347
Jumbo fixed-rate loans \| Whole loan spread to swap rate \| Minimum
Assets
Loans held-for-sale, measurement input	0.0201
Jumbo fixed-rate loans \| Whole loan spread to swap rate \| Maximum
Assets
Loans held-for-sale, measurement input	0.0400
Jumbo fixed-rate loans \| Whole loan spread to swap rate \| Weighted Average
Assets
Loans held-for-sale, measurement input	0.0202
Jumbo fixed-rate loans \| CRT dollar price \| Minimum
Assets
Loans held-for-sale, measurement input	92
Jumbo fixed-rate loans \| CRT dollar price \| Maximum
Assets
Loans held-for-sale, measurement input	92
Jumbo fixed-rate loans \| CRT dollar price \| Weighted Average
Assets
Loans held-for-sale, measurement input	92
Jumbo loans committed to sell
Assets
Loans receivable, fair value \| $	$ 115,883
Jumbo loans committed to sell \| Whole loan committed sales price \| Minimum
Assets
Loans held-for-sale, measurement input	96
Jumbo loans committed to sell \| Whole loan committed sales price \| Maximum
Assets
Loans held-for-sale, measurement input	102
Jumbo loans committed to sell \| Whole loan committed sales price \| Weighted Average
Assets
Loans held-for-sale, measurement input	98
Legacy Sequoia
Assets
Loans receivable, fair value \| $	$ 198,160
Sequoia
Assets
Loans receivable, fair value \| $	3,237,170
Liabilities
Fair value of securities owned \| $	224,000
Freddie Mac SLST
Assets
Loans receivable, fair value \| $	1,482,964
Liabilities
Fair value of securities owned \| $	335,000	445,000
Single-family rental loans
Assets
Loans receivable, fair value \| $	$ 281,105
Single-family rental loans \| Senior credit spread \| Minimum
Assets
Loans receivable, measurement input	0.0210
Single-family rental loans \| Senior credit spread \| Maximum
Assets
Loans receivable, measurement input	0.0210
Single-family rental loans \| Senior credit spread \| Weighted Average
Assets
Loans receivable, measurement input	0.0210
Single-family rental loans \| Subordinate credit spread \| Minimum
Assets
Loans receivable, measurement input	0.0275
Single-family rental loans \| Subordinate credit spread \| Maximum
Assets
Loans receivable, measurement input	0.1025
Single-family rental loans \| Subordinate credit spread \| Weighted Average
Assets
Loans receivable, measurement input	0.0458
Single-family rental loans \| Senior credit support \| Minimum
Assets
Loans receivable, measurement input	0.36
Single-family rental loans \| Senior credit support \| Maximum
Assets
Loans receivable, measurement input	0.36
Single-family rental loans \| Senior credit support \| Weighted Average
Assets
Loans receivable, measurement input	0.36
Single-family rental loans \| IO discount rate \| Minimum
Assets
Loans receivable, measurement input	0.08
Single-family rental loans \| IO discount rate \| Maximum
Assets
Loans receivable, measurement input	0.09
Single-family rental loans \| IO discount rate \| Weighted Average
Assets
Loans receivable, measurement input	0.08
Single-family rental loans \| Prepayment rate (annual CPR) \| Minimum
Assets
Loans receivable, measurement input	0.03
Single-family rental loans \| Prepayment rate (annual CPR) \| Maximum
Assets
Loans receivable, measurement input	0.03
Single-family rental loans \| Prepayment rate (annual CPR) \| Weighted Average
Assets
Loans receivable, measurement input	0.03
Single-family rental loans \| Whole loan dollar price \| Minimum
Assets
Loans receivable, measurement input	84
Single-family rental loans \| Whole loan dollar price \| Maximum
Assets
Loans receivable, measurement input	99
Single-family rental loans \| Whole loan dollar price \| Weighted Average
Assets
Loans receivable, measurement input	86
CAFL
Assets
Loans receivable, fair value \| $	$ 3,018,994
Liabilities
Fair value of securities owned \| $	314,000	$ 302,000
Bridge loans
Assets
Loans receivable, fair value \| $	$ 1,957,119
Bridge loans \| Senior credit spread \| Minimum
Assets
Loans held-for-investment, measurement input	0.0285
Bridge loans \| Senior credit spread \| Maximum
Assets
Loans held-for-investment, measurement input	0.0285
Bridge loans \| Senior credit spread \| Weighted Average
Assets
Loans held-for-investment, measurement input	0.0285
Bridge loans \| Subordinate credit spread \| Minimum
Assets
Loans held-for-investment, measurement input	3.45
Bridge loans \| Subordinate credit spread \| Maximum
Assets
Loans held-for-investment, measurement input	12
Bridge loans \| Subordinate credit spread \| Weighted Average
Assets
Loans held-for-investment, measurement input	6.80
Bridge loans \| Senior credit support \| Minimum
Assets
Loans held-for-investment, measurement input	0.43
Bridge loans \| Senior credit support \| Maximum
Assets
Loans held-for-investment, measurement input	0.43
Bridge loans \| Senior credit support \| Weighted Average
Assets
Loans held-for-investment, measurement input	0.43
Bridge loans \| Prepayment rate (annual CPR) \| Minimum
Assets
Loans held-for-investment, measurement input	0
Bridge loans \| Prepayment rate (annual CPR) \| Maximum
Assets
Loans held-for-investment, measurement input	0
Bridge loans \| Prepayment rate (annual CPR) \| Weighted Average
Assets
Loans held-for-investment, measurement input	0
Bridge loans \| Whole loan discount rate \| Minimum
Assets
Loans held-for-investment, measurement input	0.05
Bridge loans \| Whole loan discount rate \| Maximum
Assets
Loans held-for-investment, measurement input	0.15
Bridge loans \| Whole loan discount rate \| Weighted Average
Assets
Loans held-for-investment, measurement input	0.09
Freddie Mac K-Series
Assets
Loans receivable, fair value \| $	$ 427,458
Liabilities
Fair value of securities owned \| $	32,000
Trading and AFS securities
Assets
Trading and AFS securities \| $	$ 259,212
Trading and AFS securities \| CRT dollar price \| Minimum
Assets
Trading and AFS securities	73
Trading and AFS securities \| CRT dollar price \| Maximum
Assets
Trading and AFS securities	93
Trading and AFS securities \| CRT dollar price \| Weighted Average
Assets
Trading and AFS securities	85
Trading and AFS securities \| Prepayment rate (annual CPR) \| Minimum
Assets
Trading and AFS securities	0.06
Trading and AFS securities \| Prepayment rate (annual CPR) \| Maximum
Assets
Trading and AFS securities	0.65
Trading and AFS securities \| Prepayment rate (annual CPR) \| Weighted Average
Assets
Trading and AFS securities	0.12
Trading and AFS securities \| Whole loan discount rate \| Minimum
Assets
Trading and AFS securities	0.05
Trading and AFS securities \| Whole loan discount rate \| Maximum
Assets
Trading and AFS securities	0.18
Trading and AFS securities \| Whole loan discount rate \| Weighted Average
Assets
Trading and AFS securities	0.10
Trading and AFS securities \| Default rate \| Minimum
Assets
Trading and AFS securities	0
Trading and AFS securities \| Default rate \| Maximum
Assets
Trading and AFS securities	0.12
Trading and AFS securities \| Default rate \| Weighted Average
Assets
Trading and AFS securities	0.004
Trading and AFS securities \| Loss severity \| Minimum
Assets
Trading and AFS securities	0
Trading and AFS securities \| Loss severity \| Maximum
Assets
Trading and AFS securities	0.50
Trading and AFS securities \| Loss severity \| Weighted Average
Assets
Trading and AFS securities	0.25
Servicer Advance Investments
Assets
Servicing asset \| $	$ 274,934
Servicer Advance Investments \| Minimum
Assets
Expected remaining life	5 years
Servicer Advance Investments \| Maximum
Assets
Expected remaining life	5 years
Servicer Advance Investments \| Weighted Average
Assets
Expected remaining life	5 years
Servicer Advance Investments \| Prepayment rate (annual CPR) \| Minimum
Assets
Servicing asset, measurement input	0.14
Servicer Advance Investments \| Prepayment rate (annual CPR) \| Maximum
Assets
Servicing asset, measurement input	0.30
Servicer Advance Investments \| Prepayment rate (annual CPR) \| Weighted Average
Assets
Servicing asset, measurement input	0.14
Servicer Advance Investments \| Whole loan discount rate \| Minimum
Assets
Servicing asset, measurement input	0.02
Servicer Advance Investments \| Whole loan discount rate \| Maximum
Assets
Servicing asset, measurement input	0.05
Servicer Advance Investments \| Whole loan discount rate \| Weighted Average
Assets
Servicing asset, measurement input	0.04
Servicer Advance Investments \| Mortgage servicing income \| Minimum
Assets
Servicing asset, measurement input	0
Servicer Advance Investments \| Mortgage servicing income \| Maximum
Assets
Servicing asset, measurement input	0.0018
Servicer Advance Investments \| Mortgage servicing income \| Weighted Average
Assets
Servicing asset, measurement input	0.0007
MSRs
Assets
Servicing asset \| $	$ 24,796
MSRs \| Prepayment rate (annual CPR) \| Minimum
Assets
Servicing asset, measurement input	0.04
MSRs \| Prepayment rate (annual CPR) \| Maximum
Assets
Servicing asset, measurement input	0.28
MSRs \| Prepayment rate (annual CPR) \| Weighted Average
Assets
Servicing asset, measurement input	0.08
MSRs \| Whole loan discount rate \| Minimum
Assets
Servicing asset, measurement input	0.11
MSRs \| Whole loan discount rate \| Maximum
Assets
Servicing asset, measurement input	0.69
MSRs \| Whole loan discount rate \| Weighted Average
Assets
Servicing asset, measurement input	0.11
MSRs \| Per loan annual cost to service \| Minimum
Assets
Servicing asset, measurement input	93
MSRs \| Per loan annual cost to service \| Maximum
Assets
Servicing asset, measurement input	93
MSRs \| Per loan annual cost to service \| Weighted Average
Assets
Servicing asset, measurement input	93
Excess MSRs
Assets
Servicing asset \| $	$ 40,452
Excess MSRs \| Prepayment rate (annual CPR) \| Minimum
Assets
Servicing asset, measurement input	0.10
Excess MSRs \| Prepayment rate (annual CPR) \| Maximum
Assets
Servicing asset, measurement input	1
Excess MSRs \| Prepayment rate (annual CPR) \| Weighted Average
Assets
Servicing asset, measurement input	0.18
Excess MSRs \| Whole loan discount rate \| Minimum
Assets
Servicing asset, measurement input	0.13
Excess MSRs \| Whole loan discount rate \| Maximum
Assets
Servicing asset, measurement input	0.19
Excess MSRs \| Whole loan discount rate \| Weighted Average
Assets
Servicing asset, measurement input	0.18
Excess MSRs \| Excess mortgage servicing income \| Minimum
Assets
Servicing asset, measurement input	0.0008
Excess MSRs \| Excess mortgage servicing income \| Maximum
Assets
Servicing asset, measurement input	0.0019
Excess MSRs \| Excess mortgage servicing income \| Weighted Average
Assets
Servicing asset, measurement input	0.0011
Home Appreciation Option
Assets
HEIs \| $	$ 200,709
Home Appreciation Option \| Prepayment rate (annual CPR) \| Minimum
Assets
HEIs, Measurement input	0.01
Home Appreciation Option \| Prepayment rate (annual CPR) \| Maximum
Assets
HEIs, Measurement input	0.23
Home Appreciation Option \| Prepayment rate (annual CPR) \| Weighted Average
Assets
HEIs, Measurement input	0.16
Home Appreciation Option \| Whole loan discount rate \| Minimum
Assets
HEIs, Measurement input	0.10
Home Appreciation Option \| Whole loan discount rate \| Maximum
Assets
HEIs, Measurement input	0.10
Home Appreciation Option \| Whole loan discount rate \| Weighted Average
Assets
HEIs, Measurement input	0.10
Home Appreciation Option \| Home price appreciation \| Minimum
Assets
HEIs, Measurement input	(0.07)
Home Appreciation Option \| Home price appreciation \| Maximum
Assets
HEIs, Measurement input	0.04
Home Appreciation Option \| Home price appreciation \| Weighted Average
Assets
HEIs, Measurement input	0.03
HEIs held by Point HEI entity
Assets
HEIs \| $	$ 139,728
Residential loan purchase commitments, net
Assets
Loan purchase commitments, net \| $	$ 531
Residential loan purchase commitments, net \| Whole loan spread to swap rate \| Minimum
Assets
Loans receivable, measurement input	0.0201
Residential loan purchase commitments, net \| Whole loan spread to swap rate \| Maximum
Assets
Loans receivable, measurement input	0.0201
Residential loan purchase commitments, net \| Whole loan spread to swap rate \| Weighted Average
Assets
Loans receivable, measurement input	0.0201
Residential loan purchase commitments, net \| Pull-through rate \| Minimum
Assets
Loans receivable, measurement input	0.26
Residential loan purchase commitments, net \| Pull-through rate \| Maximum
Assets
Loans receivable, measurement input	1
Residential loan purchase commitments, net \| Pull-through rate \| Weighted Average
Assets
Loans receivable, measurement input	0.78
Residential loan purchase commitments, net \| Committed sales price \| Minimum
Assets
Loans receivable, measurement input	99
Residential loan purchase commitments, net \| Committed sales price \| Maximum
Assets
Loans receivable, measurement input	102
Residential loan purchase commitments, net \| Committed sales price \| Weighted Average
Assets
Loans receivable, measurement input	100