Fair Value of Financial Instruments - Schedule of Portion of Net Gains (Losses) Attributable to Level 3 Assets and Liabilities Still Held Included in Net Income (Details) - USD ($) $ in Thousands	12 Months Ended
Dec. 31, 2023	Dec. 31, 2022	Dec. 31, 2021
Loan purchase and interest rate lock commitments
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gain (loss) attributable to Level 3 liability	$ (3,414)	$ (14)	$ (503)
Net investments in HEI securitization entities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gain (loss) attributable to Level 3 asset	4,368	2,916	218
Residential loans, held-for-investment, at fair value \| Residential loans at Redwood
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gain (loss) attributable to Level 3 asset	34,730	(43,019)	5,886
Residential loans, held-for-investment, at fair value \| Net investments in consolidated Sequoia entities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gain (loss) attributable to Level 3 asset	1,659	(25,563)	12,455
Residential loans, held-for-investment, at fair value \| Freddie Mac SLST
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gain (loss) attributable to Level 3 asset	(13,726)	(76,811)	62,124
Business purpose loans
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gain (loss) attributable to Level 3 asset	(20,251)	(31,927)	9,444
Multifamily securities \| Freddie Mac K-Series
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gain (loss) attributable to Level 3 asset	1,541	110	11,599
Single-family rental loans \| Net investments in consolidated CAFL Term entities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gain (loss) attributable to Level 3 asset	5,504	(34,899)	8,198
Trading securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gain (loss) attributable to Level 3 asset	3,155	(34,027)	738
AFS Securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gain (loss) attributable to Level 3 asset	59	(2,540)	0
Servicer advance investments
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gain (loss) attributable to Level 3 asset	11,863	(11,076)	(926)
MSRs
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gain (loss) attributable to Level 3 asset	498	9,804	629
Excess MSRs
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gain (loss) attributable to Level 3 asset	(1,668)	(5,196)	(8,017)
Shared home appreciation options
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gain (loss) attributable to Level 3 asset	24,194	(670)	212
Other investments
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gain (loss) attributable to Level 3 asset	(94)	(901)	(6)
Loan purchase and interest rate lock commitments
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Net gain (loss) attributable to Level 3 asset	$ 11,518	$ 336	$ 4,633