Investment Fair Value Changes, Net - Components of Investment Activities (Details) - Investment Fair Value Changes, Net - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2017	Sep. 30, 2016	Sep. 30, 2017	Sep. 30, 2016
Investment Holdings [Line Items]
Changes in fair value of assets	$ 324	$ 11,918	$ 9,990	$ (18,686)
Residential loans held-for-investment, at Redwood
Investment Holdings [Line Items]
Changes in fair value of assets	2,881	(655)	8,902	22,161
Trading securities
Investment Holdings [Line Items]
Changes in fair value of assets	607	8,898	30,676	3,728
Legacy Sequoia
Investment Holdings [Line Items]
Changes in fair value of assets	(1,045)	(255)	(3,842)	(2,086)
Sequoia Choice
Investment Holdings [Line Items]
Changes in fair value of assets	(256)	0	(256)	0
Risk sharing investments
Investment Holdings [Line Items]
Changes in fair value of assets	(267)	15	(985)	(689)
Risk management derivatives, net
Investment Holdings [Line Items]
Changes in fair value of assets	(1,592)	4,222	(24,557)	(41,188)
Valuation adjustments on commercial loans held-for-sale
Investment Holdings [Line Items]
Changes in fair value of assets	0	(307)	300	(307)
Impairments on AFS securities
Investment Holdings [Line Items]
Changes in fair value of assets	$ (4)	$ 0	$ (248)	$ (305)