Investment Fair Value Changes, Net - Components of Investment Activities (Details) - Investment Fair Value Changes, Net - USD ($) $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2018	Jun. 30, 2017	Jun. 30, 2018	Jun. 30, 2017
Investment Holdings [Line Items]
Changes in fair value of assets	$ 889	$ 8,115	$ 2,498	$ 9,666
Residential loans held-for-investment, at Redwood
Investment Holdings [Line Items]
Changes in fair value of assets	(15,010)	8,354	(53,995)	6,021
Trading securities
Investment Holdings [Line Items]
Changes in fair value of assets	(930)	18,926	(3,885)	30,069
Legacy Sequoia
Investment Holdings [Line Items]
Changes in fair value of assets	(720)	(987)	(728)	(2,797)
Sequoia Choice
Investment Holdings [Line Items]
Changes in fair value of assets	1,072	0	986	0
Risk-sharing investments
Investment Holdings [Line Items]
Changes in fair value of assets	(209)	(513)	(348)	(718)
Risk management derivatives, net
Investment Holdings [Line Items]
Changes in fair value of assets	16,742	(17,838)	60,524	(22,965)
Commercial loan
Investment Holdings [Line Items]
Changes in fair value of assets	0	300	0	300
Impairments on AFS securities
Investment Holdings [Line Items]
Changes in fair value of assets	$ (56)	$ (127)	$ (56)	$ (244)