Loans Held for Investment and Loans Held for Investment at Fair Value - Schedule of Amortized Cost in Loans Held for Investment based on Accrual Status and by Loan Origination Year (Details) - Accrual Status - USD ($) $ in Thousands	3 Months Ended	9 Months Ended
Sep. 30, 2023	Sep. 30, 2023	Dec. 31, 2022
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2023			$ 1,243,841
Term Loans Amortized Cost Basis by Origination Year, 2022	$ 1,149,712	$ 1,149,712	1,033,695
Term Loans Amortized Cost Basis by Origination Year, 2021	943,861	943,861	168,977
Term Loans Amortized Cost Basis by Origination Year, 2020	153,806	153,806	358,100
Term Loans Amortized Cost Basis by Origination Year, 2019	301,119	301,119	231,024
Term Loans Amortized Cost Basis by Origination Year, Pre-2019	402,027	402,027	241,647
Total	2,950,525	2,950,525	3,277,284
Term Loans Amortized Cost Basis by Origination Year, 2022, Gross Charge-offs	365	376
Term Loans Amortized Cost Basis by Origination Year, 2021, Gross Charge-offs	(270)	473
Term Loans Amortized Cost Basis by Origination Year, 2019, Gross Charge-offs		447
Gross Charge-offs, Total	95	1,296
Commercial - Purchase
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2023			275,224
Term Loans Amortized Cost Basis by Origination Year, 2022	264,569	264,569	256,059
Term Loans Amortized Cost Basis by Origination Year, 2021	236,713	236,713	37,643
Term Loans Amortized Cost Basis by Origination Year, 2020	33,114	33,114	62,131
Term Loans Amortized Cost Basis by Origination Year, 2019	50,374	50,374	36,398
Term Loans Amortized Cost Basis by Origination Year, Pre-2019	62,354	62,354	33,953
Total	647,124	647,124	701,408
Commercial - Purchase \| Performing
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2023			273,950
Term Loans Amortized Cost Basis by Origination Year, 2022	258,084	258,084	249,100
Term Loans Amortized Cost Basis by Origination Year, 2021	230,472	230,472	36,064
Term Loans Amortized Cost Basis by Origination Year, 2020	32,582	32,582	56,322
Term Loans Amortized Cost Basis by Origination Year, 2019	46,101	46,101	33,193
Term Loans Amortized Cost Basis by Origination Year, Pre-2019	55,872	55,872	30,208
Total	623,111	623,111	678,837
Commercial - Purchase \| Nonperforming
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2023			1,274
Term Loans Amortized Cost Basis by Origination Year, 2022	6,485	6,485	6,959
Term Loans Amortized Cost Basis by Origination Year, 2021	6,241	6,241	1,579
Term Loans Amortized Cost Basis by Origination Year, 2020	532	532	5,809
Term Loans Amortized Cost Basis by Origination Year, 2019	4,273	4,273	3,205
Term Loans Amortized Cost Basis by Origination Year, Pre-2019	6,482	6,482	3,745
Total	24,013	24,013	22,571
Commercial - Refinance
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2023			272,766
Term Loans Amortized Cost Basis by Origination Year, 2022	262,969	262,969	222,699
Term Loans Amortized Cost Basis by Origination Year, 2021	208,331	208,331	59,650
Term Loans Amortized Cost Basis by Origination Year, 2020	55,104	55,104	127,056
Term Loans Amortized Cost Basis by Origination Year, 2019	110,178	110,178	113,569
Term Loans Amortized Cost Basis by Origination Year, Pre-2019	191,455	191,455	111,357
Total	828,037	828,037	907,097
Commercial - Refinance \| Performing
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2023			263,754
Term Loans Amortized Cost Basis by Origination Year, 2022	240,627	240,627	210,898
Term Loans Amortized Cost Basis by Origination Year, 2021	195,968	195,968	55,795
Term Loans Amortized Cost Basis by Origination Year, 2020	50,490	50,490	103,633
Term Loans Amortized Cost Basis by Origination Year, 2019	92,007	92,007	93,161
Term Loans Amortized Cost Basis by Origination Year, Pre-2019	155,865	155,865	92,723
Total	734,957	734,957	819,964
Commercial - Refinance \| Nonperforming
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2023			9,012
Term Loans Amortized Cost Basis by Origination Year, 2022	22,342	22,342	11,801
Term Loans Amortized Cost Basis by Origination Year, 2021	12,363	12,363	3,855
Term Loans Amortized Cost Basis by Origination Year, 2020	4,614	4,614	23,423
Term Loans Amortized Cost Basis by Origination Year, 2019	18,171	18,171	20,408
Term Loans Amortized Cost Basis by Origination Year, Pre-2019	35,590	35,590	18,634
Total	93,080	93,080	87,133
Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2023			256,906
Term Loans Amortized Cost Basis by Origination Year, 2022	237,538	237,538	237,342
Term Loans Amortized Cost Basis by Origination Year, 2021	215,349	215,349	12,875
Term Loans Amortized Cost Basis by Origination Year, 2020	10,798	10,798	33,998
Term Loans Amortized Cost Basis by Origination Year, 2019	27,133	27,133	20,444
Term Loans Amortized Cost Basis by Origination Year, Pre-2019	41,215	41,215	26,868
Total	532,033	532,033	588,433
Residential 1-4 Unit - Purchase \| Performing
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2023			249,625
Term Loans Amortized Cost Basis by Origination Year, 2022	217,835	217,835	227,235
Term Loans Amortized Cost Basis by Origination Year, 2021	203,026	203,026	10,710
Term Loans Amortized Cost Basis by Origination Year, 2020	9,775	9,775	31,685
Term Loans Amortized Cost Basis by Origination Year, 2019	25,710	25,710	18,891
Term Loans Amortized Cost Basis by Origination Year, Pre-2019	35,452	35,452	22,303
Total	491,798	491,798	560,449
Residential 1-4 Unit - Purchase \| Nonperforming
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2023			7,281
Term Loans Amortized Cost Basis by Origination Year, 2022	19,703	19,703	10,107
Term Loans Amortized Cost Basis by Origination Year, 2021	12,323	12,323	2,165
Term Loans Amortized Cost Basis by Origination Year, 2020	1,023	1,023	2,313
Term Loans Amortized Cost Basis by Origination Year, 2019	1,423	1,423	1,553
Term Loans Amortized Cost Basis by Origination Year, Pre-2019	5,763	5,763	4,565
Total	40,235	40,235	27,984
Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2023			360,350
Term Loans Amortized Cost Basis by Origination Year, 2022	336,826	336,826	310,218
Term Loans Amortized Cost Basis by Origination Year, 2021	282,605	282,605	31,610
Term Loans Amortized Cost Basis by Origination Year, 2020	27,850	27,850	111,954
Term Loans Amortized Cost Basis by Origination Year, 2019	95,507	95,507	55,704
Term Loans Amortized Cost Basis by Origination Year, Pre-2019	102,852	102,852	69,469
Total	845,640	845,640	939,305
Residential 1-4 Unit - Refinance \| Performing
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2023			338,959
Term Loans Amortized Cost Basis by Origination Year, 2022	291,195	291,195	285,195
Term Loans Amortized Cost Basis by Origination Year, 2021	248,864	248,864	24,703
Term Loans Amortized Cost Basis by Origination Year, 2020	20,577	20,577	84,208
Term Loans Amortized Cost Basis by Origination Year, 2019	67,145	67,145	39,870
Term Loans Amortized Cost Basis by Origination Year, Pre-2019	78,908	78,908	52,461
Total	706,689	706,689	825,396
Residential 1-4 Unit - Refinance \| Nonperforming
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2023			21,391
Term Loans Amortized Cost Basis by Origination Year, 2022	45,631	45,631	25,023
Term Loans Amortized Cost Basis by Origination Year, 2021	33,741	33,741	6,907
Term Loans Amortized Cost Basis by Origination Year, 2020	7,273	7,273	27,746
Term Loans Amortized Cost Basis by Origination Year, 2019	28,362	28,362	15,834
Term Loans Amortized Cost Basis by Origination Year, Pre-2019	23,944	23,944	17,008
Total	138,951	138,951	113,909
Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2023			42,254
Term Loans Amortized Cost Basis by Origination Year, 2022	21,039	21,039	6,156
Term Loans Amortized Cost Basis by Origination Year, 2021	710	710	16,654
Term Loans Amortized Cost Basis by Origination Year, 2020	18,461	18,461	4,716
Term Loans Amortized Cost Basis by Origination Year, 2019	4,406	4,406	104
Total	44,616	44,616	69,884
Short Term 1-4 Unit - Purchase \| Performing
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2023			40,967
Term Loans Amortized Cost Basis by Origination Year, 2022	14,760	14,760	944
Term Loans Amortized Cost Basis by Origination Year, 2021	180	180	15,659
Term Loans Amortized Cost Basis by Origination Year, 2020	17,466	17,466	4,174
Term Loans Amortized Cost Basis by Origination Year, 2019	4,406	4,406
Total	36,812	36,812	61,744
Short Term 1-4 Unit - Purchase \| Nonperforming
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2023			1,287
Term Loans Amortized Cost Basis by Origination Year, 2022	6,279	6,279	5,212
Term Loans Amortized Cost Basis by Origination Year, 2021	530	530	995
Term Loans Amortized Cost Basis by Origination Year, 2020	995	995	542
Term Loans Amortized Cost Basis by Origination Year, 2019			104
Total	7,804	7,804	8,140
Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2023			36,341
Term Loans Amortized Cost Basis by Origination Year, 2022	26,771	26,771	1,221
Term Loans Amortized Cost Basis by Origination Year, 2021	153	153	10,545
Term Loans Amortized Cost Basis by Origination Year, 2020	8,479	8,479	18,245
Term Loans Amortized Cost Basis by Origination Year, 2019	13,521	13,521	4,805
Term Loans Amortized Cost Basis by Origination Year, Pre-2019	4,151	4,151
Total	53,075	53,075	71,157
Short Term 1-4 Unit - Refinance \| Performing
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2023			35,555
Term Loans Amortized Cost Basis by Origination Year, 2022	9,858	9,858
Total	9,858	9,858	35,555
Short Term 1-4 Unit - Refinance \| Nonperforming
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2023			786
Term Loans Amortized Cost Basis by Origination Year, 2022	16,913	16,913	1,221
Term Loans Amortized Cost Basis by Origination Year, 2021	153	153	10,545
Term Loans Amortized Cost Basis by Origination Year, 2020	8,479	8,479	18,245
Term Loans Amortized Cost Basis by Origination Year, 2019	13,521	13,521	4,805
Term Loans Amortized Cost Basis by Origination Year, Pre-2019	4,151	4,151
Total	$ 43,217	$ 43,217	$ 35,602

Loans Held for Investment and Loans Held for Investment at Fair Value - Schedule of Amortized Cost in Loans Held for Investment based on Accrual Status and by Loan Origination Year (Details) - Accrual Status - USD ($)
$ in Thousands

3 Months Ended

9 Months Ended

Sep. 30, 2023

Dec. 31, 2022

Financing Receivable Recorded Investment [Line Items]

Term Loans Amortized Cost Basis by Origination Year, 2023

$ 1,243,841

Term Loans Amortized Cost Basis by Origination Year, 2022

$ 1,149,712

1,033,695

Term Loans Amortized Cost Basis by Origination Year, 2021

943,861