Loans Held for Investment and Loans Held for Investment at Fair Value - Activity in Allowance for Credit Losses (Details) - USD ($) $ in Thousands	3 Months Ended	6 Months Ended
Jun. 30, 2021	Jun. 30, 2020	Jun. 30, 2021	Jun. 30, 2020	Dec. 31, 2020
Allowance for credit losses:
Beginning balance	$ 5,881	$ 3,496	$ 5,845	$ 2,377
Provision for loan losses	(1,000)	1,800	(895)	3,088
Charge-offs	(918)	(76)	(987)	(245)
Ending balance	3,963	5,220	3,963	5,220
Allowance for Loans Individually Evaluated	1,949	1,273	1,949	1,273	$ 2,668
Allowance related to Loans collectively evaluated	2,014	3,947	2,014	3,947
Amortized cost related to Loans individually evaluated	318,974	270,772	318,974	270,772
Amortized cost related to Loans collectively evaluated	1,768,742	1,596,009	1,768,742	1,596,009
Beginning balance, prior to adoption of ASC 326				2,240
Impact of adopting ASC 326				137
Commercial Purchase Portfolio Segment
Allowance for credit losses:
Beginning balance	328	352	373	323
Provision for loan losses	48	172	3	280
Charge-offs	(114)		(114)	(79)
Ending balance	262	524	262	524
Allowance for Loans Individually Evaluated	57	181	57	181	153
Allowance related to Loans collectively evaluated	205	342	205	342
Amortized cost related to Loans individually evaluated	19,314	20,615	19,314	20,615
Amortized cost related to Loans collectively evaluated	347,423	282,759	347,423	282,759
Beginning balance, prior to adoption of ASC 326				304
Impact of adopting ASC 326				19
Commercial Refinance Portfolio Segment
Allowance for credit losses:
Beginning balance	1,990	1,594	2,093	1,078
Provision for loan losses	28	1,093	(76)	1,609
Charge-offs	(95)		(94)
Ending balance	1,923	2,687	1,923	2,687
Allowance for Loans Individually Evaluated	1,005	592	1,005	592	519
Allowance related to Loans collectively evaluated	918	2,095	918	2,095
Amortized cost related to Loans individually evaluated	92,397	113,350	92,397	113,350
Amortized cost related to Loans collectively evaluated	598,567	598,046	598,567	598,046
Beginning balance, prior to adoption of ASC 326				1,016
Impact of adopting ASC 326				62
Residential One To Four Unit Purchase Portfolio Segment
Allowance for credit losses:
Beginning balance	322	250	333	157
Provision for loan losses	(41)	211	(15)	382
Charge-offs		(58)	(37)	(136)
Ending balance	281	403	281	403
Allowance for Loans Individually Evaluated	120	96	120	96	128
Allowance related to Loans collectively evaluated	161	307	161	307
Amortized cost related to Loans individually evaluated	21,084	22,017	21,084	22,017
Amortized cost related to Loans collectively evaluated	255,812	223,561	255,812	223,561
Beginning balance, prior to adoption of ASC 326				148
Impact of adopting ASC 326				9
Residential One To Four Unit Refinance Portfolio Segment
Allowance for credit losses:
Beginning balance	1,227	1,300	1,216	819
Provision for loan losses	(229)	324	(218)	817
Charge-offs	(151)	(18)	(151)	(30)
Ending balance	847	1,606	847	1,606
Allowance for Loans Individually Evaluated	342	404	342	404	465
Allowance related to Loans collectively evaluated	507	1,203	507	1,203
Amortized cost related to Loans individually evaluated	112,294	114,790	112,294	114,790
Amortized cost related to Loans collectively evaluated	496,249	$ 491,643	496,249	491,643
Beginning balance, prior to adoption of ASC 326				772
Impact of adopting ASC 326				$ 47
Quick Fix One To Four Unit Purchase Portfolio Segment
Allowance for credit losses:
Beginning balance	386		595
Provision for loan losses	(313)		(508)
Charge-offs			(14)
Ending balance	73		73
Allowance for Loans Individually Evaluated	56		56		525
Allowance related to Loans collectively evaluated	16		16
Amortized cost related to Loans individually evaluated	23,883		23,883
Amortized cost related to Loans collectively evaluated	16,322		16,322
Quick Fix One To Four Unit Refinance Portfolio Segment
Allowance for credit losses:
Beginning balance	1,628		1,235
Provision for loan losses	(493)		(81)
Charge-offs	(558)		(577)
Ending balance	577		577
Allowance for Loans Individually Evaluated	369		369		$ 878
Allowance related to Loans collectively evaluated	207		207
Amortized cost related to Loans individually evaluated	50,002		50,002
Amortized cost related to Loans collectively evaluated	$ 54,369		$ 54,369

Amount of allowance for credit loss on financing receivable. Excludes allowance for financing receivable covered under loss sharing agreement.

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 310
-SubTopic 10
-Section 50
-Paragraph 11B
-Subparagraph (c)(1)
-URI http://asc.fasb.org/extlink&oid=121611835&loc=SL6953423-111524

Reference 2: http://www.xbrl.org/2009/role/commonPracticeRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 942
-SubTopic 210
-Section S99
-Paragraph 1
-Subparagraph (SX 210.9-03(7)(2))
-URI http://asc.fasb.org/extlink&oid=120398452&loc=d3e534808-122878

Reference 3: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 326
-SubTopic 20
-Section 45
-Paragraph 1
-URI http://asc.fasb.org/extlink&oid=121599878&loc=SL82895884-210446

Reference 4: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 326
-SubTopic 20
-Section 50
-Paragraph 13
-Subparagraph (a)
-URI http://asc.fasb.org/extlink&oid=121599337&loc=SL82919249-210447

Reference 5: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 326
-SubTopic 20
-Section 50
-Paragraph 13
-Subparagraph (f)
-URI http://asc.fasb.org/extlink&oid=121599337&loc=SL82919249-210447

+ Details

Name:	us-gaap_FinancingReceivableAllowanceForCreditLosses
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

The valuation allowance for financing receivables that are expected to be uncollectible that were collectively evaluated for impairment.

+ References

+ Details

Name:	us-gaap_FinancingReceivableAllowanceForCreditLossesCollectivelyEvaluatedForImpairment
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

Amount of valuation allowance for financing receivables that are expected to be uncollectible that were individually evaluated for impairment.

+ References

+ Details

Name:	us-gaap_FinancingReceivableAllowanceForCreditLossesIndividuallyEvaluatedForImpairment1
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

A roll forward is a reconciliation of a concept from the beginning of a period to the end of a period.

+ References

No definition available.

+ Details

Name:	us-gaap_FinancingReceivableAllowanceForCreditLossesRollForward
Namespace Prefix:	us-gaap_
Data Type:	xbrli:stringItemType
Balance Type:	na
Period Type:	duration

- Definition

Amount of writeoff of financing receivable, charged against allowance for credit loss.

+ References

Reference 1: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 326
-SubTopic 20
-Section 50
-Paragraph 13
-Subparagraph (d)
-URI http://asc.fasb.org/extlink&oid=121599337&loc=SL82919249-210447

Reference 2: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 310
-SubTopic 10
-Section 50
-Paragraph 11B
-Subparagraph (c)(3)
-URI http://asc.fasb.org/extlink&oid=121611835&loc=SL6953423-111524

Reference 3: http://www.xbrl.org/2003/role/exampleRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 326
-SubTopic 20
-Section 55
-Paragraph 79
-URI http://asc.fasb.org/extlink&oid=121647567&loc=SL82922352-210448

+ Details

Name:	us-gaap_FinancingReceivableAllowanceForCreditLossesWriteOffs
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	duration

- Definition

Amount of credit loss expense (reversal of expense) for financing receivable.

+ References

Reference 1: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 326
-SubTopic 20
-Section 50
-Paragraph 13
-Subparagraph (b)
-URI http://asc.fasb.org/extlink&oid=121599337&loc=SL82919249-210447

Reference 2: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 942
-SubTopic 220
-Section S99
-Paragraph 1
-Subparagraph (SX 210.9-04(11))
-URI http://asc.fasb.org/extlink&oid=120399700&loc=SL114874048-224260

+ Details

Name:	us-gaap_ProvisionForLoanLossesExpensed
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	duration

- Definition

Financing receivable allowance for credit losses impact of adopting accounting standards update.

+ References

No definition available.

+ Details

Name:	vel_FinancingReceivableAllowanceForCreditLossesImpactOfAdoptingAccountingStandardsUpdate
Namespace Prefix:	vel_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	duration

- Definition

Financing receivable allowance for credit losses prior to adoption of accounting standards update.

+ References

No definition available.

+ Details

Name:	vel_FinancingReceivableAllowanceForCreditLossesPriorToAdoptionOfAccountingStandardsUpdate
Namespace Prefix:	vel_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

Amortized cost of financing receivable collectively evaluated for impairment.

+ References

No definition available.

+ Details

Name:	vel_FinancingReceivableCollectivelyEvaluatedForImpairmentAmortizedCost123
Namespace Prefix:	vel_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Amortized cost of financing receivable individually evaluated for impairment.

+ References

No definition available.

+ Details

Name:	vel_FinancingReceivableIndividuallyEvaluatedForImpairmentAmortizedCost123
Namespace Prefix:	vel_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Details

Name:	us-gaap_FinancingReceivablePortfolioSegmentAxis=vel_CommercialPurchasePortfolioSegmentMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FinancingReceivablePortfolioSegmentAxis=vel_CommercialRefinancePortfolioSegmentMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FinancingReceivablePortfolioSegmentAxis=vel_ResidentialOneToFourUnitPurchasePortfolioSegmentMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FinancingReceivablePortfolioSegmentAxis=vel_ResidentialOneToFourUnitRefinancePortfolioSegmentMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FinancingReceivablePortfolioSegmentAxis=vel_QuickFixOneToFourUnitPurchasePortfolioSegmentMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FinancingReceivablePortfolioSegmentAxis=vel_QuickFixOneToFourUnitRefinancePortfolioSegmentMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type: