Commercial Mortgage Loan Portfolio (Details 2) - Commercial mortgage loans [Member] $ in Thousands	12 Months Ended
Dec. 31, 2015 USD ($) option extension	Dec. 31, 2014 USD ($) option
Condo Construction [Member] \| Condo Construction Bethesda, MD [Member]
Mortgage Loans on Real Estate [Line Items]
Option to extend loan agreement, term	6 months	6 months
Unfunded loan commitments	$ 15,100	$ 45,100
Condo Construction [Member] \| Condo Construction Potomac, MD [Member]
Mortgage Loans on Real Estate [Line Items]
Option to extend loan agreement, term	6 months	6 months
Unfunded loan commitments		$ 52,000
Hotel, Mixed Use, Multifamily and Vacation Home [Member]
Mortgage Loans on Real Estate [Line Items]
Option to extend loan agreement, term	1 year
Number of options to extend loan agreement \| extension	2
Hotel, Multifamily, and Vacation Home Portfolio [Member]
Mortgage Loans on Real Estate [Line Items]
Option to extend loan agreement, term		1 year
Number of options to extend loan agreement \| option		2
Mixed Use [Member] \| Mixed Use - Brooklyn, NY [Member]
Mortgage Loans on Real Estate [Line Items]
Unfunded loan commitments	$ 6,730
Mixed Use [Member] \| Mixed Use - Cincinnati, OH [Member]
Mortgage Loans on Real Estate [Line Items]
Option to extend loan agreement, term		1 year
Number of options to extend loan agreement \| option		2
Unfunded loan commitments		$ 145,000
Retail Redevelopment [Member] \| Retail Redevelopment - Miami, FL [Member]
Mortgage Loans on Real Estate [Line Items]
Option to extend loan agreement, term	6 months
Number of options to extend loan agreement \| option	2
Retail [Member] \| Retail, Brooklyn, NY [Member]
Mortgage Loans on Real Estate [Line Items]
Unfunded loan commitments	$ 9,000
Hotel [Member] \| Hotel - NY, NY [Member]
Mortgage Loans on Real Estate [Line Items]
Option to extend loan agreement, term	1 year
Number of options to extend loan agreement \| option	2
Unfunded loan commitments	$ 40,034
Hotel [Member] \| Hotel - U.S. Virgin Islands [Member]
Mortgage Loans on Real Estate [Line Items]
Option to extend loan agreement, term	1 year
Number of options to extend loan agreement \| option	3
Unfunded loan commitments	$ 1,500
Office [Member] \| Office - Richmond, VA [Member]
Mortgage Loans on Real Estate [Line Items]
Option to extend loan agreement, term	1 year
Number of options to extend loan agreement \| option	2
Unfunded loan commitments	$ 1,000
Condo Conversion [Member] \| Condo Conversion - NY, NY August 2013 [Member]
Mortgage Loans on Real Estate [Line Items]
Option to extend loan agreement, term		1 year
Condo Conversion [Member] \| Condo Conversion New York, NY [Member]
Mortgage Loans on Real Estate [Line Items]
Option to extend loan agreement, term		6 months
Multifamily Conversion [Member] \| Multifamily, Brooklyn, NY [Member]
Mortgage Loans on Real Estate [Line Items]
Option to extend loan agreement, term	1 year	1 year
Number of options to extend loan agreement \| option	3	3
Unfunded loan commitments		$ 4,500