<?xml version="1.0" encoding="us-ascii"?><InstanceReport xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><Version>2.4.0.8</Version><ReportLongName>2410401 - Disclosure - Concentration of Risk (Details)</ReportLongName><DisplayLabelColumn>true</DisplayLabelColumn><ShowElementNames>false</ShowElementNames><RoundingOption /><HasEmbeddedReports>false</HasEmbeddedReports><Columns><Column FlagID="0"><Id>1</Id><IsAbstractGroupTitle>false</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><CurrencyCode /><FootnoteIndexer /><hasSegments>false</hasSegments><hasScenarios>false</hasScenarios><MCU><KeyName /><CurrencySymbol /><contextRef><ContextID>D2013Q2YTD</ContextID><EntitySchema>http://www.sec.gov/CIK</EntitySchema><EntityValue>0000910108</EntityValue><PeriodDisplayName /><PeriodType>duration</PeriodType><PeriodStartDate>2013-01-01T00:00:00</PeriodStartDate><PeriodEndDate>2013-06-30T00:00:00</PeriodEndDate><Segments /><Scenarios /></contextRef><UPS><UnitProperty><UnitID>Tenants</UnitID><UnitType>Standard</UnitType><StandardMeasure><MeasureSchema>http://www.lxp.com/20130630</MeasureSchema><MeasureValue>tenants</MeasureValue><MeasureNamespace>lxp</MeasureNamespace><MeasureDescription /><MeasureType /></StandardMeasure><MultiplyMeasures><Measure><MeasureSchema /><MeasureValue /><MeasureNamespace /><MeasureDescription /><MeasureType /></Measure><Measure><MeasureSchema /><MeasureValue /><MeasureNamespace /><MeasureDescription /><MeasureType /></Measure></MultiplyMeasures><NumeratorMeasure><MeasureSchema /><MeasureValue /><MeasureNamespace /><MeasureDescription /><MeasureType /></NumeratorMeasure><DenominatorMeasure><MeasureSchema /><MeasureValue /><MeasureNamespace /><MeasureDescription /><MeasureType /></DenominatorMeasure><Scale>0</Scale></UnitProperty></UPS><CurrencyCode /><OriginalCurrencyCode /></MCU><CurrencySymbol /><Labels><Label Key="CalendarSupplement" Id="0" Label="6 Months Ended" /><Label Key="Calendar" Id="1" Label="Jun. 30, 2013" /><Label Key="" Id="2" Label="tenants" /></Labels></Column><Column FlagID="0"><Id>2</Id><IsAbstractGroupTitle>false</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><CurrencyCode /><FootnoteIndexer /><hasSegments>false</hasSegments><hasScenarios>false</hasScenarios><MCU><KeyName /><CurrencySymbol /><contextRef><ContextID>D2012Q2YTD</ContextID><EntitySchema>http://www.sec.gov/CIK</EntitySchema><EntityValue>0000910108</EntityValue><PeriodDisplayName /><PeriodType>duration</PeriodType><PeriodStartDate>2012-01-01T00:00:00</PeriodStartDate><PeriodEndDate>2012-06-30T00:00:00</PeriodEndDate><Segments /><Scenarios /></contextRef><UPS><UnitProperty><UnitID>Tenants</UnitID><UnitType>Standard</UnitType><StandardMeasure><MeasureSchema>http://www.lxp.com/20130630</MeasureSchema><MeasureValue>tenants</MeasureValue><MeasureNamespace>lxp</MeasureNamespace><MeasureDescription /><MeasureType /></StandardMeasure><MultiplyMeasures><Measure><MeasureSchema /><MeasureValue /><MeasureNamespace /><MeasureDescription /><MeasureType /></Measure><Measure><MeasureSchema /><MeasureValue /><MeasureNamespace /><MeasureDescription /><MeasureType /></Measure></MultiplyMeasures><NumeratorMeasure><MeasureSchema /><MeasureValue /><MeasureNamespace /><MeasureDescription /><MeasureType /></NumeratorMeasure><DenominatorMeasure><MeasureSchema /><MeasureValue /><MeasureNamespace /><MeasureDescription /><MeasureType /></DenominatorMeasure><Scale>0</Scale></UnitProperty></UPS><CurrencyCode /><OriginalCurrencyCode /></MCU><CurrencySymbol /><Labels><Label Key="CalendarSupplement" Id="0" Label="6 Months Ended" /><Label Key="Calendar" Id="1" Label="Jun. 30, 2012" /><Label Key="" Id="2" Label="tenants" /></Labels></Column></Columns><Rows><Row FlagID="0"><Id>1</Id><IsAbstractGroupTitle>true</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><Level>3</Level><ElementName>us-gaap_ConcentrationRiskLineItems</ElementName><ElementPrefix>us-gaap_</ElementPrefix><IsBaseElement>true</IsBaseElement><BalanceType>na</BalanceType><PeriodType>duration</PeriodType><IsReportTitle>false</IsReportTitle><IsSegmentTitle>false</IsSegmentTitle><IsCalendarTitle>false</IsCalendarTitle><IsEquityPrevioslyReportedAsRow>false</IsEquityPrevioslyReportedAsRow><IsEquityAdjustmentRow>false</IsEquityAdjustmentRow><IsBeginningBalance>false</IsBeginningBalance><IsEndingBalance>false</IsEndingBalance><IsReverseSign>false</IsReverseSign><FootnoteIndexer /><Cells><Cell FlagID="0" ContextID="" UnitID=""><Id>1</Id><IsNumeric>false</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>0</NumericAmount><RoundedNumericAmount>0</RoundedNumericAmount><NonNumbericText /><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell><Cell FlagID="0" ContextID="" UnitID=""><Id>2</Id><IsNumeric>false</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>0</NumericAmount><RoundedNumericAmount>0</RoundedNumericAmount><NonNumbericText /><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell></Cells><ElementDataType>xbrli:stringItemType</ElementDataType><SimpleDataType>string</SimpleDataType><IsTotalLabel>false</IsTotalLabel><UnitID>0</UnitID><Label>Concentration Risk [Line Items]</Label></Row><Row FlagID="0"><Id>2</Id><IsAbstractGroupTitle>false</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><Level>4</Level><ElementName>lxp_NumberOfTenantsRepresentingMoreThan10PercentOfRentalRevenue</ElementName><ElementPrefix>lxp_</ElementPrefix><IsBaseElement>false</IsBaseElement><BalanceType>na</BalanceType><PeriodType>duration</PeriodType><IsReportTitle>false</IsReportTitle><IsSegmentTitle>false</IsSegmentTitle><IsCalendarTitle>false</IsCalendarTitle><IsEquityPrevioslyReportedAsRow>false</IsEquityPrevioslyReportedAsRow><IsEquityAdjustmentRow>false</IsEquityAdjustmentRow><IsBeginningBalance>false</IsBeginningBalance><IsEndingBalance>false</IsEndingBalance><IsReverseSign>false</IsReverseSign><PreferredLabelRole>terseLabel</PreferredLabelRole><FootnoteIndexer /><Cells><Cell FlagID="0" ContextID="D2013Q2YTD" UnitID="Tenants"><Id>1</Id><IsNumeric>true</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>0</NumericAmount><RoundedNumericAmount>0</RoundedNumericAmount><NonNumbericText /><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell><Cell FlagID="0" ContextID="D2012Q2YTD" UnitID="Tenants"><Id>2</Id><IsNumeric>true</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>0</NumericAmount><RoundedNumericAmount>0</RoundedNumericAmount><NonNumbericText /><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell></Cells><ElementDataType>xbrli:integerItemType</ElementDataType><SimpleDataType>integer</SimpleDataType><ElementDefenition>Number of Tenants Representing More Than 10 Percent of Rental Revenue</ElementDefenition><ElementReferences>No definition available.</ElementReferences><IsTotalLabel>false</IsTotalLabel><UnitID>256</UnitID><Label>Number of tenents representing more than 10% of rental revenue</Label></Row><Row FlagID="0"><Id>3</Id><IsAbstractGroupTitle>false</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><Level>0</Level><ElementName /><ElementPrefix /><IsBaseElement>true</IsBaseElement><BalanceType /><PeriodType /><IsReportTitle>false</IsReportTitle><IsSegmentTitle>true</IsSegmentTitle><IsCalendarTitle>false</IsCalendarTitle><IsEquityPrevioslyReportedAsRow>false</IsEquityPrevioslyReportedAsRow><IsEquityAdjustmentRow>false</IsEquityAdjustmentRow><IsBeginningBalance>false</IsBeginningBalance><IsEndingBalance>false</IsEndingBalance><IsReverseSign>false</IsReverseSign><PreferredLabelRole /><FootnoteIndexer /><Cells><Cell FlagID="0" ContextID="" UnitID=""><Id>1</Id><IsNumeric>false</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>0</NumericAmount><RoundedNumericAmount>0</RoundedNumericAmount><NonNumbericText /><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell><Cell FlagID="0" ContextID="" UnitID=""><Id>2</Id><IsNumeric>false</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>0</NumericAmount><RoundedNumericAmount>0</RoundedNumericAmount><NonNumbericText /><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell></Cells><OriginalInstanceReportColumn FlagID="0"><Id>3</Id><IsAbstractGroupTitle>false</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><CurrencyCode /><FootnoteIndexer /><hasSegments>true</hasSegments><hasScenarios>false</hasScenarios><MCU><KeyName /><CurrencySymbol /><contextRef><ContextID>D2013Q2YTD_us-gaap_ConcentrationRiskByTypeAxis_lxp_TenantConcentrationRiskMember_us-gaap_RangeAxis_us-gaap_MaximumMember</ContextID><EntitySchema>http://www.sec.gov/CIK</EntitySchema><EntityValue>0000910108</EntityValue><PeriodDisplayName /><PeriodType>duration</PeriodType><PeriodStartDate>2013-01-01T00:00:00</PeriodStartDate><PeriodEndDate>2013-06-30T00:00:00</PeriodEndDate><Segments><anyType xsi:type="Segment"><IsDefaultForEntity>false</IsDefaultForEntity><Name /><IsFromEntityGroup>false</IsFromEntityGroup><ValueName>Maximum [Member]</ValueName><ValueType>us-gaap_RangeAxis</ValueType><Namespace>xbrldi</Namespace><Schema>http://xbrl.org/2006/xbrldi</Schema><DimensionInfo><Id>us-gaap_MaximumMember</Id><dimensionId>us-gaap_RangeAxis</dimensionId><type>explicitMember</type></DimensionInfo></anyType><anyType xsi:type="Segment"><IsDefaultForEntity>false</IsDefaultForEntity><Name /><IsFromEntityGroup>false</IsFromEntityGroup><ValueName>Tenant Concentration Risk [Member]</ValueName><ValueType>us-gaap_ConcentrationRiskByTypeAxis</ValueType><Namespace>xbrldi</Namespace><Schema>http://xbrl.org/2006/xbrldi</Schema><DimensionInfo><Id>lxp_TenantConcentrationRiskMember</Id><dimensionId>us-gaap_ConcentrationRiskByTypeAxis</dimensionId><type>explicitMember</type></DimensionInfo></anyType></Segments><Scenarios /></contextRef><UPS><UnitProperty><UnitID>number</UnitID><UnitType>Standard</UnitType><StandardMeasure><MeasureSchema>http://www.xbrl.org/2003/instance</MeasureSchema><MeasureValue>pure</MeasureValue><MeasureNamespace>xbrli</MeasureNamespace><MeasureDescription /><MeasureType /></StandardMeasure><MultiplyMeasures><Measure><MeasureSchema /><MeasureValue /><MeasureNamespace /><MeasureDescription /><MeasureType /></Measure><Measure><MeasureSchema /><MeasureValue /><MeasureNamespace /><MeasureDescription /><MeasureType /></Measure></MultiplyMeasures><NumeratorMeasure><MeasureSchema /><MeasureValue /><MeasureNamespace /><MeasureDescription /><MeasureType /></NumeratorMeasure><DenominatorMeasure><MeasureSchema /><MeasureValue /><MeasureNamespace /><MeasureDescription /><MeasureType /></DenominatorMeasure><Scale>0</Scale></UnitProperty></UPS><CurrencyCode /><OriginalCurrencyCode /></MCU><CurrencySymbol /><Labels><Label Key="us-gaap_RangeAxis:us-gaap_MaximumMember" Id="0" Label="Maximum [Member]" /><Label Key="us-gaap_ConcentrationRiskByTypeAxis:lxp_TenantConcentrationRiskMember" Id="1" Label="Tenant Concentration Risk [Member]" /><Label Key="Calendar" Id="2" Label="1/1/2013 - 6/30/2013" /></Labels></OriginalInstanceReportColumn><ElementDataType>na</ElementDataType><SimpleDataType>na</SimpleDataType><IsTotalLabel>false</IsTotalLabel><UnitID>0</UnitID><Label>Maximum [Member] | Tenant Concentration Risk [Member]</Label></Row><Row FlagID="0"><Id>4</Id><IsAbstractGroupTitle>true</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><Level>3</Level><ElementName>us-gaap_ConcentrationRiskLineItems</ElementName><ElementPrefix>us-gaap_</ElementPrefix><IsBaseElement>true</IsBaseElement><BalanceType>na</BalanceType><PeriodType>duration</PeriodType><IsReportTitle>false</IsReportTitle><IsSegmentTitle>false</IsSegmentTitle><IsCalendarTitle>false</IsCalendarTitle><IsEquityPrevioslyReportedAsRow>false</IsEquityPrevioslyReportedAsRow><IsEquityAdjustmentRow>false</IsEquityAdjustmentRow><IsBeginningBalance>false</IsBeginningBalance><IsEndingBalance>false</IsEndingBalance><IsReverseSign>false</IsReverseSign><FootnoteIndexer /><Cells><Cell FlagID="0" ContextID="" UnitID=""><Id>1</Id><IsNumeric>false</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>0</NumericAmount><RoundedNumericAmount>0</RoundedNumericAmount><NonNumbericText /><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell><Cell FlagID="0" ContextID="" UnitID=""><Id>2</Id><IsNumeric>false</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>0</NumericAmount><RoundedNumericAmount>0</RoundedNumericAmount><NonNumbericText /><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell></Cells><ElementDataType>xbrli:stringItemType</ElementDataType><SimpleDataType>string</SimpleDataType><IsTotalLabel>false</IsTotalLabel><UnitID>0</UnitID><Label>Concentration Risk [Line Items]</Label></Row><Row FlagID="0"><Id>5</Id><IsAbstractGroupTitle>false</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><Level>4</Level><ElementName>us-gaap_ConcentrationRiskPercentage1</ElementName><ElementPrefix>us-gaap_</ElementPrefix><IsBaseElement>true</IsBaseElement><BalanceType>na</BalanceType><PeriodType>duration</PeriodType><IsReportTitle>false</IsReportTitle><IsSegmentTitle>false</IsSegmentTitle><IsCalendarTitle>false</IsCalendarTitle><IsEquityPrevioslyReportedAsRow>false</IsEquityPrevioslyReportedAsRow><IsEquityAdjustmentRow>false</IsEquityAdjustmentRow><IsBeginningBalance>false</IsBeginningBalance><IsEndingBalance>false</IsEndingBalance><IsReverseSign>false</IsReverseSign><PreferredLabelRole>terseLabel</PreferredLabelRole><FootnoteIndexer /><Cells><Cell FlagID="0" ContextID="D2013Q2YTD_us-gaap_ConcentrationRiskByTypeAxis_lxp_TenantConcentrationRiskMember_us-gaap_RangeAxis_us-gaap_MaximumMember" UnitID="number"><Id>1</Id><IsNumeric>true</IsNumeric><IsRatio>true</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>0.1</NumericAmount><RoundedNumericAmount>0.1</RoundedNumericAmount><NonNumbericText /><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell><Cell FlagID="0" ContextID="D2012Q2YTD_us-gaap_ConcentrationRiskByTypeAxis_lxp_TenantConcentrationRiskMember_us-gaap_RangeAxis_us-gaap_MaximumMember" UnitID="number"><Id>2</Id><IsNumeric>true</IsNumeric><IsRatio>true</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>0.1</NumericAmount><RoundedNumericAmount>0.1</RoundedNumericAmount><NonNumbericText /><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell></Cells><ElementDataType>num:percentItemType</ElementDataType><SimpleDataType>pure</SimpleDataType><ElementDefenition>For an entity that discloses a concentration risk in relation to quantitative amount, which serves as the "benchmark" (or denominator) in the equation, this concept represents the concentration percentage derived from the division.</ElementDefenition><ElementReferences>Reference 1: http://www.xbrl.org/2003/role/presentationRef

 -Publisher FASB

 -Name Accounting Standards Codification

 -Topic 825

 -SubTopic 10

 -Section 50

 -Paragraph 21

 -Subparagraph (a)

 -URI http://asc.fasb.org/extlink&amp;oid=28364263&amp;loc=d3e13537-108611



Reference 2: http://www.xbrl.org/2003/role/presentationRef

 -Publisher FASB

 -Name Accounting Standards Codification

 -Topic 275

 -SubTopic 10

 -Section 50

 -Paragraph 20

 -URI http://asc.fasb.org/extlink&amp;oid=6927468&amp;loc=d3e6404-108592



Reference 3: http://www.xbrl.org/2003/role/presentationRef

 -Publisher FASB

 -Name Accounting Standards Codification

 -Topic 825

 -SubTopic 10

 -Section 50

 -Paragraph 20

 -URI http://asc.fasb.org/extlink&amp;oid=28364263&amp;loc=d3e13531-108611



Reference 4: http://www.xbrl.org/2003/role/presentationRef

 -Publisher FASB

 -Name Accounting Standards Codification

 -Topic 275

 -SubTopic 10

 -Section 50

 -Paragraph 18

 -URI http://asc.fasb.org/extlink&amp;oid=6927468&amp;loc=d3e6351-108592



</ElementReferences><IsTotalLabel>false</IsTotalLabel><UnitID>0</UnitID><Label>Concentration Risk, Percentage</Label></Row></Rows><Footnotes /><IsEquityReport>false</IsEquityReport><ReportName>Concentration of Risk (Details)</ReportName><MonetaryRoundingLevel>UnKnown</MonetaryRoundingLevel><SharesRoundingLevel>UnKnown</SharesRoundingLevel><PerShareRoundingLevel>UnKnown</PerShareRoundingLevel><ExchangeRateRoundingLevel>UnKnown</ExchangeRateRoundingLevel><HasCustomUnits>true</HasCustomUnits><IsEmbedReport>false</IsEmbedReport><IsMultiCurrency>false</IsMultiCurrency><ReportType>Sheet</ReportType><RoleURI>http://www.lxp.com/role/ConcentrationOfRiskDetails</RoleURI><NumberOfCols>2</NumberOfCols><NumberOfRows>5</NumberOfRows></InstanceReport>
