<PAGE>

                                                                      Exhibit 23

               Consent of Ernst & Young LLP, Independent Auditors

We consent to the incorporation by reference in the following J. Alexander's
Corporation Registration Statements:

a)   Form S-8 Registration Statement (No. 333-40556) pertaining to the Employee
     Stock Incentive Plan, filed on June 30, 2000;

b)   Form S-8 Registration Statement (No. 333-91431) pertaining to the J.
     Alexander's Corporation 1999 Loan Program, filed on November 22, 1999;

c)   Form S-8 Registration Statement (No. 333-49393) pertaining to the 1994
     Employee Stock Incentive Plan, filed on April 3, 1998;

d)   Form S-8 Registration Statement (No. 33-77476) pertaining to the 1994
     Employee Stock Incentive Plan, filed on April 6, 1994;

e)   Form S-8 Registration Statement (No. 33-39870) pertaining to the 1990 Stock
     Option Plan for Outside Directors, filed on April 9, 1991;

f)   Form S-8 Registration Statement (No. 2-78139) pertaining to the 1982
     Employee Stock Purchase Plan, filed on June 25, 1982;

of our report dated February 20, 2004, with respect to the consolidated
financial statements and schedule of J. Alexander's Corporation included in the
Annual Report (Form 10-K) for the year ended December 28, 2003.

                                                        /s/ Ernst & Young LLP

Nashville, Tennessee
March 23, 2004

                                       50

