<PAGE>

                                                                      Exhibit 23

               Consent of Ernst & Young LLP, Independent Auditors

We consent to the incorporation by reference in the following J. Alexander's
Corporation Registration Statements:

a)   Form S-8 Registration Statement (No. 333-40556) pertaining to the Employee
     Stock Incentive Plan, filed on June 30, 2000;

b)   Form S-8 Registration Statement (No. 333-91431) pertaining to the J.
     Alexander's Corporation 1999 Loan Program, filed on November 22, 1999;

c)   Form S-8 Registration Statement (No. 333-49393) pertaining to the 1994
     Employee Stock Incentive Plan, filed on April 3, 1998;

d)   Form S-8 Registration Statement (No. 33-77476) pertaining to the 1994
     Employee Stock Incentive Plan, filed on April 6, 1994;

e)   Form S-8 Registration Statement (No. 33-39870) pertaining to the 1990 Stock
     Option Plan for Outside Directors, filed on April 9, 1991;

f)   Form S-8 Registration Statement (No. 2-78139) pertaining to the 1982
     Employee Stock Purchase Plan, filed on June 25, 1982;

of our report dated February 20, 2004, except for Notes A, B and F, as to which
the date is May 17, 2004, with respect to the consolidated financial statements
and schedule of J. Alexander's Corporation included in the Annual Report (Form
10-K/A) for the year ended December 28, 2003.

                                                        /s/ Ernst & Young LLP

Nashville, Tennessee
May 17, 2004

                                       50

