
<PAGE>   1
                                                                    EXHIBIT 10.1



                          FIFTH AMENDMENT AND WAIVER TO
            SECOND AMENDED AND RESTATED CREDIT AGREEMENT AND GUARANTY

                  FIFTH AMENDMENT AND WAIVER TO SECOND AMENDED AND RESTATED
CREDIT AGREEMENT AND GUARANTY (the "FOURTH AMENDMENT") dated as of June 8, 1998
among BISCAYNE APPAREL, INC., BISCAYNE APPAREL INTERNATIONAL, INC. MACKINTOSH OF
NEW ENGLAND CO. AND M & L INTERNATIONAL, INC. (individually, each a "BORROWER"
and collectively, the "BORROWERS" and individually, each a "GUARANTOR" and
collectively, the "GUARANTORS"), THE CHASE MANHATTAN BANK, CORESTATES BANK,
N.A., BANKBOSTON, N.A. (formerly known as The First National Bank of Boston),
FLEET BANK N.A. and MILBERG FACTORS, INC. (individually, each a "LENDER" and
collectively, the "LENDERS"), THE CHASE MANHATTAN BANK, as agent for the Lenders
(in such capacity, together with its successors in such capacity, the "AGENT")
and MILBERG FACTORS, INC., as servicing agent for the Lenders (in such capacity,
together with its successors in such capacity, the "SERVICING AGENT" and
together with the Agent, the "AGENTS").

                             PRELIMINARY STATEMENTS:

                  WHEREAS, the Borrowers, the Guarantors, the Lenders and the
Agents have entered into a Second Amended and Restated Credit Agreement and
Guaranty dated as of March 24, 1997, as amended by a First Amendment, dated as
of May 22, 1997, a Second Amendment, dated as of February 18, 1998, a Third
Amendment, dated as of March 6, 1998 and a Fourth Amendment, dated as of March
25, 1998 (as so amended, the "CREDIT AGREEMENT"); and

                  WHEREAS, the terms defined in the Credit Agreement are used in
this Fifth Amendment as in the Credit Agreement unless otherwise defined herein;

                  WHEREAS, the Borrowers and the Guarantors have requested that
certain provisions of the Credit Agreement be amended and certain Defaults and
Events of Default be waived as hereinafter set forth;

                  NOW, THEREFORE, the parties hereto agree as follows:

                  SECTION 1. PERMANENT WAIVER. Each Lender and each Agent hereby
waive (i) the non-compliance with the provisions contained in the definition of
Revolving Credit Loans Permitted Overadvance (Month End) which requires that the
aggregate outstanding amount of Revolving Credit Loans made to BAI as of May 31,
1998 which were used to finance Varon shall be less than the Eligible Accounts
of Varon included in the computation of the Collateral Borrowing Base as set
forth on the Reconciliation Report for the month ended May 31, 1998 in an amount
equal to or greater than $3,000,000 and (ii) the Event of Default arising as a
result of the failure by the Borrowers to make mandatary prepayments pursuant to
Section 2.08(ii) of the Credit Agreement following the occurrence of the
foregoing event.



<PAGE>   2

                  SECTION 2. LIMITED WAIVER. During the period June 1, 1998 
through June 30, 1998 (the "WAIVER PERIOD"), each Lender and each Agent hereby
waive compliance with:

         (i)      the provisions contained in the definition of Revolving Credit
                  Loans Permitted Overadvance (During the Month) which requires
                  that the aggregate outstanding amount of Revolving Credit
                  Loans made to BAI during the month of June 1998 which are used
                  to finance Varon shall be less than the Eligible Accounts of
                  Varon included in the computation of the Collateral Borrowing
                  Base as set forth on the Reconciliation Report for the month
                  of June 1998 in an amount equal to or greater than $3,750,000;
                  PROVIDED, that the amount by which such Revolving Credit Loans
                  shall be less than such Eligible Accounts shall not be equal
                  to or greater than $4,500,000 during the Waiver Period;

         (ii)     the provisions contained in the definition of Revolving Credit
                  Loans Permitted Overadvance (During the Month) which requires
                  that the aggregate outstanding amount of Revolving Credit
                  Loans during the month of June 1998 shall be less than the
                  Eligible Accounts included in the computation of the
                  Collateral Borrowing Base as set forth on the Reconciliation
                  Report during the month of June 1998 in an amount equal to or
                  greater than $9,000,000; PROVIDED, that the amount by which
                  the Revolving Credit Loans shall be less than such Eligible
                  Accounts shall not be equal to or greater than $12,500,000;
                  and

         (iii)    the provisions contained in the definition of Revolving Credit
                  Loan Maximum Outstanding for the period June 1, 1998 through
                  June 30, 1998 which require that Revolving Loans should not
                  exceed $18,250,000 during such period, PROVIDED, that
                  Revolving Loans shall not exceed $25,000,000 during such
                  period.

                  SECTION 3. AMENDMENTS TO CREDIT AGREEMENT. Section 10.02 of
the Credit Agreement is amended by (i) adding the words "and Apparel" after the
phrase "guaranties by BAI" appearing in clause (b) of such section and (ii)
adding the following proviso at the end of such section immediately before the
period:

                  ; PROVIDED, that any guaranty by Apparel shall be limited (i)
                  in the case of guaranties provided to Fortsmann Co. and
                  NationsBank, as factor for Warshaw Woolens, to $500,000 per
                  entity in respect of all extensions of credit by such
                  creditors and (ii) for all other entities to orders for the
                  purchase of raw materials which are placed on or after June 8,
                  1998.

                  SECTION 4. CONDITIONS OF EFFECTIVENESS TO THIS FIFTH
AMENDMENT. This Fifth Amendment shall become effective on the date on which each
of the following conditions have been satisfied or waived:

         (i)      the Borrowers, the Lenders and the Agents shall each have
                  executed and delivered this Fifth Amendment;

         (ii)     payment by the Borrower of all costs and expenses of the
                  Agents and the Lenders (including, without limitation,
                  reasonable attorneys' fees and expenses) incurred in
                  connection with this Fifth Amendment and the Credit Agreement;
                  and



                                       2
<PAGE>   3

         (iii)    receipt of such other documents, opinions or agreements as
                  either of the Agents or any of the Lenders may reasonably
                  request.

                  SECTION 5. REFERENCE TO AND EFFECT ON THE FACILITY DOCUMENTS.
Upon the effectiveness of Section 4 hereof, on and after the date hereof each
reference in the Credit Agreement to "this Agreement", "hereunder", "hereof",
"herein" or words of like import, and each reference in the other Facility
Documents to the Credit Agreement, shall mean and be a reference to the Credit
Agreement as amended hereby. Except as specifically amended above, the Credit
Agreement and all other Facility Documents shall remain in full force and effect
and are hereby ratified and confirmed. Except as specifically waived herein, the
execution, delivery and effectiveness of this Fifth Amendment shall not operate
as a waiver of any right, power or remedy of any Lender or Agent under any of
the Facility Documents, nor constitute a waiver of any provision of the Facility
Documents.

                  SECTION 6. RESERVATION OF RIGHTS. Each of the Borrower agrees
that notwithstanding any present decision of the Lenders and the Agents to waive
certain Events of Default and to waive compliance with certain provisions of the
Credit Agreement as provided in Sections 2 and 3 of this Fifth Amendment, such
course of conduct by the Lenders and the Agents and the execution and delivery
of this Fifth Amendment does not now, and will not in the future, constitute a
waiver by any such party of its right to exercise any of its rights and remedies
at any time now or in the future with respect to such Events of Default or any
other Event of Default. Each of the Borrowers further agrees that, in the
absence of an extension of the Waiver Period or other agreement waiving the
provisions specified in Section 2 hereof (or upon the occurrence of an Event of
Default), the Lenders and the Agents shall be entitled to exercise all rights
and remedies set forth in the Facility Documents which arise upon the occurrence
of an Event of Default relating to the provisions of the Credit Agreement which
are being waived during the Waiver Period (or any other Event of Default).

                  SECTION 7. REPRESENTATIONS AND WARRANTIES. The Borrowers
hereby represent and warrant that each of the representations and warranties
contained in Article 8 of the Credit Agreement and in each of the other Facility
Documents is true and correct as of the date hereof (provided that any
representations and warranties which speak to a specific date shall remain true
and correct as of such date).

                  SECTION 8. COSTS AND EXPENSES. The Borrowers agree to pay the
Agent, the Servicing Agent, and the Lenders on demand all costs, expenses and
charges, in connection with the preparation, reproduction, execution, delivery,
filing, recording and administration of this Fifth Amendment and any other
instruments and documents to be delivered hereunder, including, without
limitation, the fees and out-of-pocket expenses of counsel for the Agent, the
Servicing Agent, and each Lender with respect thereto and with respect to
advising the Agent, the Servicing Agent, and each Lender as to its rights and
responsibilities under such documents, and all costs and expenses, if any, in
connection with the enforcement of any such documents.

                  SECTION 9. GOVERNING LAW. This Fifth Amendment shall be
governed by and construed in accordance with the laws of the State of New York
applicable to contracts made and to be performed wholly within such State.

                                       3

<PAGE>   4

                  SECTION 10. HEADINGS. Section headings in this Fifth Amendment
are included herein for convenience of reference only and shall not constitute a
part of this Fifth Amendment for any other purpose.

                  SECTION 11. COUNTERPARTS. This Fifth Amendment may be executed
in any number of counterparts, all of which taken together shall constitute one
and the same instrument, and any party hereto may execute this Fifth Amendment
by signing any such counterpart.

                  IN WITNESS WHEREOF, the parties hereto have caused this Fourth
Amendment to be duly executed as of the day and year first above written.

                                   BISCAYNE APPAREL, INC.



                                   By: /s/ Peter Vandenberg
                                      -------------------------------------
                                      Name:  Peter Vandenberg
                                      Title: President



                                   BISCAYNE APPAREL INTERNATIONAL, INC.


                                   By: /s/ Peter Vandenberg
                                      -------------------------------------
                                      Name:  Peter Vandenberg
                                      Title: President



                                   MACKINTOSH OF NEW ENGLAND CO.


                                   By: /s/ Peter Vandenberg
                                      -------------------------------------
                                      Name:  Peter Vandenberg
                                      Title: President



                                   M & L INTERNATIONAL, INC.


                                   By: /s/ Peter Vandenberg
                                      -------------------------------------
                                      Name:  Peter Vandenberg
                                      Title: President



                                       4

<PAGE>   5



                                    THE CHASE MANHATTAN BANK,
                                        as Lender


                                    By: /s/ John Murphy
                                       ------------------------------------
                                       Name:  John Murphy
                                       Title: Vice President



                                    MILBERG FACTORS, INC., as Lender


                                    By: /s/ David J. Milberg
                                       ------------------------------------
                                       Name:  David J. Milberg
                                       Title: Vice President



                                    CORESTATES BANK, N.A., as Lender


                                    By: /s/ Don D. Mishlien
                                       ------------------------------------
                                       Name:  Don D. Mishlien
                                       Title: Vice President



                                    BANKBOSTON, N. A., as Lender


                                    By:  /s/ Corrine M. Barrett
                                       ------------------------------------
                                       Name:  Corrine M. Barrett
                                       Title: Vice President



                                    FLEET BANK, N.A., as Lender


                                    By:  /s/ Amy H. Witryil
                                       ------------------------------------
                                       Name:  Amy H. Witryil
                                       Title: Vice President



                                    THE CHASE MANHATTAN BANK, as Agent


                                    By: /s/ John Murphy
                                       ------------------------------------
                                       Name:  John Murphy
                                       Title: Vice President



                                    MILBERG FACTORS, INC., as
                                      Servicing Agent


                                    By: /s/ David J. Milberg
                                       ------------------------------------
                                       Name:  David J. Milberg
                                       Title: Vice President



                                       5
