
<PAGE>

                      SECURITIES AND EXCHANGE COMMISSION
                             Washington, D.C. 20549

                                  Schedule 13D

                   Under the Securities Exchange Act of 1934
                               (Amendment No. 4)*

                              Polymer Group, Inc.
            -------------------------------------------------------
                                (Name of Issuer)

                     Common Stock, par value $.01 per share
            -------------------------------------------------------
                         (Title of Class of Securities)

                                  731745 10 5
            -------------------------------------------------------
                                 (CUSIP Number)

                                                     Copy to:
   Jerry Zucker                                      H. Kurt von Moltke
   The InterTech Group, Inc.                         Kirkland & Ellis
   FTG, Inc.                                         200 E. Randolph Drive
   4838 Jenkins Avenue                               Chicago, Illinois 60601
   North Charleston, South Carolina 29405            (312) 861-2000
   (803) 744-5174

--------------------------------------------------------------------------------
                 (Name, Address and Telephone Number of Person
               Authorized to Receive Notices and Communications)

                                 April 5, 2001
            -------------------------------------------------------
            (Date of Event which Requires Filing of this Statement)

If the filing person has previously filed a statement on Schedule 13G to report
the acquisition which is the subject of this Schedule 13D, and is filing this
schedule because of Rule 13d-1(b)(3) or (4), check the following box [_].

Note: Six copies of this statement, including all exhibits, should be filed with
the Commission. See Rule 13d-1(a) for other parties to whom copies are to be
sent.

*The remainder of this cover page shall be filled out for a reporting person's
initial filing on this form with respect to the subject class of securities, and
for any subsequent amendment containing information which would alter
disclosures provided in a prior cover page.

The information required on the remainder of this cover page shall not be deemed
to be "filed" for the purpose of Section 18 of the Securities Exchange Act of
1934 ("Act") or otherwise subject to the liabilities of that section of the Act
but shall be subject to all other provisions of the Act (however, see the
Notes).

                         (Continued on following pages)

                               Page 1 of 11 Pages
<PAGE>

---------------------                                     --------------------
 CUSIP NO. 731745105                   13D                 Page 2 of 11 Pages
---------------------                                     --------------------

------------------------------------------------------------------------------
      NAME OF REPORTING PERSONS
 1
      Jerry Zucker

      S. S. OR I.R.S. IDENTIFICATION NO. OF ABOVE PERSON: [             ].
------------------------------------------------------------------------------
      CHECK THE APPROPRIATE BOX IF A MEMBER OF A GROUP*
 2                                                              (a) [_]
                                                                (b) [X]
------------------------------------------------------------------------------
      SEC USE ONLY
 3

------------------------------------------------------------------------------
      SOURCE OF FUNDS*
 4
      PF
------------------------------------------------------------------------------
      CHECK BOX IF DISCLOSURE OF LEGAL PROCEEDINGS IS REQUIRED PURSUANT
      TO ITEMS 2(d) or 2(e)                                         [_]
 5
------------------------------------------------------------------------------
      CITIZENSHIP OR PLACE OF ORGANIZATION
 6
      United States
------------------------------------------------------------------------------
                          SOLE VOTING POWER
                     7
     NUMBER OF            3,655,929 (See Item 5)

      SHARES       -----------------------------------------------------------
                          SHARED VOTING POWER
   BENEFICIALLY      8
                          3,879,178 (See Item 5)
     OWNED BY
                   -----------------------------------------------------------
       EACH               SOLE DISPOSITIVE POWER
                     9
    REPORTING             3,655,929 (See Item 5)

      PERSON       -----------------------------------------------------------
                          SHARED DISPOSITIVE POWER
       WITH          10
                          3,879,178 (See Item 5)
------------------------------------------------------------------------------
      AGGREGATE AMOUNT BENEFICIALLY OWNED BY EACH REPORTING PERSON
11
      7,535,107 (See Item 5)

------------------------------------------------------------------------------
      CHECK BOX IF THE AGGREGATE AMOUNT IN ROW (11) EXCLUDES CERTAIN SHARES*
12
                                                                    [X]
------------------------------------------------------------------------------
      PERCENT OF CLASS REPRESENTED BY AMOUNT IN ROW (11)
13
      23.4% (See Item 5)
------------------------------------------------------------------------------
      TYPE OF REPORTING PERSON*
14
      IN
------------------------------------------------------------------------------

      * See Instructions
<PAGE>

---------------------                                     --------------------
 CUSIP NO. 731745105                   13D                 Page 3 of 11 Pages
---------------------                                     --------------------

------------------------------------------------------------------------------
      NAME OF REPORTING PERSONS
 1
      The InterTech Group, Inc.

      S. S. OR I.R.S. IDENTIFICATION NO. OF ABOVE PERSON: [             ].
------------------------------------------------------------------------------
      CHECK THE APPROPRIATE BOX IF A MEMBER OF A GROUP:
 2                                                              (a) [_]
                                                                (b) [X]
------------------------------------------------------------------------------
      SEC USE ONLY
 3

------------------------------------------------------------------------------
      SOURCE OF FUNDS:
 4
      Not Applicable (See Item 3)
------------------------------------------------------------------------------
      CHECK BOX IF DISCLOSURE OF LEGAL PROCEEDINGS IS REQUIRED PURSUANT
      TO ITEMS 2(d) or 2(e)                                         [_]
 5
------------------------------------------------------------------------------
      CITIZENSHIP OR PLACE OF ORGANIZATION
 6
      South Carolina
------------------------------------------------------------------------------
                          SOLE VOTING POWER
                     7
     NUMBER OF            3,599,557 (See Item 5)

      SHARES       -----------------------------------------------------------
                          SHARED VOTING POWER
   BENEFICIALLY      8
                          None (See Item 5)
     OWNED BY
                   -----------------------------------------------------------
       EACH               SOLE DISPOSITIVE POWER
                     9
    REPORTING             3,599,557 (See Item 5)

      PERSON       -----------------------------------------------------------
                          SHARED DISPOSITIVE POWER
       WITH          10
                          None (See Item 5)
------------------------------------------------------------------------------
      AGGREGATE AMOUNT BENEFICIALLY OWNED BY EACH REPORTING PERSON
11
      3,599,557 (See Item 5)

------------------------------------------------------------------------------
      CHECK BOX IF THE AGGREGATE AMOUNT IN ROW (11) EXCLUDES CERTAIN SHARES:
12
                                                                    [X]
------------------------------------------------------------------------------
      PERCENT OF CLASS REPRESENTED BY AMOUNT IN ROW (11)
13
      11.2% (See Item 5)
------------------------------------------------------------------------------
      TYPE OF REPORTING PERSON:
14
      CO
------------------------------------------------------------------------------

      * See Instructions

<PAGE>

---------------------                                     --------------------
 CUSIP NO. 731745105                   13D                 Page 4 of 11 Pages
---------------------                                     --------------------

------------------------------------------------------------------------------
      NAME OF REPORTING PERSON
 1
      FTG, Inc.

      S. S. OR I.R.S. IDENTIFICATION NO. OF ABOVE PERSON: [             ].
------------------------------------------------------------------------------
      CHECK THE APPROPRIATE BOX IF A MEMBER OF A GROUP*
 2                                                              (a) [_]
                                                                (b) [X]
------------------------------------------------------------------------------
      SEC USE ONLY
 3

------------------------------------------------------------------------------
      SOURCE OF FUNDS*
 4
      Not Applicable (See Item 3)
------------------------------------------------------------------------------
      CHECK BOX IF DISCLOSURE OF LEGAL PROCEEDINGS IS REQUIRED PURSUANT
 5    TO ITEMS 2(d) or 2(e)                                         [_]

------------------------------------------------------------------------------
      CITIZENSHIP OR PLACE OF ORGANIZATION
 6
      South Carolina
------------------------------------------------------------------------------
                          SOLE VOTING POWER
                     7
     NUMBER OF            261,651 (See Item 5)

      SHARES       -----------------------------------------------------------
                          SHARED VOTING POWER
   BENEFICIALLY      8
                          None (See Item 5)
     OWNED BY
                   -----------------------------------------------------------
       EACH               SOLE DISPOSITIVE POWER
                     9
    REPORTING             261,651 (See Item 5)

      PERSON       -----------------------------------------------------------
                          SHARED DISPOSITIVE POWER
       WITH          10
                          None (See Item 5)
------------------------------------------------------------------------------
      AGGREGATE AMOUNT BENEFICIALLY OWNED BY EACH REPORTING PERSON
11
      261,651 (See Item 5)

------------------------------------------------------------------------------
      CHECK BOX IF THE AGGREGATE AMOUNT IN ROW (11) EXCLUDES CERTAIN SHARES*
12
                                                                    [X]
------------------------------------------------------------------------------
      PERCENT OF CLASS REPRESENTED BY AMOUNT IN ROW (11)
13
      0.8% (See Item 5)
------------------------------------------------------------------------------
      TYPE OF REPORTING PERSON*
14
      CO
------------------------------------------------------------------------------

      *See Instructions
<PAGE>

          This Amendment No. 4 (this "Amendment No. 4") amends and supplements
the statement on Schedule 13D (as previously amended and supplemented, the
"Statement") with respect to the Common Stock, par value $.10 per share (the
"Common Stock"), of Polymer Group, Inc. (the "Issuer"), as previously filed and
amended by Jerry Zucker, The InterTech Group, Inc. ("InterTech") and FTG, Inc.
("FTG") (together, the "Reporting Persons"). James G. Boyd is not a Reporting
Person for purposes of this form, but information on Mr. Boyd is provided by
virtue of the fact that he is Executive Vice President, Treasurer, Secretary and
a Director of both InterTech and FTG.

          Capitalized terms used but defined herein shall have the meanings
ascribed to them in the Statement. Except as indicated herein, the information
set forth in the Statement remains unchanged.

Item 2.   Identity and Background.
          -----------------------

          Item 2 is hereby amended to incorporate by reference the amended
          Exhibit A, which is filed as an exhibit to this Amendment No. 4.

Item 3.   Source and Amount of Funds or Other Consideration.
          -------------------------------------------------

          On March 28, 2001, Mr. Zucker purchased an aggregate of 82,500 shares
          of Common Stock for $119,625; on April 3, 2001, Mr. Zucker purchased
          an aggregate of 117,500 shares of Common Stock for $176,250; and on
          April 5, 2001, Mr. Zucker purchased an aggregate of 100,000 shares of
          Common Stock for $115,000, in each case using personal funds.

Item 4.   Purpose of Transaction.
          ----------------------

          In addition to the shares of Common Stock already held by Mr. Zucker,
          all of the shares purchased in March and April of 2001 were purchased
          for investment purposes. The Reporting Persons may acquire additional
          shares of Common Stock from time to time for investment purposes. The
          Reporting Persons have no other plans or proposals which would relate
          to or result in any action described in the instructions to this Item
          4.


Item 5.   Interest in Securities of the Issuer.
          ------------------------------------

          (a)  Amount Beneficially Owned:

               The shares of Common Stock covered by this statement beneficially
               owned by the Reporting Persons and Mr. Boyd are as follows:
               (based on 32,004,200 shares outstanding in the case FTG and
               InterTech, 32,144,200 shares in the case of Mr. Zucker and
               32,064,200 shares in the case of Mr. Boyd)

                               Page 5 of 11 Pages

<PAGE>

               <TABLE>
               <CAPTION>

                                Shares Beneficially  Percentage of
               Name                    Owned             Class
               ---------------  -------------------  --------------
             --------------------------------------------------------
               <S>              <C>                  <C>
               Jerry Zucker               7,535,107           23.4%
             --------------------------------------------------------
               InterTech                  3,599,557           11.2%
             --------------------------------------------------------
               FTG                          261,651            0.8%
             --------------------------------------------------------
               James G. Boyd              4,399,979           13.7%
             --------------------------------------------------------
               </TABLE>


               The shares of Common Stock reflected above include 3,655,929
               shares held by Mr. Zucker (including 140,000 shares subject to
               options exercisable within 60 days), 7,080 held by Mr. Zucker's
               wife, 10,890 held in trust for Mr. Zucker's children, 3,599,557
               shares held by InterTech, 261,651 shares held by FTG and 538,771
               shares held by Mr. Boyd (including 60,000 shares subject to
               options exercisable within 60 days). Mr. Zucker is Chairman,
               Chief Executive Officer, a Director and the controlling
               stockholder of both InterTech and FTG, and as a result shares
               voting and dispositive power over the shares held by InterTech
               and FTG. Mr. Boyd is the Executive Vice President, Treasurer,
               Secretary, a Director and a stockholder of both InterTech and
               FTG, and as a result shares voting and dispositive power over the
               shares held by InterTech and FTG. Messrs. Zucker and Boyd each
               expressly disclaim beneficial ownership of the shares held by
               each of InterTech and FTG. Mr. Zucker disclaims beneficial
               ownership of the shares held in the names of his wife and
               children.

               In addition to the shares described above, Mr. Zucker holds
               options to purchase shares of Common Stock which are not
               currently beneficially owned by Mr. Zucker pursuant to Rule 13d-
               3(d)(1), because such options are not exercisable within 60 days.
               Such currently unexercisable options consists of the following:
               (A) under the Issuer's 1996 Key Employee Stock Option Plan (the
               "Plan"), Mr. Zucker holds options to purchase an additional
               80,000 shares of Common Stock (at a purchase price of $12.00 per
               share), which options vest in four equal annual installments
               beginning on August 11, 2001, and (B) under the Plan, Mr. Zucker
               holds options to purchase an additional 40,000 shares of Common
               Stock (at a purchase price of $14.25 per share), which options
               vest on August 25, 2001.

               The Reporting Persons and certain other shareholders of the
               Issuer parties to a Voting Agreement. The Voting Agreement
               provides that the parties thereto will vote in the same manner
               on certain issues, including the election of directors.
               Collectively, the parties to the Voting Agreement own 15,960,842,
               or 49.6%, of the outstanding shares of Common Stock. Each of the

                               Page 6 of 11 Pages

<PAGE>

               Reporting Persons disclaims beneficial ownership of shares of
               Common Stock owned by the other parties to the Voting Agreement.
               Exhibit A of this Amendment No. 4, which is incorporated herein
               by reference, sets forth the following information with respect
               to each party to the Voting Agreement: (i) name, (ii) address of
               principal business office, and (iii) the number of shares of
               Common Stock owned by such party.

               (b)  Number of Shares as to which such person has:

                    (i)   Sole power to vote or to direct the vote:

                          Jerry Zucker     3,655,929
                          InterTech        3,599,557
                          FTG                261,651
                          James G. Boyd      538,771

                    (ii)  Shared power to vote or to direct the vote:

                          Jerry Zucker     3,879,178
                          James G. Boyd    3,861,208

                    (iii) Sole power to dispose or to direct the disposition of:

                          Jerry Zucker     3,655,929
                          InterTech        3,599,557
                          FTG                261,651
                          James G. Boyd      538,771

                    (iv)  Shared power to dispose or to direct the disposition
                          of:

                          Jerry Zucker     3,879,178
                          James G. Boyd    3,861,208

               The filing of this Statement shall not be construed as an
               admission by Mr. Zucker, InterTech, FTG, Mr. Boyd or any party to
               the Voting Agreement that such person is, for the purpose of
               Section 13(d), 13(g) or any other Section of the Exchange Act,
               the beneficial owner of any securities covered by this Statement.

          (c)  Transactions Within the Past 60 Days:  In the past 60 days, the
               following transactions have occurred involving the persons named
               in paragraph (a) above.

                               Page 7 of 11 Pages

<PAGE>

               <TABLE>
               <CAPTION>

               Person Who Effected   Transaction     Number of      Price Per
                 the Transaction        Date      Shares Purchased    Share
               -------------------   -----------  ----------------  ---------
               <S>                   <C>          <C>               <C>
               Jerry Zucker          March 28           82,500         $1.45
               Jerry Zucker          April 3           117,500         $1.50
               Jerry Zucker          April 5           100,000         $1.15
               </TABLE>

               All purchases were effected through unsolicited brokers'
               transactions on the New York Stock Exchange.

          (d)  Right to Receive or Power to Direct: No person other than the
               Reporting Persons has the right to receive or the power to direct
               the receipt of dividends from or the proceeds from the sale of
               the Common Stock owned beneficially by any of the Reporting
               Persons.

          (e)  Date Reporting Person Ceased to be 5% Owner:  Not applicable.

Item 6.   Contracts, Arrangements, Understandings or Relationships With Respect
          ---------------------------------------------------------------------
          to Securities of the Issuer.
          ---------------------------

          Reference is made to the information disclosed under Items 2, 3, 4 and
          5 of the Statement and this Amendment No. 4 regarding the Voting
          Agreement.

Item 7.   Materials to be Filed as Exhibits.
          ---------------------------------

     EXHIBIT A:     Exhibit A is hereby amended to incorporate by reference the
                    information set forth in amended Exhibit A, which is filed
                    herewith.

     EXHIBIT G:     Joint Filing Agreement among the Reporting Persons pursuant
                    to Rule 13d-1(f) under the Securities Exchange Act of 1934,
                    as amended, relating to the filing of this Amendment No. 4
                    to Schedule 13D.

                               Page 8 of 11 Pages

<PAGE>

                                   SIGNATURE

          After reasonable inquiry and to the best of my knowledge and belief, I
certify that the information set forth in this statement is true, complete and
correct.


Date: April 16, 2001


                                       /s/ Jerry Zucker
                                       ---------------------------------
                                       Print Name: Jerry Zucker


                                       The InterTech Group, Inc.


                                       By: /s/ Jerry Zucker
                                           -----------------------------
                                       Print Name:  Jerry Zucker
                                       Its:  Chairman, President and CEO


                                       FTG, Inc.


                                       By: /s/ Jerry Zucker
                                           -----------------------------
                                       Print Name:  Jerry Zucker
                                       Its:  Chairman, President and CEO


                              Page 9 of 11 Pages

<PAGE>

                                   Exhibit A

                        Parties to the Voting Agreement
                        -------------------------------

<TABLE>
<CAPTION>
                          Address of Principal           Number of Shares
Name                      Business Office                of Common Stock (a)
----------------------    ---------------------------    -------------------
<S>                       <C>                            <C>
Jerry Zucker              4838 Jenkins Avenue                 3,655,929
                          North Charleston, SC 29405

James G. Boyd             4838 Jenkins Avenue                   538,771
                          North Charleston, SC 29405

The InterTech             4838 Jenkins Avenue                 3,599,557
  Group, Inc.             North Charleston, SC 29405

Golder, Thoma, Cressey    c/o Golder, Thoma, Cressey,         7,109,096
  Fund III, L.P.          Rauner, Inc.
                          6100 Sears Tower
                          Chicago, IL 60606-6402

FTG, Inc.                 4838 Jenkins Avenue                   261,651
                          North Charleston, SC 29405

Leeway & Co.              c/o State Street Bank and             795,838
                          Trust Co.
                          Master Trust Division-Q4W
                          P.O. Box 1992
                          Boston, MA 02110
</TABLE>

(a) Share information provided in this Exhibit A is based on the most recent
information available to the Reporting Persons through public disclosure or
otherwise. Each Reporting Person makes no representations as to the accuracy of
information set forth herein other than for itself.


                              Page 10 of 11 Pages

<PAGE>

                                   Exhibit F

              SCHEDULE 13D AMENDMENT NO. 4 JOINT FILING AGREEMENT
              ---------------------------------------------------

          The undersigned and each other person executing this joint filing
agreement (this "Agreement") agree as follows:

     (i) The undersigned and each other person executing this Agreement are
individually eligible to use the Schedule 13D to which this Exhibit is attached
and this Amendment No. 4 to Schedule 13D is filed on behalf of the undersigned
and each other person executing this Agreement; and

     (ii) The undersigned and each other person executing this Agreement
are responsible for the timely filing of such Schedule 13D and any amendments
thereto, and for the completeness and accuracy of the information concerning
such person contained therein; but none of the undersigned or any other person
executing this Agreement is responsible for the completeness or accuracy of the
information statement concerning any other persons making the filing, unless
such person knows or has reason to believe that such information is inaccurate.

          This Agreement may be executed in any number of counterparts, each of
which shall be deemed to be an original, but all of which, taken together, shall
constitute one and the same instrument.

                           *     *     *     *     *

Date: April 16, 2001

                                       /s/ Jerry Zucker
                                       ------------------------
                                       Print Name: Jerry Zucker


                                       The InterTech Group, Inc.

                                       By:  /s/ Jerry Zucker
                                            -------------------
                                       Print Name: Jerry Zucker
                                       Its: Chairman, Chief Executive
                                            Officer and President


                                       FTG, Inc.

                                       By:  /s/ Jerry Zucker
                                            -------------------
                                       Print Name: Jerry Zucker
                                       Its: Chairman, Chief Executive
                                            Officer and President

                              Page 11 of 11 Pages

