Deposit those withholdings along with your employer share of FICA. While large employers must deposit these taxes via the Electronic Federal Tax Payment System (EFTPS), a system that authorizes transfers from your bank account via telephone or your computer, you can mail or deliver the taxes to a Federal Reserve Bank or authorized financial institution. (If quarterly withheld taxes are less than $1,000, you can pay them directly with your quarterly tax return instead of depositing them.) As a matter of convenience, you may wish to use EFTPS even if you're not required to do so. To find out more about EFTPS, call 800-945-8400 or 800-555-4477.
In addition to federal requirements, your state may require you to withhold taxes as well, so check your state's requirements to ensure you've filed all the necessary employment taxes in a timely fashion.