Goldman Sachs Group banker Byron Trott is leaving the investment bank, according to a person familiar with the matter, and will start his own merchant-banking firm.

Mr. Trott gained renown as a close adviser to Warren Buffett, having assisted Mr. Buffett in a series of merger and acquisitions transactions.