To Warren Buffett, it is the one area where he says his investing track record is “awful,” “pretty bad” or “really bad.” Berkshire Hathaway Inc.’s tiny gaggle of retail businesses—from See’s Candies to Ben Bridge Jeweler and Nebraska Furniture Mart—gets little attention from investors and analysts, and the companies are profitable. At Berkshire’s annual meeting in May, Mr. Munger singled out the retail business as one under threat. The 83-year-old, who built Berkshire, based in Omaha, Neb., into a behemoth with $316 billion in market value, declined to comment.