The luxury handbag maker’s shares have dropped significantly, but that doesn’t mean its stock should be in your portfolio.