Yes, blackmail is probably the primary concern. But anyone who has your SSN, DOB, employment history, and every address you've lived at... they can convince banks, cc companies, etc., that they are you, because those phone calls usually involve questions about where you've lived and worked.