I agree most people will conflate them, but someone who's allergic to peanuts but not tree nuts (or vice versa), i.e. the people the labels are intended for, are going to care about the difference.