Think of it as you are reducing clutter in your personal life. If these people are not really important, then they must not be really important. If someone was really a good friend, they would definitely tell you what they feel about it.