Saying GMO's are safe is an oversimplification. No matter how many times someone may say it, we don't know that GMO's are 100% safe for 100% of people. Traditional foods are safe for the vast majority of people, but it's a fact that 2% of adults and 8% of children are allergic to some sort of traditional food. We learned that traditional foods were safe by trial and error and a lot of people died as part of the experiment.
The way that we allow people to make free, informed and healthy choices about what they out into their bodies is by requiring disclosure and that disclosure is a cost of doing business in a well-informed society. If GMO's are labeled people can make an informed choice. If GMO producers are worried that people's irrational fears will drive them away from GMO products, those producers can choose to inform consumers.