Games : What is happening to my food in Banished?

Whenever I build a farm in banished, and the farmers finish harvesting the food, where does it go? Because I see the farmers carrying bins/baskets of food away from the fields, but they do not go into my barn, and my people are still starving, so it doesn't go to the people's homes either. So, where does all that food go to?