Early on, with the silo, composter, manufactory and mills, it is very rarely the building itself that is the bottle neck.  It is almost always the resources needed to supply the building.  Upgrading eats up workers and your buildings spend more time empty.  A level 2 silo can process 8200 dung into 400 bags of fertilizer per day to keep 6 orchards running almost non-stop.  Upgrading to level 3 will do nothing for you fertilizer supply chain if you are not producing more then 8200 dung per day.  A level 3 mill does 60 bags of feed.  Perfect for 2 cows 3 times a day, milled overnight. (A level 5 mill does 90 bags for 3 cows, level 6 mill does 120 bags for 4 cows)  If your mills are different sizes, use your smaller mill to feed your cows (overnight) and your larger mill for other feed.  Two level 6 mills and six level 6 fields can keep animals fed well into level 100.