Hi both of you,
This happens if your Arduino Yun has an old firmware.
Before Release 1.3 (July 14th, 2014) the Yun would trigger the fail-save boot mode if there is any activity on the serial interface during the boot process. With Release 1.3 it has been changed to trigger the fail-save boot mode with typing "ard" on the serial interface. A timeout at start was not necessary anymore and therefore we removed it. It looks like that you must have a board that is older then release 1.3.
If you want to update your yun firmware, you need to do the following instructions to update your arduino yun: https://www.arduino.cc/en/Tutorial/YunUBootReflash
It is a bit more complicated...
If it is to complicated then you can just add the delay in the setup, the way you have described it.