As I remember, the issue was precession (tendency of a gyroscope - which is what a flywheel is - to react to its axis of rotation being disturbed by a force 90 degrees to the disturbance).

So, it's more a matter of the axis of rotation remaining the same than that axis having a particular relation to the planet (e.g. vertical).

I had one of those gyro toys, too. It was easiest to spin up with the axis vertical but that was an artifact of how the toy was built.