Help me understanding why this gearbox stucks

Hi, I've built this simple 6-gear gearbox (not my design). It works perfectly fine when the left axle is used as an input as intended. But I wanted to do small experiment and use the output axle as an input. I expected that it will work with 1/original ratio for each gear. But in that configuration the original gear 3,4,5 and 6 works fine and 1 and 2 are completely stuck. And I have no idea why. On the picture below the second gear is used. Driving right axle should transfer power over top...