Question to Watchmakers - ETA 2836 - What am I doing wrong?

I have a consistent dilemma when playing with the ETA 2836. When I re-install the stem, I always manage to kick the clutch wheel out from under the clutch lever. Very frustrating as I have to take everything back apart, lift the set bridge, push the wheel back in place and start over. As they say doing the same thing over and over and expecting a different result...
I always push the button to lift the set lever when installing the stem (very little pressure).
The question: should I be doing something different? Maybe I'm missing a worn piece. Maybe more lubrication on the stem. Or is it simply the nature of the beast.
Any help provided is greatly appreciated.

It's likely how far you are pushing the button down - many people use tweezers, but the slot you see around the button shows you that you should be using a screwdriver - an 80. By using this, it limits how far down the button can be pressed. I press the button on removal and insertion of the stem.

People will tell you the crown needs to be in a specific position when you remove the stem, but honestly any will work - I always do it with the crown all the way in, in the winding position. You need to be somewhat careful when you insert the stem, so you just can't ram it in there off at odd angles, but generally this is a pretty easy thing - same as a 2824 in that regard.

So if I remove the stem while in "time set" position, it may solve my problems? This is something I have never done other than some Seiko movements (if I recall).

Good point with the button. I always use tweezers. I'll use the 80 screwdriver moving forward.

As I said it doesn't matter. I have it in the winding position and have no problems.