Yes, you can remove all lines from the turbo, just like in this official Audi TSB:
https://www.goapr.com/support/tsb/21e7.pdf
I'm not sure why the IS12 upgrade thread linked here says you can't remove the oil return from the turbo. That poster might have had bad luck but I had MUCH better luck removing the return from the turbo than I did from the block. Although the bolt holding it in is a bear to get to, the line came right out. Obviously YMMV on which lines are more stuck.
On the AWD and Performance Pack cars it's so, so much easier to pull all 4 lines off the turbo and not touch the block, which I'd assume is why Audi suggest it for the S3.
The oil return is held in by an M8 triple square and is on the bottom of the turbo CHRA. I recommend removing the coolant drain (again, from the rear of the turbo) prior to the oil return. Another trick: if you're having trouble getting the M8 triple square to work, you can very, very carefully use a T40 torx as these screws aren't generally torqued down very far. This gives you a little more "wiggle", sort of like using a ball-end hex socket. Obviously be careful not to overuse this horrible shade-tree trick and if you feel too much resistance, switch to the correct bit before you strip the screw out.
This job is also much easier with no downpipe, although you can do it with the downpipe simply pushed aside.