Useful explanation. The aim here is to prevent the helix interfering with the 10 GHz feed pattern.The best solution to that is not to try and challenge physics and instead use separate TX and RX dishes where compromises for dual band use do not need to be made.
I looked at adding 1, 2, 3 director elements to the POTY antenna to better suit longer F/D dishes, but it only gained a dB or so before I exceeded the limits of the student CST. Anything more complex was well beyond the capacity of student CST though if someone has access to the full version they could have a go. For example, I could not simulate even a short Helix.
Getting the phase centres in the right place is important, it's not quite right with the POTY but it's close enough. As HB9PZK suggested, a choke surround on the reflector is useful to clean up the backwards sidelobe. This improves the antenna noise temperature but is of no consequence for QO100 as the antenna is only used on TX, hence it's not there. I concluded the extra mechanical work in adding director elements for a dB of gain just wasn't worth it, but my dish is 0.6 f/D. Assuming Willi's simulations are correct, the efficiency is 50% and therefore absolute maximum gain would be less than 3 dB, realistically at best 2 dB. With a longer focal length, where the current feed is less efficient, it might be worthwhile, but what really kills this is the ready availability of 2.4 GHz power amplifiers. Until you get to high power, 2 dB more RF is a lot easier than improving dish efficiency from 50% to 80%.
On receive, I think we might realistically gain in sensitivity with smaller dishes by using a better LNB. The noise figure of standard LNBs at 10.5 GHz is not great, perhaps 3 dB. It should be possible to do much better. I have not seen many people using better LNBs, perhaps because it's so much more expensive.
Mike