Completly forgot to reply to this, but yes, give Gardeners more to do, they don't have enough trays as it is.
And yes, I agree with jackboot, just make it as easy as harvesting plants. No need for a minigame every time I want to catch a fish, it would get annoying pretty quickly.