First, grout, like any cement product, doesn't 'dry', it cures (it's a chemical reaction, not a drying action). But, that time seems about right for it to get hard.
Grouting is a labor intensive operation that generally must be done fast! Epoxy grout might not be better, especially as we get into warmer weather. You can extend the gout set time some by using very cold water. The chemical reaction releases heat, and when it starts colder, it takes longer to reach the point where it fires off and becomes stiff.
Mixing smaller batches of grout works sometimes, but it can happen that the color component isn't perfectly blended in, or if you mix with slightly different amounts of water, the color can change between batches.
Not a pro, multiple Schluter Workshops (Schluterville and 2013 and 2014 at Schluter Headquarters),
Mapei Training 2014, Laticrete Workshop 2014, Custom Building Products Workshop 2015, and Longtime Forum Participant.