Welcome to the forum, Lyle.
If you really want to know the color of your grout, grab a handful of leftover grout and make yourself a small sample board. Permacolor is easily the most consistent colored cement-based grout I've worked with. It dries the same, time after time. If your sample doesn't match what's on the floor/wall, then it's almost certainly contaminated with dirt/dust.
Time to put on the brakes from all
your testing. If it really is drywall dust, the only solution is a painstakingly slow scrubbing and wet vacuuming, followed by repeating that many times. In some severe cases, it's faster and easier to simply regret the area.
Pick up the phone and talk to the person you paid. The person you made the contract with. Tell them that the project was left with contaminated grout and you didn't pay for contaminated grout. And that you intend to pay them the balance they are owed AFTER the grout has been cleaned up properly. Hopefully, you still have something of an outstanding balance left to pay.
P.S. Muriatic acid has a
lot more destructive capability than most folks think. From scarring lung tissue from berating in fumes....to rusting everything those fumes come in contact with. It doesn't belong anywhere inside a house at any time. Ever.