If the sealer was left on the face and let to dry, sometimes you can wipe it off with the same sealer. Might be worth a try in a closet or corner.
Who ever told you to butt the joints was wrong, you should have left a small joint to give the grout a place to go and something to grab onto. All you can do at this point is to scrape out all the grout (or at least in the bad areas) and regrout the floor using more force when spreading the grout. Hopefully it will get into the joints enough to grab ahold. Make sure the new grout isn't too thick when spreading.