2 Had so enrankled her malicious heart

enrankled > embittered (the "en-" prefix denotes "inward" action, and is also an intensive)

3 That she desired the abridgement of her fate,

fate > {Life, term of life allotted by the Fates}

4 Or long enlargement of her painful smart.

smart > anguish, sorrow

5 Now when the beast (which by her wicked art 6 Late forth she sent) she back returning spied,

Late > Recently

7 Tied with her broken girdle, it a part 8 Of her rich spoils (whom he had erst destroyed)

whom > [she whom, i.e. Florimell] erst > recently; first (before apparently making free with "her rich spoils")