The bullet that is added to the magazine upon landing a critical hit is not taken from the reserves. This means that as long as critical hits are scored, the gun does not consume ammo.
The flavor text is a Latin phrase that translates to "Remember that you must die." Memento mori objects (like coins or, in this case, a gun) serve to remind one of the inevitability of death.
The name of the weapon refers to the Christian hymn Amazing Grace published in 1779.
This weapon is likely a reference to the Ace of Spades exotic hand cannon from the Destiny video game series. Aside from both weapons being very rare and unique revolvers, the Ace of Spades in Destiny 2 also has its perk named "Memento Mori", and the special weapon effect for Amazing Grace functions identically to the "Maverick" exotic perk that the Ace of Spades has in the first Destiny game.