Using the talisman from the inventory have two problems:
1. You can sell it by mistake - playing long enough you'll do it soon or late and you'll wait long before get it again.
2. You use to click '1' for the talisman, but if you move it in the bank and turn it back it no longer keep the first position. When new dust come and take the first position you click '1' during the boss fight and you lose the dust, then you may click '2' as you remember the talisman is on position 2, but it is no loner there and you lose a second dust
... ok, I haven't clicked '2', but I already lost 3 dust clicking '1'.Proposed solution - make the talisman to stay invisible like Protection Charm, Cap of Wisdom and the other similar, and use it with a key like '~'.
Workaround for 2: bank everything from the inventory then unbank the talisman - it will take the first position and will not move ... hopefully