i think in the case of kind 4, 1059 and 1060 the right to do this is very clear
the sender is in the event "pubkey" field and the receiver is named in the event "p" tag
doubly so in the case of the 1059/1060 giftwrapped and other related types, especially the ones that nobody but the recipient can even recognise relates to them, the "giftwrap" key is a one shot throwaway, so it can't even be deleted by the event creator if they didn't store that secret