Missing Release of Memory after Effective Lifetime

Summary

CometBFT is a Byzantine Fault Tolerant (BFT) middleware that takes a state transition machine and replicates it on many machines. The "mempool" maintains two data structures to keep track of outstanding transactions: a list and a map. These two data structures are supposed to be in sync all the time in the sense that the map tracks the index (if any) of the transaction in the list. In versions through 0.34.28, 0.37.0-alpha.3 through 0.37.1 and v0.38.0-alpha.1 through v0.38.0-rc1, it is possible to have them out of sync. When this happens, the list may contain several copies of the same transaction. Because the map tracks a single index, it is then no longer possible to remove all the copies of the transaction from the list. This happens even if the duplicated transaction is later committed in a block. The only way to remove the transaction is by restarting the node. The above problem can be repeated on and on until a sizable number of transactions are stuck in the "mempool", in order to try to bring down the target node. Some workarounds are available. Increasing the value of "cache_size" in "config.toml" makes it very difficult to effectively attack a full node. Not exposing the transaction submission RPC's would mitigate the probability of a successful attack, as the attacker would then have to create a modified (byzantine) full node to be able to perform the attack via p2p. This issue also affects the package github.com/tendermint/tendermint versions v0.31.1 through v0.37.0-rc2.