The blockchain part of a custodial wallet is the easy part. The hard part is deciding when a deposit becomes money the user can spend.
Watching addresses and parsing transfers is a solved problem. What is not solved for you is finality. A transaction you saw in the mempool can vanish. A block you credited against can be reorganized away. And the user is already withdrawing.
When I built custodial BTC and ETH wallets and an on-chain payment system, the design that held up was to stop treating "deposit detected" and "balance available" as the same event. Detection creates a pending ledger entry. Confirmation depth promotes it. Only the promotion touches spendable balance, and the promotion is a normal, replayable ledger operation rather than a special case.
That gives you one thing that matters more than speed: a reorg becomes a compensating entry, not an incident. You already have the vocabulary to reverse it, and the history shows what happened instead of a silently corrected number.
The trade-off is user-visible latency, and every product wants to shorten it.
If you run custodial balances: where does your confirmation threshold come from — the asset, the deposit size, or the user's withdrawal behaviour?