RSA is two algorithms, one for Asymmetric Key Cryptography, and one for Digital Signatures.
Asymmetric Key Cryptography and Key-Exchange are somewhat equivalent.
Using Asymmetric Key Cryptography, you can do a Key-Exchange by virtue of generating a random Symmetric Key (a bunch of random bytes) and perform Encryption that with the recipient's Public Key.
Conversely, you can do Asymmetric Key Cryptography with Key-Exchange by using the key resulting from the Key-Exchange to perform Encryption using a Symmetric Key Cryptography.
An advantage of Diffie-Hellman over RSA for generating Ephemeral Keys. Generating a new Diffie-Hellman a new Ephemeral Key key pair is extremely fast (provided that some "DH parameters", i.e. the Diffie-Hellman Group into which DH is computed, are reused, which does not entail extra risks, as far as we know). This is not a really big issue for big servers, because a very busy TLS server could generate a new "ephemeral" RSA key pair every ten seconds for a very small fraction of his computing power, and keep it in RAM only, and for only ten seconds, which would be enough for Perfect Forward Secrecy.
Regardless, RSA Ephemeral Key has fallen out of fashion, and, more importantly, out of standardization. In the context of TLS, if you want Perfect Forward Secrecy, you need to use Diffie-Hellman Ephemeral, because that's what is defined and supported by existing TLS implementations.
Diffie-Hellman is a Key-Exchange algorithm.
Since these algorithms don't do the same thing, you could prefer one over the other depending on the usage context.
TLS 1.3 has dropped support for RSA key-exchange due to the lack of support for Perfect Forward Secrecy
Diffie-Hellman Ephemeral provides Authentication for both Alice and Bob where RSA by default only provides Authorization for Alice