Except in exotic applications, nobody sensible encrypts data directly with a public-key primitive like the RSA trapdoor permutation of integers modulo a large semiprime $n$, for a variety of reasons.
If you are a sensible practitioner, you will use a public-key key encapsulation method, where the sender uses the recipient's public key to randomly generate, at the same time, (a) a fresh secret session key, and (b) an encapsulation of the secret session key from which the recipient knowing the private key can derive the secret session key.
Then the sender uses the secret session key for a secret-key authenticated encryption scheme like AES-GCM to conceal arbitrarily large volumes of plaintext—in this case, your video stream. Finally, the sender transmits the encapsulation of the session key alongside the authenticated ciphertext from AES-GCM. Pseudocode:
// Sender, given pubkey and plaintext.
key, encap <- kem_encapsulate(pubkey);
ciphertext <- aes256gcm_encrypt(key, 0, plaintext);
packet <- encap || ciphertext;
// Receiver, given privkey, encap, and ciphertext.
if ((key <- kem_decapsulate(privkey, encap)) = null)
goto fail;
if ((plaintext <- aes256gcm_decrypt(key, 0, ciphertext)) = null)
goto fail;
A common, though complicated, example of a KEM is RSAES-OAEP with a randomly chosen session key, or its predecessor RSAES-PKCS1-v1_5, sometimes also called ‘hybrid encryption’. This is what you will find in protocols like OpenPGP and S/MIME. Although you can view RSAES-OAEP as a way to encrypt short strings, essentially all applications of it use it like a KEM by randomly choosing a session key, encapsulating it, and then using the session key for secret-key authenticated encryption.
And essentially all new public-key encryption schemes, such as those under consideration for NIST PQCRYPTO, are built out of this KEM structure—including many that don't work by first choosing a session key and then encapsulating it, such as Shoup's RSA-KEM. Besides the additional flexibility in design of the primitives, this KEM structure does not even tempt you to try to feed data into the public-key primitives directly.
You can also use the secret session key for many messages without doing additional public-key computations, as long as you use a different message sequence number for each message—that's the parameter that I wrote as ‘0’ in the pseudocode above. For example, you might break your video stream into chunks, so that even if one chunk gets corrupted at least the surrounding chunks will still be viable.
Alternatively, you can use a public-key authenticated encryption composition like NaCl crypto_box, with an ephemeral key that you can reuse if you want to send many chunks at a time. In this case, the pseudocode will look like this:
// Sender, given pubkey, seqno, and plaintext.
temppriv, temppub <- crypto_box_keypair();
key <- crypto_box_beforenm(pubkey, temppriv);
ciphertext <- crypto_box_afternm(plaintext, seqno, key);
// Receiver, given privkey, seqno, temppub, and plaintext.
key <- crypto_box_beforenm(temppub, privkey);
if ((ciphertext <- crypto_box_open_afternm(ciphertext, seqno, key)) = null)
goto fail;
Note that you can reuse the key for many different messages and seqnos, as you will likely want to do with video streams broken into chunks.
