RSA decrypting of a huge file by parts

Question

I need to decrypt a huge file that I own previously encoded by myself with a RSA public key (it's possible for this step using a symmetric algorithm key). Problem is that I can't load it in my disposable memory for specific embedded architecture reasons.

I need to decrypt it with the private key associated to the identity of that architecture (or the shared symmetric case if there were the algorithm chosen). I want to know if its possible to decrypt the file loading it in memory by parts and decrypting each of them, without losing security against brute force attacks. As far as I know, in a typical RSA encoding process first you transform the whole char string that is the file into a number, to be encrypted, so my assumption is that is not possible in a typical RSA encoding-decoding process. However, as I have the file I can encrypt it line by line or char by char, instead of encoding the whole, to decrypt it later again slice by slice. I'm assuming that a line or a char after encryption would have the same length in bytes, which may be a wrong assumption.

EDIT: Let's complicate things a little bit more: After decrypting through RSA or symmetric key, I'll have to encrypt again the slices of my huge file, this time with a RSA public key. So, first: Encrypting huge file on a custom basis to make it decryptable part by part; second: take each part and encrypt it with RSA.

Is this a feasible solution? Am I losing efficiency and would be more exposed to brute force attacks?

EDIT2: Keysizes would be the typical for opensource implementations: a minimum of 256bits. I've to investigate the hybrid encryption, can't tell yet. When I say a line I mean chunks of data of, let's say, 50-60 chars. I pretend to encrypt-decrypt a binary file, so char length doesn't matter (consider them 1 bytes ASCII if it helps)

Answer 1

As far as I know, in a typical RSA encoding process first you transform the whole char string that is the file into a number, to be encrypted, so my assumption is that is not possible in a typical RSA encoding-decoding process.

That's not typical. A typical real world RSA encryption process uses hybrid encryption, encrypting the data with a single-use symmetric key that is then encrypted using the actual RSA algorithms.

Depending on what the symmetric encryption is, you may be able to "seek" within the encrypted file without decrypting other parts. In most cases you'll at least be able to read it from the start and throw away decrypted parts as you go, avoiding excessive memory use.

However, as I have the file I can encrypt it line by line or char by char, instead of encoding the whole, to decrypt it later again slice by slice. I'm assuming that a line or a char after encryption would have the same length in bytes, which may be a wrong assumption.

Is this a feasible solution? Am I losing efficiency and would be more exposed to brute force attacks?

Using context-dependent (e.g. lines) blocking could leak data on the plaintext, so I wouldn't recommend it.

If you need to be able to quickly jump within the encoded file, you should use a symmetric encryption algorithm (under RSA if you want) that makes this easy. For example, AES in CTR or CBC modes. You will not be able to safely modify a small part of the file, but for read-only access that would be fine.

EDIT: Let's complicate things a little bit more: After decrypting through RSA or symmetric key, I'll have to encrypt again the slices of my huge file, this time with a RSA private key. So, first: Encrypting huge file on a custom basis to make it decryptable part by part; second: take each part and encrypt it with RSA.

I.e. the file is read-write?

In that case, I would divide the file into constant-sized blocks that get independently encrypted using any symmetric encryption algorithm that accepts a large nonce (24+ bytes would be good, but 16 at the very least). Generate a single symmetric encryption key, which you encrypt with RSA. For each block generate a random nonce/IV that gets prepended to the ciphertext. When something is modified, rewrite the whole block(s), generate new random nonces.

Block size should be chosen to balance the memory expansion caused by nonces with the write amplification caused by rewriting blocks. Something like 4KiB blocks would probably be OK, I would think.

Instead of random nonces, one could use an incrementing counter, but in that case there might need to be additional safeguards to prevent an attacker feeding you an old version of the file, so random nonces are simpler.

Answer 2

As I already wrote at https://math.stackexchange.com/questions/832652/rsa-decrypting-of-a-huge-file-by-parts, it is you, who knows the encryption details! But normally it is a misuse of RSA to encrypt large files. RSA works up to the key size (typically max 2048 or 4096 bit), therefore you must have encrypted the huge file using these small chunks, and decryption also has to work on these. As I see there are two scenarios:

1. If you did the encryption with just one RSA encryption primitive, you have no chance to recover the plain text, example: From $1234567^7 = 126\; \mathrm{mod}\;(11*13)$, there is no realistic way to get back the $1234567$ from the $126$.

2. You have encrypted in say 2048 bit chunks and have written the result chunks as hex (i.e. 512 chars per chunk) to the file. Then you can reverse the process.