Questions tagged [compression]

Use this tag for questions about encoding information using fewer bits than the original representation.

Compression involves encoding information using fewer bits than the original representation.

Compression can be lossless or lossy. Lossless compression reduces bits by eliminating statistical redundancy so that no information is lost. Lossy compression reduces bits by removing unnecessary or less important information.

The process of reducing the size of a data file is often called data compression. In the context of data transmission, it is called source coding, which is encoding done at the source of the data before it is stored or transmitted. Source coding should not be confused with channel coding, for error detection and correction, or line coding, the means for mapping data onto a signal.

Compression is useful because it reduces resources required to store and transmit data. Computational resources are consumed in the compression process and, usually, in the reversal of the process (decompression). Data compression is subject to a space-time complexity trade-off. For instance, a compression scheme for video may require expensive hardware for the video to be decompressed fast enough to be viewed as it is being decompressed, and the option to decompress the video in full before watching it may be inconvenient or require additional storage. Design of data compression schemes involves trade-offs among various factors including the degree of compression, the amount of distortion introduced (if using lossy data compression), and computational resources required to compress and decompress the data.

114 questions
18
votes
6 answers

Why can't I compress an encrypted file?

Let's say I have a txt file, called harry_potter.txt. I can easily compress it with any compression algorithm. So the entropy of the file is "smaller" than its size on the disk. But if I encrypt the file with AES-256-CBC or AES-256-BCB (I used…
Yuxuan Lu
  • 293
9
votes
4 answers

Compressing the primes using simple addition?

Consider the sets of integers $$ A = \{1, 3, 7, 13, 27\} \\ B = \{4, 10, 16, 40, 100\} $$ Elementwise addition of sets $A, B$ looks like $A + B := \{ a + b: a \in A, b \in B\}$. Now elementwise-add them to form $A + B$. Here is the…
6
votes
2 answers

Power values of polynomial

$f(x) = a_0 + a_1 x + a_2 x^2 + \dots + a_n x^n$ is a polynomial of degree $n$ with positive integer coefficients. Primary problem statement: Is the Exponential Diophantine Equation $f(f(a) + 1) = y^m$ solvable in integers $y, m \geq 2,…
5
votes
1 answer

How many distinct simply connected shapes can be represented on an n by n binary image?

This is a problem which I found to be quite challenging. I ask this because I want to be able to losslessly compress simply connected shapes (connected shapes without holes) with fewer bits than $n^2$, which can have many downstream applications. By…
5
votes
4 answers

How to maximize the expected number of corrected guesses?

A, B are to play heads or tails for $N$ rounds. They win a round if both guess correctly. A and B are allowed to communicate their strategy before the game starts. A knows the full sequence of $N$ results right after the game starts, before making…
arax
  • 2,815
5
votes
1 answer

Spare storage of a tree

I can store any undirected simple graph N vertices using $b = (N-1)N/2$ bits, by creating a mask of the edges on the upper diagonal of the adjacency matrix. For example the adjacency matrix of $K_3$ is $$ A = [[0,1,1],[1,0,1],[1,1,0]] $$ which can…
Hooked
  • 6,785
4
votes
4 answers

What is the maximum number of primes that I can pack in a 30KB text file?

If I store in trivial way I can store roughly 5100 primes ie primes upto 50k in a 30KB file. Actually I need primes till $2^{30}$ but obviously its not possible to store such a huge list in a file of size of the order of some KBs. So my goal is the…
user1080747
4
votes
1 answer

Is compressibility a good test for randomness of a pseudorandom sequence?

I am interested in tests and definitions of randomness of a sequence generated by a pseudo-random number generator. A similar question was asked a few years ago, and the response was to use a Kolmogorov-Smirnov test for membership in the…
4
votes
2 answers

squashed sine wave

Sinewave I'm slightly out of my comfort zone with this one. I need to produce a function for use in an animation, but a sine wave isn't quite right. I tried adding a square wave, but that didn't work either. What think I need is a "squashed" sine…
Grill
  • 43
3
votes
3 answers

Ways to code two arbitrary binary strings into one without loss of information, and relevant bounds

If the title was not clear, I'm examining methods of taking two binary strings as input and outputting one binary string in such a way that the two original strings can be extracted from the output, and I want to know about how efficiently you can…
G. H. Faust
  • 1,786
3
votes
2 answers

Is there a mistake in the table on page 69 of Cover and Thomas, Elements of Information Theory, Second Edition?

The book reference is here. The problem $(3.13)$ concerns the typical set for a sequence of i.i.d. binary random variables, $X_{1}, X_{2},...,X_{25}$, where the probability that $X_i = 1$ is $0.6$ (and therefore the probability that $X_i = 0$ is…
3
votes
2 answers

Compression of metacomposition strings (https://oeis.org/A133494)

I am looking for a grammar or algorithm to write down what I now call 'metacompositions' (or just compositions of compositions? see https://oeis.org/A133494) of character strings with the absolute minimum of characters, without loosing any…
3
votes
1 answer

Efficient algorithm to solve a sparse recovery problem

I come across with a problem of the form $y=Hx + z \in \mathbb{R}^m$, where $z\in \mathbb{R}^m$ is the noise vector, and $x \in \mathbb{R}^N$ is partially known. $H\in \mathbb{R}^{m \times N}$ can be regarded as a measurement matrix we ourselves…
3
votes
1 answer

Kolmogorov Complexity and Compression Schemes

My question concerns strings with low Kolmogorov Complexities and if there is a single compression scheme that can be used to compress them I have been introduced to Kolmogorov Complexity through Introduction to Theory of Computation, According to…
3
votes
1 answer

PCA for data compression

I would like to use PCA (Principal Component Analysis) to compress a sequence of vectors, $v_0 \ldots v_n$. My plan is to concatenate these vectors into a matrix: $M = [ v_0 \ldots v_n ]$ I will then use PCA to create a smaller representative set of…
1
2 3 4 5 6 7 8