Is there a data structure for lazy-loading time-series data?

Question

I am writing a UI that needs to display a chat log, similar to Slack and Discord. In addition to being able to scroll and lazy-fetch additional pages in either direction, I need to be able to jump to a specific point in time and fetch the message records "around" it (a backend API problem, of course).

I need a "lazy range" data structure that can store "intervals" (each with bounds and an ordered list of items), respond to queries for "more before" or "more after" including items and some kind of indication of gaps, and insert new intervals (pages fetched from the backend), merging existing intervals when overlaps are detected and eliminating the gap indicator.

I have not been able to find an existing named data structure that handles this kind of access pattern. It seems similar to a range tree but has only a single level of interval nodes, and it is similar to an interval tree in that we want to detect bound overlaps/gaps, but then we want to merge overlaps. Does such a data structure already exist?

Answer 1

The binary search tree of @dw seems a good solution.

For something simpler, if the time density of messages is homogeneous, you can consider successive time intervals of a well-chosen fixed duration and associate to every interval the doubly-linked list of events that occurred, in chronological order. The intervals are stored in an array.

From the current event, you can move forward or backward, and switch to the next/previous interval easily. For random access, just find the relevant interval.

The duration should be such that a single interval roughly corresponds to the size of the scrolled area.

---

If your data structure is external (on disk), a B-tree indexed on the timestamp is appropriate.

Answer 2

I find it hard to understand your requirements, but I suspect a balanced binary search tree might meet your needs. I assume each record is associated with a timestamp, and you want to support queries like "find the record nearest time $t$", "find the 50 records immediately preceding time $t$", "find the $i$th record", and so on.

All of those can be handled in $O(\log n)$ time with a balanced binary search tree.

I am assuming here that each record represents a single chat message, and so has a single timestamp associated with it, and that a page is simply a window of consecutive records.