In-Memory Database (IMDB)

An in-memory database (IMDB) is a type of database management system (DBMS) that primarily relies on main memory (RAM) for data storage and retrieval, as opposed to traditional disk-based databases. In an IMDB, data is stored and processed directly in memory, which allows for faster and more efficient operations compared to disk-based databases.

Here are some key characteristics of in-memory databases:

Data Storage: In-memory databases store data in the main memory, which is much faster to access compared to disk-based storage. This enables faster read and write operations, resulting in improved performance.
Data Persistence: While the primary storage is in memory, in-memory databases can also provide mechanisms for persisting data to disk storage. This ensures data durability in case of power outages or system failures back to a known savepoint. IMDB persistence is available for database images that are initially read from disk storage (inmemory_read and inmemory_keep).
Data Structures: IMDBs typically use specialized data structures optimized for in-memory storage and retrieval. These structures are designed to maximize data access speed and minimize memory footprint. Even though the in-memory representation of the data may vary from a disk based storage representation, the schema used will be identical which allows an IMDB to be persisted to permanent storage and read back into an IMDB.
Query Processing: In-memory databases often employ advanced query processing techniques, such as in-memory indexing and caching, to accelerate data retrieval and manipulation. This allows for efficient execution of complex queries and analytical operations.
Real-time Applications: In-memory databases are well-suited for real-time or high-performance applications that require low-latency data access. Industries such as finance, telecommunications, e-commerce, and gaming frequently utilize in-memory databases to support their demanding workloads.

It's worth noting that in-memory databases are not meant to replace traditional disk-based databases entirely. Instead, they serve as a complementary technology that addresses specific performance and latency requirements for certain use cases where speed is critical.

The RaimaDB IMDB is designed to be very simple to use. There is no need for proprietary keywords in the database schema to utilize the IMDB instead of the on-disk engine. Instead, the developer sets an option prior to opening the database to configure and use the IMDB. This allows the same database to be used with either the on-disk engine or the IMDB.

IMDB Configuration Options

When a database is opened there are several options available to configure the RaimaDB storage engine. If no option value is given RaimaDB defaults to the on-disk engine. The different configurations inform the engine what actions to perform when the database is first opened and when the database is closed by the last client to have it open. All clients that have the database open simultaneously must use the same storage configuration options. In addition, it is required that clients simultaneously opening a database use as an identical schema (or no schema at all). The following storage configuration options are supported

On-Disk	When the first client opens a database with no storage configuration option set, or the storage configuration option set to `ondisk` the on-disk engine will be used.
In-memory Volatile	When the first client opens a database with the storage configuration option set to `inmemory_volatile`, the newly created database is considered to be "throw away." Database contents cannot be persisted. When the last client closes the database any changes since the last database persist request will be discarded. If no calls were made to persist the database, then the entire database is discarded. The RaimaDB Transaction File Server (TFS) maintains an open count to determine when a database is first opened and when it is no longer in use by any clients.
In-memory Load	When the first client opens a database with the storage configuration option set to `inmemory_load`, the RaimaDB TFS will look for a pack file. If they are found the in-memory database will be loaded with the current contents of the on-disk files. When the last client closes the database any updates will not be automatically be written to the on-disk files. Changes can be saved by programmatically requesting the databases to be persisted.
In-memory Keep	The `inmemory_keep` option is similar to the `inmemory_volatile` option with the exception that the database will ONLY be discarded when the TFS handling this database is terminated.
In-memory Persist	When the first client opens a database with the storage configuration option set to `inmemory_persist`, the TFS will look for a pack file. If they are found the in-memory database will be loaded with the current contents of the on-disk files. When the last client closes the database any rows that have been modified will be written to the on-disk files. At any time, changes can be persisted programmatically. In this case only changes made since the last time the database was persisted will need to be written to disk.