You are here: Users Guide > Multi User Database Control

7.1 Introduction

This chapter contains the information you need to build your application in multi-user mode. It explains the lock manager utility, multi-user operation, and how to use an RDM Embedded multi-user environment. Platform specific notes about Lock Manager usage on various platforms can be found in the Platform Release Notes.

7.2 Lock Manager Theory

The most important feature of a multi-user database is protecting data integrity when many applications are reading from and writing to the same set of files. In this chapter we will examine the components that provide safe multi-user access, how the components communicate, and how the components can dependably work with your data.

7.2.1 RDM Embedded Multi-user Components

With RDM Embedded, a lock manager is used to manage file locks and automatically recover failed transactions. The lock manager itself never reads or writes database, LOG, or TAF files.

Applications based on RDM Embedded communicate with the lock manager to request permission to access a database file. The code for this communication is built into the RDM Embedded runtime library, and uses the facilities of the hosting environment.

The applications will also create files (TAF and LOG) that are by-products of the locking process.

7.2.2 Establishing Communication

The applications connect to the lock manager to open their databases. When the d_open function successfully returns, the runtime will have already created a persistent connection to the running lock manager that will be maintained until all databases of all users have been closed. Associated with each connection is a user identifier of not more than 15 characters, set through d_dbuserid or through the environment variable DBUSERID. One connection and one unique userid exist per DB_TASK.

Applications use this connection to submit requests to the lock manager. Requests can include file locks and unlocks, transaction commits, status requests, and the terminal close that will end the connection. The lock manager never initiates contact with an application; all packets from the lock manager are responses to application requests.

Network problems, crashed applications, or an unexpectedly terminated lock manager can abruptly interrupt this orderly exchange of data. If the lock manager loses contact with an application, it will clear out the user's data and prepare for recovery if necessary. If an application loses contact with the lock manager, it will receive an error code of S_LMCERROR in response to a call to the runtime library.

7.2.3 Using The Database

Most requests are lock requests, where the application asks for read or write locks on files in the opened databases. No locks are active on a just-opened database.

The life cycle of a lock begins with the application sending a lock request. If it cannot be granted immediately, the request is queued with all other lock requests. The oldest pending request that can be granted is always granted first. The lock manager rejects a lock request only if it cannot be granted before the user's specified timeout elapses.

All locks are file locks. Any number of users can have concurrent read locks. If any read locks are outstanding, write locks on the same file will be queued to be granted as soon as possible. A write lock prevents any other lock from being granted on the same file.

7.2.4  Transactions

The lock manager is not involved in a transaction until the application calls d_trend. After writing all database changes to the LOG file and before writing to the database files, the runtime sends a "transaction commit begin" message (which includes the LOG filename) to the lock manager. The lock manager flags the user as "in the process of committing a transaction" and then stores the LOG filename.

When the runtime receives the lock manager's confirmation, it is free to begin writing to the database. The changes are migrated from the LOG file to the database files. When all writing is complete, the runtime sends a "transaction commit end" message to the lock manager. The lock manager un-flags the user. This completes the life cycle of a successful transaction.

If communication is disrupted before the "transaction commit end" message is received, the lock manager will know that a transaction was in progress and prepare for auto-recovery. For preparation, the lock manager tells the next application sending any database open or lock request packet to perform a recovery based on the LOG file written by the lost application. The automatic recovery process is described in the Database Recovery section of the User's Guide.

7.2.5 SQL Multi-user Behavior

The SQLSetConnectAttr function can be used to set autocommit mode. Autocommit mode impacts the timing of transactions and the effect of aborts.   Multi-user locking rules will be enforced in various ways, depending on whether autocommit mode is set, and depending on whether a transaction has been explicitly started by the programmer.

If autocommit is off, SQL will begin a transaction upon receiving the first select or update statement, will obtain and retain locks, and will keep the transaction active until an explicit transaction abort or commit. Locks obtained outside a transaction will be retained if an explicit transaction is started. If autocommit mode is on, a commit is performed following each statement execution.

Explicit transaction behavior is overridden by autocommit mode. When autocommit mode is off, read locks are requested for table selects, and write locks are requested for table updates. Read locks may be upgraded to write locks if an update affects a table that was previously selected. Read and/or write locks may exist prior to the explicit transaction begin, but all locks are released when the transaction is aborted or committed.

7.2.6 RDM Embedded System Files

The LOG file contains all the data written to a database in a single transaction, and provides data integrity in the case of an application crash during writes to the database files. The LOG file is written by the runtime, and must be stored in a location visible to all applications using the same lock manager. All LOG files must be stored in the same directory. The LOG directory is specified using d_dblog, rdm.ini, or the environment variable DBLOG. One LOG file exists per user (per DB_TASK) and cannot be shared. The name of the LOG file (without extension or path) is the userid truncated to 8 characters.

The TAF file has two purposes:

  1. It ensures that all users of a database use the same lock manager, and
  2. It provides pointers to LOG files for outstanding transactions in the event of a lock manager or application crash. Each database has only one TAF file, but a single TAF file may serve more than one database.

All users of a database must use the same TAF path, which is specified using d_dbtaf, rdm.ini, or the environment variable DBTAF. If the TAF file does not exist, the RDM Embedded runtime creates it when you call d_open, writing to the TAF file the lock manager name (as set through d_lockmgr, etc.) and type (for example, "TCP"). If the current lock manager name and type do not match those in an existing TAF file, d_open will fail and return S_TAFSYNC. Deleting the TAF file eliminates this error. However, if the database is corrupt and needs recovery, deleting the TAF file will prevent the recovery from occurring and you will be left with a corrupted database.

Unless the MULTITAF option is enabled, RDM Embedded will only create one TAF file, which will be shared by all users. RDM Embedded uses operating system specific functions to lock the TAF file, to prevent multiple users from writing to it simultaneously. However, on some networked systems these locking functions may be slow. RDM Embedded therefore provides the MULTITAF option, which causes a TAF file to be created for each user, avoiding the need for locking. The TAF files must all be located in the same directory, otherwise RDM Embedded's auto-recovery mechanism will not work correctly, in the event of system failure during a transaction. The directory is specified through d_dbtaf, rdm.ini, or the environment variable DBTAF; trailing characters after the last directory character will be ignored.

7.3 Lock Manager Operation

The lock manager must communicate with each application accessing your database files. The network transport available for an RDM Embedded application is TCP/IP. A shared memory implementation of the lock manager (called the Internal lock manager) is available as well, for systems in which all applications are running on the same computer.

On some platforms RDM Embedded can use either of these lock managers to perform the communication. You must decide which lock manager to use, and configure each of your applications to communicate with it.

The term "lock manager" is used to describe the component of RDM Embedded that manages file locks, even though this may or may not be a separate process. Thus the Internal lock manager does not use a separate process to coordinate data access – it uses a shared file, or shared memory, to store lock status information. This lock manager (if present) is built into the RDM Embedded runtime library, so there is no separate lock manager program to be started. 

Three RDM Embedded utilities are used with the lock manager: wconsole, lmclear and lmping.

The wconsole program allows you to monitor and control the TCP/IP lock manager from a remote machine.

The lmclear utility is used to clean up after unexpected communication loss between a lock manager and an application.

The lmping utility can be used to verify that the lock manager is running and that the name and protocol are correct.

In addition, two functions provided by the RDM Embedded runtime library allow you to monitor the operation of the lock manager: d_lmstat and d_lmclear.

These functions are documented in Lock Manager Reference.

7.3.1 Choosing a Lock Manager

Three lock manager transports (in addition to "none") are available for use on RDM Embedded. Not all lock manager transports are available on all supported platforms so please refer to the Platform Release Notes for the list of supported on your operating system.

The first is the internal lock manager, which coordinates data access among tasks on a single machine. The second is a network lock manager, TCP/IP. The third is the lock manager for UNIX domain sockets. For all clients accessing a single database, the same lock manager must be used for each client.

LM Transport

Lock Manager Command Line

TCP

lm -mt 

IP (UNIX only)

lm -mp

Internal lock manager

no lock manager process

The first consideration is whether you require a local or network lock manager. A local lock manager can only service clients on the same system as the lock manager and will almost always have significantly better performance than network lock managers. Network lock managers are necessary when a database is shared over a network file system and the client applications may be on different machines. Note that in the case of network lock managers, none, some, or all clients can be on the same machine as the lock manager. However, if all clients are on the same machine, a local lock manager is normally preferable.

The UNIX domain socket lock manager is local and the TCP lock manager is network. The local/network distinction is blurred by the TCP lock manager, which can (and by default does) use UNIX domain sockets for local connections to provide local communication speed where possible while still supporting network connections. Additional configuration, performance, and reliability issues arise when using a network lock manager with client applications on a remote machine.

If the above considerations do not conclusively determine which lock manager to use, you may wish to discriminate based on performance. However, aside from the network vs. local performance difference, the differences tend to be primarily platform dependent.

7.3.2 Temporary Files

The UNIX domain socket, TCP, and Internal lock managers place files used during communication with RDM Embedded applications in the /tmp directory (the default RDM Embedded temporary directory).

If your UNIX system has background processes that arbitrarily delete files from /tmp, communication problems will arise when critical files are deleted (for example, you may receive the error message S_NOLOCKMGR).

The default temporary directory can be changed for a lock manager either by using the command line option -z dir_name or by setting the DBTMP environment variable to the directory you want to use.

The environment variable DBTMP, the rdm.ini file, and the d_dbtmp function can all be used to change the default RDM Embedded temporary directory for an RDM embedded application. Note that it is critical for a lock manager and all the RDM Embedded applications that will be using it to have the same temporary directory.

7.3.3 Starting a Lock Manager

The lock manager can be started from the command line. On UNIX, the lock manager will by default use fork() to spawn a copy of itself to run in the background, and then the command line invocation will return. This behavior can be suppressed by the -n switch. Windows does support the lock manager running as a service using the instlm utility.

To start a lock manager (named invntory) from a command line:

lm -n -a invntory 

To verify that the lock manager is running:

lmping -a invntory 

7.3.4 Terminating a Lock Manager

The lock managers discussed in this section can be terminated cleanly from the console utility using the SHUTDOWN command or by using the -l option on the lmclear command.

Note that all lock managers attempt to detect clients that have terminated abnormally and then perform the necessary cleanup. However, due to limitations of the type of communication being used, OS limitations, or the specific nature of the abnormal termination, it may not be possible for the lock manager to detect the termination and carry out the cleanup.

In the event of an abnormal termination of the UNIX domain socket lock manager, the TCP lock manager, or an application using one of them, it may be necessary to remove UNIX domain socket files from DBTMP manually.

7.3.5 Resource Issues

All the lock managers discussed in this section consume system or per process resources largely as a function of the number of client applications (users) they have. It is usually necessary to increase system defined limits for these resources to support large numbers of users. The memory used by the lock managers for any practical number of users is not typically significant.

The TCP and UNIX domain socket lock managers consume file descriptors on a one per user basis plus a fixed overhead. For safety and simplicity, a reasonable estimation for the number of file descriptors needed is ten plus the number of users. Carefully read your relevant system administration documentation if you believe this may be an issue.

7.3.6 Lock Manager Naming

Setting Lock Manager Names

When not using the default lock manager name (see below for specific cases), the name of a lock manager is most typically specified using the -a command line switch when starting the lock manager. However it can also be set via the LOCKMGR environment variable. The name of the lock manager being used by an application can be set using the d_lockmgr API call, the LOCKMGR environment variable, or the rdm.ini file.

Local Lock Managers

Naming local lock managers is quite simple. The UNIX domain socket lock manager has the default name of "lockmgr". When using only one lock manager of given type on a system, no complications arise from using the default and you avoid the necessity of setting the lock manager name for both the lock manager and lock manager clients. When using multiple lock managers on the same system, it is necessary to set the appropriate names as described above (although one lock manager can continue to use the default name if desired).

TCP Lock Manager

Naming for the TCP lock manager is more complicated than for any of the others, partly because of its inclusion of the UNIX domain lock manager functionality. As with the other UNIX lock managers the default name is "lockmgr". This works perfectly provided only local connections are used. This limited case is equivalent to using the UNIX domain socket lock manager, and the same naming considerations apply.

However, to obtain a network (TCP) connection to the lock manager it is necessary to identify the lock manager's host and (either implicitly or explicitly) the port on which it is listening. You can implicitly specify the port by putting a "rdm tcp" port specification in each system's /etc/services file, or in a network NIS services map, or by allowing it to default to 1523.

A complete explicit TCP lock manager name for a TCP connection is of the form hostname:port where hostname is either a host's primary name, an alias registered in /etc/hosts, in NIS host maps, or with DNS (some of which will not be available on many systems), or an IP address. See the following list as an example:

Description Example
IP address:port 192.0.1.205:1523
hostname:port rdm:1611
hostname:port rdm.domain.com:1611
IP address (implicit port) 192.0.1.204
hostname:port (implicit port) rdm
hostname:port (implicit port) rdm.domain.com
local name (for UNIX domain sockets only) lockmgr2

As a result of these considerations, the default name "lockmgr" can only work for TCP connections if the lock manager port is specified implicitly, and the host it is on has a primary name or alias of "lockmgr". However, note that it is still possible to mix TCP and UNIX domain socket connections in the same lock manager. Simply choose your name by taking TCP considerations into account and then using the hostname part of the name as the lock manager name for local clients. See the following as an example:

lm -mt -a wlw:1550 
setenv LOCKMGR wlw 
ida -mt 

If you specify a name in hostname:port format, even if the lock manager is local, a TCP connection will be established. Finally, keep in mind that the UNIX domain lock manager and the UNIX domain socket part of the TCP lock managers share the same name space on the local system.

While any of the formats described above will work, you cannot mix formats. If you do, you will get a S_TAFSYNC TAF synchronization error, because lock manager names written to the TAF files will not match. Choose one representation for the lock manager name and stay with it.

More than one TCP lock manager can be run from the same host provided that both the name and the port are unique. For example, two lock managers named lockmgr:1523 and lockmgr2:1523 cannot be run on the same host, nor can wlw:1523 and wlw:1524. You must set up one or more host aliases to obtain the distinct hostname components of the lock manager names.

7.3.7 NFS Filesystems

The TCP lock manager relies on file identifiers generated by the runtime of the form fsname@hostname plus the i-node. The fsname is the absolute path of a mount point and hostname is the server name or alias for that mount point as extracted from the local mounted file system table. The i-node parameter is the i-node. Creating alternate paths to the mount point or individual files using symbolic links can also confuse the TCP lock manager and lead to data corruption.

For example, an NFS host named ufs2 may have a database stored in a directory named /Birdstep/shared/db/tims, with one of the files named tims.d01 at an i-node of 34453. An NFS client may have that directory mounted as /db/tims. The runtime library will construct the following canonical identifier for the file:

/Birdstep/shared/db/tims@ufs2:34453 

This identifier will be the same from other workstations that may have the database drive mounted on a different local path. 

Databases accessed by RDM Embedded that are kept on NFS file systems are more susceptible to data loss and corruption than files kept on a local disk because of the extra levels of caching involved. If an NFS client system crashes (on most UNIX systems this happens rarely if ever) or suffers a power loss, data updates in its local cache that were not written back to the file server will be lost. RDM Embedded provides the SYNCFILES option to help address this problem. The SYNCFILES option causes a cache synchronization operation to be carried out after every write operation (usually by the fsync() function call).

On a few older NFS file systems, this operation may not work reliably. This option has an impact on database performance, and should be used only when there is a possibility of losing important data.

7.4 Using the Lock Manager

When connecting your application to the lock manager, you must specify two things before opening the database: the lock manager type (TCP/IP, IP, or Internal), and the lock manager name. Specification of the protocol is discussed elsewhere in this chapter, but note that it can affect the choice and use of a lock manager name.

The lock manager name will uniquely identify a single lock manager in an environment that may host several. Normally the name used with the -a parameter for the lock manager is exactly the same as the one you will use in your application. See the TCP/IP network protocol section in this reference manual for exceptions.

There are three ways the application can specify the lock manager name. The application can call the d_lockmgr function prior to calling d_open, you can set the operating system environment variable LOCKMGR, or you can specify the lock manager name in the rdm.ini file. A lock manager name set by calling d_lockmgr will always be used. The rdm.ini file is checked only if d_lockmgr is not called and there is no LOCKMGR environment variable.

In the following examples, assume the lock manager was started with the name "lmtest".

To set the lock manager name, the application can call the d_lockmgr function:

d_lockmgr( "lmtest", task ); 

Or the environment variable can be set as follows: 

In UNIX (csh):

setenv LOCKMGR lmtest 
In UNIX (sh): 
LOCKMGR=lmtest 
export LOCKMGR 

Finally, you can set the lock manager name in the rdm.ini file, assigning a value to "name" in the [LOCKMGR] section: 

[LOCKMGR] 
name=lmtest 

Having set the name, you will also need to designate a network protocol to match the lock manager you are using. The network protocol can be specified to the runtime in two ways: through an entry in rdm.ini, or through a call to d_lockcomm.

The constants used in both cases are listed in Choosing a Network Protocol. A call to d_lockcomm takes precedence over an entry in rdm.ini, just as with the lock manager name.

The following line of code chooses the TCP lock manager and must precede the call to d_open:

d_lockcomm( psp_lmcFind("TCP"), task ); 

See the example entry below, which chooses TCP as the network protocol in rdm.ini:

[LOCKMGR] 
type=TCP 

An attempt to use RDM Embedded in shared or exclusive mode without a call to d_lockcomm or an entry in rdm.ini produces the user error S_NOLOCKCOMM, indicating that the lock manager communication layer was not selected. 

7.5 Lock Manager Compatibility

The RDM Embedded lock managers accept simultaneous connections from applications. To allow concurrent compatible database access, you need only ensure that the database structure is interpreted the same by all applications, as discussed in the following section.

7.5.1 SYNCFILES

The RDM Embedded runtime option SYNCFILES forces synchronization of files after every write (thus ensuring the presence of the data on the disk before proceeding).

For performance reasons, this option is disabled by default. While transaction performance is degraded by file synchronization, the possibility of database corruption exists without it. Specifically, corruption is possible if the transaction log file has not been committed to disk before the data and key files are updated, and the computer fails (power outage or program crash). When restarted, the runtime library must find correct data on the disk to perform a reliable recovery, and may not be able to do so. 

Two methods exist to enable/disable SYNCFILES:

  1. You can specify the syncfiles key in the RDM section of the rdm.ini file, or
  2. You can call d_on_opt / d_off_opt with the SYNCFILES flag.

Enabling the SYNCFILES option guarantees that when a transaction is complete, RDM Embedded will physically write all database changes to disc before returning to the application. Files will be committed to disk at the appropriate time using the fsync function on UNIX platforms, the ioctl function on VxWorks, or the FlushFileBuffers function in the Win32 API. 

7.5.2 PORTABLE

An RDM Embedded application accesses the TAF file frequently. Therefore, in a multi-user environment, several applications may potentially be accessing the TAF file simultaneously. To prevent corruption of the TAF file, RDM Embedded uses a form of file locking to serialize access (not the same mechanism as used by data and key file locks). The lock manager is not used. RDM Embedded, by default, uses operating system file locks instead.

Some systems fail to provide any form of file locking. RDM Embedded therefore provides an alternative, portable locking mechanism for the TAF file, which uses a "guard file" to indicate whether the TAF file is locked. As long as the guard file exists, the TAF file is not locked. If the guard file does not exist, then the TAF file is locked.

This mechanism can be switched on by calling d_on_opt(PORTABLE) or by adding "portable=1" to the [rdm] section of rdm.ini

RDM Embedded applications using the same TAF file must all use the same mechanism to lock the TAF file.

7.6 TAF File Issues on UNIX

As discussed elsewhere in this manual, lock managers are not involved in the locking of the TAF file. Instead this task is handled strictly by the runtime library code.

On UNIX, the default method for locking the TAF file is the UNIX OS lockf() function. On Linux, flock() is used. Its functionality is primarily provided by a daemon rpc.lockd, which runs on the file server. Thus, this function is capable of handling lock requests from applications on multiple machines for the same file shared via a network file system.

As such, it is fully capable of working with applications using the RDM Embedded network locking approach as implemented by the TCP lock manager. Historically, there have been significant correctness and reliability problems with lockf() and specifically rpc.lockd. In recent years these problems have largely been corrected.

However, source customers still have the option of compiling the runtime to use the predecessor to lockf(), the flock() function. The locks for flock() are only visible on the local machine, but it has the advantage of being faster than lockf() (usually ) and not having the correctness problems seen in older versions of lockf(). Be aware, however, that UNIX is phasing out flock() and the function is already not available on some platforms.

An alternate method of locking the TAF file that is not dependent on OS functions (that may have bugs or are not available) is the "portable locking" option. This option is implemented using a "guard file" strategy, and the code is enabled by calling the function d_on_opt with the PORTABLE constant. To lock the TAF file, a process will attempt to delete a file named rdm.lfg, which serves as the guard file. If the unlink succeeds, then the TAF file is considered locked by that process. If the unlink fails, the lock is not granted and the process will sleep for one second and try again. The process that does succeed will re-create rdm.lfg after accessing the TAF file.

When using this portable locking method it may be necessary to create rdm.lfg before any processes begin execution. If rdm.taf does not exist, RDM Embedded automatically creates both it and rdm.lfg.

7.7 Multi-User Database Control

In the example below, two users are attempting to update a record from the database at about the same time. The left column gives a relative time and the other columns identify the actions that occur for the respective user at each time.

Table 7-1. Two Users Updating a Record

Time

User 1

User 2

1

Read record

 

2

 

Read record

3

Modify record

 

4

 

Modify record

5

Write record

 

6

 

Write record

At Time 1, User 1 reads the record from the database. At Time 2, User 2 reads the record. Both users then modify and write the record back to the database. But User 2's copy of the record does not have User 1's changes (they were made after User 2 read the record). Thus, when User 2's record is written to the database, User 1's changes are lost. If User 2 had not read the record until after Time 5, the changes from User 1 would not have been lost. But without any control, there is no way to guarantee that User 2 will put off reading until User 1's updates are completed. 

The solution is to provide a mechanism whereby updates to shared data are synchronized, so that only one user can be updating the shared data at a time. The mechanism usually is some form of a lock, which is used to serialize updates to data shared among multiple users. A lock must be applied before shared data is updated, so that other users cannot update the locked data. Thus, through use of a lock, the above example would proceed as shown in the following table:

Table 7-2. Two Users Updating a Record, Using Locks

Time

User 1

User 2

1

Request record lock

2

Lock granted

Request record lock

3

Read record

4

Modify record

5

Write record

6

Free lock

Lock granted

7

Read record

8

Modify record

9

Write record

10

Free lock

Once User 1's lock request is granted at Time 2, User 2 will wait for the record to be unlocked before continuing. When User 1 has completed the update, the lock is freed at Time 6. User 2 is then granted the lock, and the record (which now includes User 1's changes) is read and then updated with User 2's changes.

As the above example illustrates, the key issue in multi-user database applications is data integrity, ensuring that no data is lost and that the data is logically consistent (that is, the interdata relationships that should exist do exist). Data integrity is supported in RDM Embedded through three interrelated facilities:

File locking and record locking are used, as in the example, to prevent loss of data by synchronizing access to shared files and records so that only one user at a time can update them.

Multi-user database application programming is a difficult task, which should be approached with careful planning and design. The problems that can occur in multi-user programs are often very difficult to resolve. The capabilities provided in RDM Embedded are sufficient for you to ensure both data integrity and good performance. The remainder of this chapter explains the RDM Embedded implementation of these capabilities, including guidelines for their effective use.

7.8 Operational Environment

The RDM Embedded multi-user operational environment is depicted in Figure 7-1.

 

Fig. 7-1. Multi-User Operational Environment

Each process executes its own copy of the database program. On a multi-user UNIX/Linux system, the program executes as a separate process. Each process will be referred to as a user. In reality, database users equate to database tasks (DB_TASKs), but usually there is only one database task per process. Each user must have a unique user identifier. The user id can be specified either through the DBUSERID environment variable or through a call to function d_dbuserid prior to the d_open call. The rdm.ini file cannot be used to specify the user id.

If transaction logging is to be used, each user will have a unique transaction log file. By default, the name of the log file is userid.log. The log file's name and location can be specified through either the DBLOG environment variable or the rdm.ini file or function d_dblog, which also must be called before the database is opened. The transaction log contains the database changes made by the user within a transaction, and is used to support recovery.

Management of the RDM Embedded locks, and control of automatic recovery, is performed by the RDM Embedded lock manager. Each process sends lock requests to the lock manager, which queues the requests on a "first come, first served" basis. The process will wait for the lock manager to send a reply either granting or denying the request. Locks are denied when the lock request has waited on the queue for a period longer than the timeout value specified by the requesting process. The lock manager is a program that can be executed from any node on a network or as a background task on a stand-alone machine.

A database-family transaction activity file (TAF) is used by the RDM Embedded runtime to control database recovery in the event the lock manager goes down. This file is named rdm.taf by default, but can be named or located through either environment variable DBTAF or function d_dbtaf, or the rdm.ini file, which must be called before d_open.

Each database family must use only one TAF so that automatic recovery is done correctly. Also, all log files for the database family (TAF) must be located in the same physical directory. The application must enforce these rules through the consistent use of the environment variables, their corresponding d_ functions, and the rdm.ini file.

Multi-user database programs must open the database in shared access mode. This is done by passing an open type of "s" to function d_open. For example, to open the tims database for multi-user access, issue the following call:

if (d_open("tims", "s", task) == S_UNAVAIL)
{
	printf("database unavailable\n");
	exit(1);
}

Status code S_UNAVAIL is returned when some other database program has opened the tims database in exclusive access mode.

7.9 File Locking

7.9.1 Introduction

The principal locking mechanism provided by RDM Embedded is the file lock. An advisory record locking capability, which is used in conjunction with file locking, is also provided and is described in the Advisory Record Locks section. Although file locking is often regarded as an inferior multi-user locking mechanism, in the RDM Embedded environment it can be used very effectively without sacrificing performance. In fact, well-performing multi-user applications can and have been written using only file locking.

Much of the file locking functionality provided in RDM Embedded has been developed to optimize multi-user performance. In addition, the Program Design Considerations section gives some general program design guidelines to help you effectively use the RDM Embedded capabilities. Table 7-3 lists the file locking functions.

Table 7-3. File Locking Functions

Function

Definition

d_reclock

Lock data and key files associated with record type

d_setlock

Lock data files associated with set type

d_keylock

Lock key file containing key field

d_lock

Lock group of record and set types

d_recfree

Free lock on record type

d_setfree

Free lock on set type

d_keyfree

Free lock on key field type

d_freeall

Free all read locks

d_reclstat

Get record type's lock status

d_setlstat

Get set type's lock status

d_keylstat

Get key field type's lock status

d_timeout

Specify wait time before timeout

Note that functions d_trend and d_trabort are also used with file locking although they are not listed here. Recall that both functions free locked files. The remaining pages of this section describe the functions listed in Table 7-3.

Function d_reclock locks the data file containing the record type, as well as all key files containing key fields defined in the specified record type and all vardata files containing varchar fields in the specified record type. For example, assume the following DDL statements appear in your schema:

data file "tims.d01" contains system, key_word, intersect;
data file "tims.d02" contains author, borrower, info, text;
key file "tims.k01" contains id_code;
key file "tims.k02" contains friend, word;
...
record info {
	unique key char id_code[16]; 	/* dewey dec. code */
	char info_title[80]; 		/* title of book, article, mag. */
	char publisher[32]; 		/* name of publisher */
	char pub_date[12]; 		/* date of publication */
	int info_type; 			/* 0=book, 1=mag, 2=art */
}
...
record key_word {
	unique key char word[32]; 	/* subject key words */
}
...
set key_to_info {
	order last;
	owner key_word;
	member intersect;
}

The call d_reclock(INFO, "r", task, CURR_DB) will cause tims.d02 and tims.k01 to be read-locked. The call d_reclock(KEY_WORD,"r", task, CURR_DB) will cause tims.d01 and tims.k02 to be read-locked. Record locks are always necessary when creating a record with keys, or when modifying key fields in records, because they guarantee that all of the files related to the record will be locked.

Function d_setlock locks the data files that contain the owner and member record types of the specified set. The call d_setlock(KEY_TO_INFO, "r", CURR_DB) will cause only file tims.d01 to be read-locked, because both record types are contained in the same file. This function does not lock key files associated with records. Set locks are necessary when connecting or disconnecting records in sets, or when traversing sets, because they guarantee that all of the files related to the set will be locked. Note, however, that if record locks had already been applied to both the key_word and intersect record types, the set lock would not be needed (although it would be accepted). Any time the correct files have been locked, even though by a different locking call, functions that use the files will execute without locking errors.

Function d_keylock locks the key file that contains the specified key field. The call d_keylock(ID_CODE, "r", CURR_DB) will lock only file tims.k01. By locking only a key file, you may scan keys, as in the following code:

d_keylock(ID_CODE, "r", task, CURR_DB);
for ( (status=d_keyfrst(ID_CODE, task, CURR_DB));
		status == S_OKAY;
		(status=d_keynext(ID_CODE, task, CURR_DB)) )
{
	d_keyread(id_code, task);
	printf("id code: %s\n", id_code);
}
d_keyfree (ID_CODE, task, CURR_DB);

By locking only the key file, you eliminate the overhead of obtaining locks on the data file, and possibly other key files related to the same record type. Also, in a multi-user environment, it is important not to lock any files that are not going to be used, because you may be preventing other users from making progress.

Function d_lock is used to lock a group of record and set types, and is described in detail in Grouped Lock Requests.

Table 7-4 lists the types of locks that can be applied.

Table 7-4. File Lock Types

Type

Description

'r'

Read lock

'w'

Write lock

'x'

Exclusive lock

'k'

Keep lock

Functions d_recfree, d_setfree, and d_keyfree free read locks or exclusive locks on their respective record, set, or key field types for their database. Function d_freeall will free all read-locked files across all databases.

The current lock status of a record, set, or key type can be found by calling functions d_reclstat, d_setlstat, or d_keylstat. The lock status indicates whether the calling task has the specified item locked, and if so, with what type of lock. It does not indicate whether other tasks have the item locked.

Function d_timeout is used to inform the lock manager of the number of seconds that lock requests from this process are to wait on the queue before being denied.

7.9.2 Read Locks

A lock type 'r' passed to a locking function specifies that the files are to be read-locked. A read lock on a file prevents other processes from placing a write or exclusive lock on that file. Other processes are allowed to read-lock the file. Thus, when a file is read-locked, other processes are allowed to read the file but are prevented from updating the file.

Functions that only read from the database but cannot be safely executed concurrently with updates require that the files involved be read-locked. In addition, the key access functions and set navigation functions require that the files be read-locked. The record access functions listed in Table 7-5 can be executed concurrently with updates and, therefore, do not require that the data file be locked. If the record being read happens to have been deleted, status code S_DELETED will be returned by the function.

Using these functions without a lock will retrieve data from your local cache without ensuring that it is the most up-to-date data. To ensure synchronization of cache pages, use a read lock.

Table 7-5. Read Functions that Do Not Need a Lock

Function

Definition

d_cmtype

Get record type of current member

d_cotype

Get record type of current owner

d_crread

Read field from current record

d_csmread

Read field from current member

d_csoread

Read field from current owner

d_members

Get count of set members

d_recfrst

Position to first occurrence of record type

d_reclast

Position to last occurrence of record type

d_recnext

Position to next occurrence of record type

d_recprev

Position to previous occurrence of record type

d_recread

Read current record

Read locks can be requested either from outside or within a transaction. Read locks that are issued within a transaction, or are not freed prior to a call to d_trbegin, are freed by either d_trend or d_trabort (unless the lock is to be kept, see Upgrading/Downgrading of File Locks). Read locks requested outside a transaction are freed by calling the appropriate lock freeing function, or by the termination of an intervening transaction.

7.9.3 Write Locks

In order to update the database, the files to be updated must be write-locked. Write locks are requested by passing a 'w' lock type to the lock function. A write lock gives the requesting process exclusive access to the locked file.

Write locks can only be requested within a transaction. Error code S_TRNOTACT will be returned by the lock function if a transaction is not active when the write lock request is made. Write locks are freed by d_trend or d_trabort.

A multi-user programming problem that often occurs is called deadlock. Deadlock is the situation where multiple processes each hold locks on certain resources, and each is waiting to lock resources that are held by the others. The simplest example of deadlock is where Process 1 has Resource A locked and is requesting a lock on Resource B, while Process 2 has a lock on Resource B and is requesting a lock on Resource A. Both processes require access to both resources in order to complete their transactions, but both will wait forever for the other to free its resources.

Deadlock situations can arise in RDM Embedded when multiple processes are attempting to write lock the same files. Only one process will be granted the write lock, and the remaining processes will wait on the queue until the locked file is freed. In a deadlock, those processes will wait on the queue forever because the locked file is never freed. To avoid this situation, RDM Embedded provides function d_timeout, which allows the application to specify how long to wait on the queue before denying a lock request. The default wait time is 10 seconds. An integer argument is passed to d_timeout to specify the number of seconds before timing out. A negative value disables timeouts (you should be certain your program is deadlock free). A value of zero may be used to cause a timeout when a lock request cannot be immediately granted. When status S_UNAVAIL is returned from a lock function (indicating a timeout) the program should abort the transaction and then restart it, in order to prevent deadlock. Status S_UNAVAIL is also returned when the lock manager is temporarily unable to handle the lock request.

7.9.4 Exclusive Locks

The ability to gain exclusive access to database files is provided by passing an exclusive lock type ('x') to the appropriate lock function. Exclusive access to a record or set type differs from a write lock in that it allows updates independent of transactions and other locks. Updates to files in exclusive access can still be made within transactions, if desired. The purpose of this capability is to allow unlocked updates to be made to only a portion of the database while allowing other programs to have shared access to other parts of the database.

Exclusive locks are only released by calling d_recfree, d_setfree, or d_keyfree. Unless explicitly freed, exclusive access locks are held even after the database has been closed by the process that applied the exclusive locks. In this case, the exclusive locks will still be in place when that same user re-opens the database. This provides for situations (for example, month-end closings) where large time-consuming updates requiring exclusive file access aren't completed by the end of the day, and the database program doing the update needs to be terminated and will not be restarted until the next day. Note, however, that if the lock manager goes down, any exclusive locks will be lost.

7.9.5 Upgrading/Downgrading of File Locks

Record, set, and key types that are read-locked do not need to be freed when a transaction is started. Moreover, the capability to upgrade a read lock to a write or exclusive lock is provided. Similarly, a capability is provided to prevent d_trend from freeing a lock, keeping the record or set type read-locked after the transaction has ended. This provides for improved performance by allowing database data to remain in the virtual page buffers, reducing the amount of required disk I/O.

To upgrade a read lock to a write lock, leave the record, set, or key type locked and re-issue the d_reclock, d_setlock, or d_keylock call (with type 'w') after d_trbegin. The upgrade is only granted if there are no upgrade requests queued and all other read locks have been freed. Granted upgrade requests ensure that no changes have been made by other processes between the read and write lock activity. If any normal write locks (that is, not upgrades) are on the queue, the requested upgrade will be granted after all read locks have been freed but before the first queued write lock. If another upgrade request has been queued by the lock manager, the upgrade will be denied immediately (S_UNAVAIL status) but the record or set type will remain read-locked. The proper action in this situation is as follows:

  1. Abort the transaction.
  2. Start a new transaction.
  3. Request a normal write lock.
  4. Update the necessary data.

If the normal write lock request is denied, you should abort and restart the transaction as usual. To upgrade a read lock to an exclusive lock, simply re-issue the d_reclock or d_setlock with type 'x'. The constraints that apply to upgrades from read locks to write locks also apply to upgrades from read locks to exclusive locks.

To downgrade a write lock to a read lock, or to keep a record or set type read-locked after execution of d_trend, re-issue a d_reclock or d_setlock call with type 'k' prior to d_trend.

7.9.6 Grouped Lock Requests

Functions d_reclock and d_setlock only allow for locking record or set types one at a time, requiring each request to be granted by the lock manager before the next lock request is processed. Often it is desirable to lock a group of record or set types in a single request, none of the locks being granted unless all of the requested files are available. In fact, deadlock-free programs result when all needed locks are requested at the beginning of each transaction and processing only continues when all have been granted. This is one of the standard techniques for avoiding deadlock.

Function d_lock provides this grouped lock request capability. The first argument passed to d_lock is an integer containing the count of the number of items to be locked. The second argument is a pointer to the packet of lock requests. Each lock request is placed in a LOCK_REQUEST structure entry as defined below and declared in file rdm.h.

typedef struct {
	unsigned int item; 	/* number of record or set to be locked */
	char type; 		/* type of lock */
} LOCK_REQUEST;

Structure field item contains the record or set constant (as defined in dbname.h) for the record or set to be locked. Key types cannot be locked using function d_lock. Field type contains the type of lock to be applied: 'r', 'w', 'x', or 'k'. (Note that these are single character constants and not strings.)

Grouped lock requests can be statically defined. For example, suppose that, for transaction trxc02, record types EMP and JOBS and set DEPT_EMPS are to be locked. The following variables could be declared and used as follows:

LOCK_REQUEST trxc02[] = {
	{EMP, 		'w'},
	{JOBS, 		'w'},
	{DEPT_EMPS, 	'r'}
};
int trxc02_count = 3; 		/* sizeof(trxc02) / sizeof(trxc02[0]) */
...
d_trbegin("trxc02", task);
if (d_lock(trxc02_count, trxc02, task, CURR_DB) == S_OKAY) {
	
	... 			/* process transaction */
d_trend(task);
}
else
	d_trabort(task);

The trade-off associated with using d_lock, as opposed to waiting until you actually need the lock before you request it, is that the files are locked and others are prevented from using them for a longer period of time. The actual impact will depend on how you design your application program. The peace of mind associated with knowing your program is free of deadlock is often worth some performance penalty.

7.9.7 Lock Status Functions

Three lock status functions are provided, which return the current lock status of a record, set, or key for the requesting process. Function d_reclstat returns the current lock status of the specified record type. Function d_setlstat returns the current lock status of the specified set type. Function d_keylstat returns the current lock status of the specified key field type. A pointer to a character variable that is to contain the lock status is passed to each of the functions. Note that these functions only return the status of the locks held by the requesting process.

7.9.8 Static Records

In a multi-user environment, there are applications in which certain information in the database is virtually static (that is, it does not change). Examples of this kind of data would be menus, data entry forms, help screens, user messages, etc. Rather than requiring database programs to place locks on this data in order to access it, a record attribute called static can be defined in the DDL to identify this kind of information as follows:

[static] record recname {   
	... 
} 

Data files that contain a static record must contain only static records. Static records can be used in sets, but only when the set itself also does not change.

To change static records, the database must be opened in exclusive access mode.

The principal advantage in using static records is performance improvements due to better virtual paging of the static data. Dynamic data requires that the page buffers be cleared when a file lock is requested, in order to ensure concurrent database synchronization. Static data does not need to be cleared because no synchronization is necessary when there are no changes.

7.10 Timestamping

7.10.1 Introduction

Consider the following RDM Embedded database access scenario:

d_reclock(REC, "r", task, CURR_DB); 	/* lock data and key files */
d_keyfind(KEY, value, task, CURR_DB); 	/* find record by key */
d_crget(&dba, task, CURR_DB); 		/* save currency */
d_recfree(REC, task, CURR_DB); 		/* free record lock */
...
...
d_reclock(REC, "r", task, CURR_DB); 	/* relock record type */
d_crset(&dba, task, CURR_DB); 		/* restore currency */
d_recread(REC, task, CURR_DB); 		/* read and process record */
...

The first four statements lock a record type, locate a record occurrence through a key field, save the database address of that record, and then free the lock. The last three statements re-lock the record type, set the current record to the saved database address, and then read the record from the database. However, while the record type was unlocked by this process another process may have updated the database and modified or deleted the record associated with the saved database address. Thus, when the record type is re-locked, the record occurrence is read directly through the database address, which may now contain a deleted record (which would result in a database error) or a modified record (in which case the user would not be made aware of the modifications).

One way to avoid this problem is to re-access the record with another d_keyfind, which would return a "not found" status if the record had been deleted. However, this would not reflect changes to the record, and it would be slower than a direct access through the database address.

Timestamping is a technique whereby it is possible to detect whether a particular record occurrence still exists, and if it has been modified since it was last accessed by a given process. This allows direct access to a record after the file has been unlocked and subsequently re-locked for those records that haven't been updated in the interim.

7.10.2 Implementation

Each timestamped record occurrence contains two values in its record slot header. One is the time when the record was created and the other is the time when the record was last updated. When a record slot is accessed, the maximum of these two values (that is, the time of last modification) is placed, along with the database address, in the currency tables. Functions are provided that compare the value placed in the currency table against the times stored in the record, in order to determine if the record has changed.

There may also be situations in which it is desirable to know if a set has changed since last accessed. Thus, the time of last update value (connect or disconnect) is stored in the timestamped set pointers, to be used to detect changed sets.

The time values cannot simply be the system clock time. In a network environment, the clocks of the different computers are not synchronized. What is used is a sequence counter, which is accessible by all processes. This sequence counter is stored in the file headers (page zero) for each database file, and is incremented each time a write lock is placed on the file or the database is opened for exclusive access. The counter is initialized to zero only when the database is initialized.

Timestamping is an optional feature enabled through the following DDL specifications:

timestamp records [ recname [, recname... ] ] ; 

timestamp sets [ setname [, setname... ] ] ; 

These statements may appear anywhere within a DDL schema specification file. If no record or set names are specified, all records or sets will be timestamped.

Record timestamping adds eight bytes of additional space overhead to each record slot. Set timestamping adds four bytes of additional space overhead to each set pointer contained in set owner records.

The RDM Embedded runtime support for timestamping includes the following:

If timestamping is enabled for a record, the currency table will associate the most recent modification time of that record with its database address. Whenever the currency of that record changes (current owner, current record, or current set) through the navigation functions, that modification time will be adjusted. These timestamp values are then used by the five functions listed in Table 7-6 to test the modification status of that record.

Table 7-6. Timestamp Status Functions

Function

Definition

d_crstat

Test last access status of current record

d_costat

Test last access status of current owner

d_cmstat Test last access status of current member

d_csstat

Test last access status of current set

d_recstat

Test last access status of record

The functions listed in Table 7-8 provide the ability to get the creation timestamp values, or get and set the last access (or currency) timestamp values stored with the timestamps. All timestamp variables are of type "unsigned long."

Table 7-8. Timestamp Currency Functions

Function

Definition

d_ctscr

Get creation timestamp of current record

d_ctsco

Get creation timestamp of current owner

d_ctscm

Get creation timestamp of current member

d_gtscr

Get last access timestamp of current record

d_gtsco

Get last access timestamp of current owner

d_gtscm

Get last access timestamp of current member

d_gtscs

Get last access timestamp of current set

d_stscr

Set last access timestamp of current record

d_stsco

Set last access timestamp of current owner

d_stscm

Set last access timestamp of current member

d_stscs

Set last access timestamp of current set

Whenever a database address in the currency table is to be saved and restored, it is imperative that its corresponding last access timestamp be saved and restored along with it, if the record type is timestamped. Table 7-9 lists the standard currency access functions from the Database Manipulation section, and their timestamp counterparts. These must be called together when the subject record type is timestamped. Functions d_gtscs and d_stscs need to be called with d_csoget and d_csoset when the specified set type is timestamped.

Table 7-9. Corresponding Currency Access Functions

If called

Also call

d_crget

d_gtscr

d_csoget

d_gtsco, d_gtscs

d_csmget

d_gtscm

d_crset

d_stscr

d_csoset

d_stsco, d_stscs

d_csmset

d_stscm

Using these capabilities, the example given in the introduction could be coded as follows:

int status;
DB_ADDR dba;
ULONG rts;
...

d_reclock(REC, "r", task, CURR_DB); 	/* lock data and key files */
d_keyfind(KEY, value, task, CURR_DB); 	/* find record by KEY */
d_crget(&dba, task, CURR_DB); 		/* save current record */
d_gtscr(&rts, task, CURR_DB); 		/* and its timestamp */
d_recfree(REC, task, CURR_DB); 		/* free record lock */
...
...
d_reclock(REC, "r", task, CURR_DB); 	/* relock record type */

/* reset current record and its timestamp */
d_crset(&dba, task, CURR_DB);
d_stscr(rts, task, CURR_DB);

/* test update status of current record */
status = d_crstat(task, CURR_DB);
if (status == S_OKAY)
{
	d_recread(&rec, task, CURR_DB);
	... /* process record */
}
else if (status == S_UPDATED)
{
	... /* check modifications */
}
else
{
	... /* record was deleted */
}
...

The functions listed in Table 7-10 can be used to extract the update timestamps directly from the record slots and set pointers. These functions are used by the timestamping functions defined above. They will not generally need to be used by the user, but they are made available in case access to the actual timestamp values is desired.

Table 7-10. Timestamp Access Functions

Function

Definition

d_utscr

Get update timestamp of current record

d_utsco

Get update timestamp of current owner

d_utscm

Get update timestamp of current member

d_utscs

Get update timestamp of current member

Status code S_TIMESTAMP is returned when the record or set involved in the function is not timestamped. Status S_NOTLOCKED is returned when the record or set type involved in the function is not locked.

7.11 Advisory Record Locks

Three functions that support the locking of individual record occurrences are given in Table 7-11. These functions are used in conjunction with file locking to support end-user modification of data without having to keep a file locked.

Table 7-11. Record Locking Functions

Function

Definition

d_rlbset

Set current record's lock bit

d_rlbtst

Test current record's lock bit

d_rlbclr

Clear current record's lock bit

A bit in the record is used as a flag that indicates whether the record occurrence is locked. This bit is called the record lock bit. The record locking functions are used to set, test, and clear the record lock bit of the current record.

Function d_rlbset is called to set the record lock bit of the current record. The data file containing the record need not be locked. The RDM Embedded runtime will automatically synchronize access to the record. The current record is accessed directly, without using the cache, so that upon successful completion of the call the bit is physically set in the record on the disk. Function d_rlbset will return status S_UNAVAIL if the system cannot gain access to the data file (due to its being write or exclusively locked). Status S_LOCKED is returned if the current record's lock bit is already set. Status S_OKAY is returned when the function has successfully set the lock bit. If d_rlbset is called within a transaction, the operation still goes directly to the file, bypassing use of the cache. Thus, a d_trabort call does not undo any record lock bit settings that were performed within the transaction.

Function d_rlbclr is called to clear the record lock bit of the current record. After a record occurrence is locked, the record will remain locked until function d_rlbclr is called to clear the lock bit. If d_rlbclr is called from within a transaction, the data file containing the current record must be write-locked. User error S_NOTLOCKED is returned if the function is called within a transaction and the data file of the current record is not write-locked. The clearing of the record lock bit cannot be applied until the transaction ends because other modifications to the record are usually involved in the transaction. If d_trabort is called after d_rlbclr, the clear is aborted, and the record remains locked. When d_rlbclr is called within a transaction, the lock bit is cleared through the cache rather than directly to the file. If d_rlbclr is called outside of a transaction, the file does not need to be locked and the operation will directly clear the lock bit on the file. This is used to clear a lock after it has been determined that the record no longer needs to be updated.

Function d_rlbtst can be called to test the lock bit status of the current record. Status code S_LOCKED is returned if the lock bit of the current record is set; code S_UNLOCKED is returned if the lock bit is clear. The data file containing the current record does not need to be locked in order to call d_rlbtst.

No records that have been locked by the program should remain locked after the database is closed. The database consistency check utility, dbcheck, can be used to report all record occurrences that have the lock bit set, or the utility dbclrlb can be used to clear them.

The setting of a record's lock bit (RLB) will not automatically prevent another process from updating the locked record. It is the responsibility of the application to check if a record occurrence is locked before allowing updates to it, by calling d_rlbtst after reading the record's contents. The RDM Embedded functions that modify the database do not use the status of the RLB.

Use of these functions will be illustrated in the next section.

7.12 Program Design Considerations

7.12.1 Basic Guidelines

The primary guideline to follow in designing a multi-user application that uses the RDM Embedded locking functionality is:

Minimize the amount of time that a file is locked. 

The application program has complete control over the use of file locks. If the application's design adheres to this first principle, the result can be well-performing multi-user applications. A necessary corollary to the above principle is:

Caution: Never allow the end-user to control the duration of a file lock!

That is, never design the program so that a file is kept locked while the program is waiting for the end-user to perform some function. With the capabilities provided in RDM Embedded there is never any reason for these guidelines to be violated.

The design techniques that follow show how these principles are applied in data entry, deletion, modification, and retrieval.

7.12.2 Data Entry

The entry of new records and set connections into a database is very straightforward. The steps involved are outlined below.

  1. Collect all relevant data from user.
  2. Begin transaction.
  3. Write-lock necessary record and set types.
  4. Create new record occurrences and make necessary set connections.
  5. End transaction.

The goal is to minimize the amount of time that a file is locked. In this procedure, the files are locked for only the amount of time it takes to perform Steps 4 and 5. If the time required to perform those steps is great (more than a few seconds), it may be beneficial to try to divide the transaction into two or more smaller ones. In general, however, this will not be required. A segment of the ent_info code from the Database Manipulation section is given below with multi-user control functions added.

int info_cnt = 5;
LOCK_REQUEST info_pkt[] = {
	{AUTHOR_LIST, 'w'},
	{INFO, 'w'},
	{ABSTRACT, 'w'},
	{INTERSECT, 'w'},
	{KEY_WORD, 'w'}
};
	...
/* Enter technical information records into TIMS database */
ent_info()
{
	char s[32]; /* Generic string variable */
	int status;

	/* Enter tech info into TIMS database */
	while (get_info() != EOF)
	{
		d_trbegin("ent_info", task);
		do
		{
			status = d_lock(info_cnt, &info_pkt, task, CURR_DB);
		} while (status == S_UNAVAIL);
			/* Wait until locks are available */
	if (status != S_OKAY)
	{
		d_trabort(task);
		... /* Notify user of problem */
		return (0);
	}
	/* See if author exists */
	...
	/* Create new technical info record */
	if (d_fillnew(INFO, &irec, task, CURR_DB) == S_DUPLICATE)
	{
		d_trabort(task);
		printf("duplicate id_code: %s\n", irec.id_code);
		return;
	}
	else
	{
		/* Connect to author record */
		...
		enter_key_words();
		enter_abstract();
	}
	d_trend(task);
	}
}                                   

In the info_pkt lock request packet, one set type and four record types are locked, even though five set types are involved in the transaction. The RDM Embedded runtime system does not care how a file lock is obtained, as long as the appropriate files are locked when the operation requiring the lock is performed. The data files involved in the set operations are locked when the record types are locked. The lock packet could have listed every set type and every record type involved in the transaction, but this would make it larger than necessary. (RDM Embedded does optimize the lock requests so that a lock request for a given file is made only once.) The d_lock call is repeated until the file locks are granted. This cannot produce a deadlock in this situation since no other files are locked. Notice that if the id_code is a duplicate, the transaction is aborted.

In this case, however, another data entry guideline has been violated, which is that files should not be kept locked while control has been given to the user. Functions enter_key_words and enter_abstract both prompt the user for information while the files are locked. To properly implement the function, it would be necessary to redesign the program so that the user first enters all of the needed information (that is, have get_info also prompt for the key words and abstract), after which the transaction is processed.

7.12.3 Data Deletion

Deletion of records and sets follows basically the same guidelines as data entry:

  1. From user, identify data to be deleted.
  2. Read-lock relevant files (if necessary).
  3. Read data to be deleted.
  4. Free files.
  5. Display data to user and request confirmation. Cancel operation if not confirmed.
  6. Begin transaction.
  7. Write-lock necessary set and record types.
  8. Perform necessary set disconnections and record deletions.
  9. End transaction.

Steps 2 through 5 are optional. They should not occur between Steps 7 and 8, however, because of the confirmation request. In this procedure, the files are read-locked for the duration of Step 3 and write-locked for the duration of steps 8 and 9. In Step 2, read locks are necessary if the data is to be accessed by key or by set. If it can be accessed directly (for example, d_recread), the file need not be read-locked at all.

Function del_info from the Database Manipulation section illustrates the previous procedure.

int read_cnt = 2;
LOCK_REQUEST read_pkt[] = {
	{AUTHOR, 'w'},
	{INFO, 'w'},
};

int info_cnt = 5;
LOCK_REQUEST info_pkt[] = {
	{AUTHOR_LIST, 'w'},
	{INFO, 'w'},
	{ABSTRACT, 'w'},
	{INTERSECT, 'w'},
	{KEY_WORD, 'w'}
};

del_info()
{
	struct info irec;
	long count;
	char id[SIZEOF_ID_CODE], name[SIZEOF_NAME];

	printf("id_code: " ); /* get info to delete */
	getstring(id,SIZEOF_ID_CODE);

	if (d_lock(read_cnt, &read_pkt, task, CURR_DB) != S_OKAY)
	{
		printf("data not available\n");
		return;
	}
	if (d_keyfind(ID_CODE, id, task, CURR_DB) == S_NOTFOUND)
	{
		d_freeall(task, CURR_DB);
		printf("id_code %s not on file\n", id);
		return;
	}
	d_recread(&irec, task, CURR_DB);
	d_findco(HAS_PUBLISHED, task, CURR_DB);
	d_crread(NAME, name, task, CURR_DB);
	d_freeall(task, CURR_DB);
	... 		/* display data and confirm delete request */
	d_trbegin("del_info", task);
	while (d_lock(info_cnt, &info_pkt, task, CURR_DB) == S_UNAVAIL)
		; 	/* wait till locks are available */
	/* disconnect and delete abstract */
	...
	/* disconnect, delete intersect and key word */
	...
	/* disconnect info rec from author, delete */
	...
	/* delete author, if no other pubs records */
	...
	if (stat == S_OKAY)
		d_trend(task);
	else 
		d_trabort(task)
}

7.12.4 Data Modification

The general procedure for the modification of data follows:

  1. From user, identify data to be modified.
  2. Read-lock relevant record and set types (if necessary).
  3. Read data to be modified.
  4. Free files.
  5. Allow user to edit data.
  6. Begin transaction.
  7. Write-lock necessary set and record types.
  8. Perform necessary modifications.
  9. End transaction.

Here again, no files are locked while the user is editing the data. But notice that since the files are not locked, it is possible that some other process may try to modify the same data at the same time. In many cases, however, even though it may be theoretically possible, it is not practically possible. The nature of the application may render it impossible for two users to modify the same data at the same time. For example, in the tims database, it isn't possible for the same book to be returned by more than one borrower at the same time. In designing your application, you should carefully analyze the real-world use of your data and avoid incorporating unnecessary code to handle impossible circumstances.

However, there are many situations where it is possible for two users to attempt to modify the same data at the same time. Record locking is used to synchronize these updates, as in the following procedure:

  1. From user, identify record to be modified.
  2. Lock record occurrence (using function d_rlbset).
  3. Cancel operation if record is already locked.
  4. Allow user to edit record.
  5. Begin transaction.
  6. Write-lock record type.
  7. Modify record.
  8. Clear record occurrence lock (using function d_rlbclr).
  9. End transaction.

This illustrates simple modification of a single record occurrence. If a set occurrence is to be modified, the program could adopt a convention whereby if the owner record occurrence is locked, neither the data in that record nor any sets owned by that record can be modified by a separate process. The use and interpretation of the record locking functionality is a program design decision, but it must be consistently applied throughout all applications accessing the same databases. An implementation of this procedure is shown in the following example of an info record occurrence modification:

struct info irec;
int stat;
	...
printf("enter id code: ");
getstring(irec.id_code,SIZEOF_ID_CODE);

stat = d_keylock(ID_CODE, "r", task, CURR_DB);
if (stat != S_OKAY)
{
	printf("data not available\n");
	return;
}
stat = d_keyfind(ID_CODE, irec.id_code, task, CURR_DB);
d_keyfree(ID_CODE, task, CURR_DB);
if (stat != S_OKAY)
	printf("record not found\n");
else if (d_rlbset(task, CURR_DB) != S_OKAY)
	printf("unable to access record\n");
else
{
	d_recread(&irec, task, CURR_DB);
	...
	/* display record and allow edits of all but id_code */
	d_trbegin("mod_info", task);
	while (d_reclock(INFO, "w", task, CURR_DB) == S_UNAVAIL)
		; /* wait until locks are available */
	stat = d_recwrite(&irec, task, CURR_DB); /* update record */
	d_rlbclr(task, CURR_DB); 		/* clear record lock bit */
	if (stat == S_OKAY)
		d_trend(task); 		/* write changes to database */
	else
		d_trabort(task);
}

7.12.5 Data Retrieval

In a typical database application, data retrieval operations usually fall into one of the following three types:

  1. Locating a record or related record occurrences.
  2. Browsing through a list of records where there may be one record per line on the display.
  3. Producing reports where many records in the database may need to be accessed.

For Type 1 data retrieval in RDM Embedded, the procedure is simply to lock the necessary record, set, or key types, access the data, then free the locks and display the results. For example, the following code displays an item by id code from the tims database:

struct info irec; 		/* info record variable */
char name[SIZEOF_NAME]; 	/* author's name */
	...
printf("enter id code: ");
getstring(irec.id_code,SIZEOF_ID_CODE);
if (d_keylock(ID_CODE, "r", task, CURR_DB) != S_OKAY)
{
	... /* tell the user */
	return;
}
if (d_setlock(HAS_PUBLISHED, "r", task, CURR_DB) != S_OKAY)
{
	... /* tell the user */
	return;
}
if (d_keyfind(ID_CODE, irec.id_code, task, CURR_DB) == S_OKAY)
{
	d_keyfree(ID_CODE, task, CURR_DB);
	d_recread(&irec, task, CURR_DB);
	d_findco(HAS_PUBLISHED, task, CURR_DB);
	d_recread(AUTHOR, name, task, CURR_DB);
	d_setfree(HAS_PUBLISHED, task, CURR_DB);
	... 			/* display author & info record contents */
}
else
{
	d_freeall(task);
	printf("info record not found\n");
}

Since the info record is accessed by id_code, d_keylock is used to lock only the key file containing id_code keys. Function d_reclock could also be used to lock the data and key files associated with info. In this case it would make no difference which one you used. In general, however, d_reclock may lock key files that are not involved in the operation (since it locks all key files and some may not be used). The d_setlock call locks the data files containing the owner and member record types of set HAS_PUBLISHED, which in this case is simply tims.d02 (see tims DDL specification in the Database Design section). Notice that the file locks are freed as soon as they are no longer needed.

Data retrieval of Types 2 and 3 use the following procedure:

  1. Lock necessary record, set, or key types.
  2. Retrieve some predetermined amount of data. For example, if browsing, this may be a screen full.
  3. Free locks.
  4. If browsing, display retrieved data.
  5. Go to step 1.

With interactive browsing, it is imperative that the locks be freed at Step 3. Otherwise, the duration of the locks would be dependent on the end user, who typically would press some key to view the next page of data.

In a report, the data is printed in Step 2, and Step 4 is eliminated. Although the locks could remain in place for the duration of the report, this would prevent any updates to the locked files until the report completes. This procedure allows updates by iteratively freeing and re-locking the files during the report. The duration of the locks should remain reasonably small (for example, access 10 to 20 records each time). The following code will print a report of all info records contained in the tims database in id_code order:

struct info irec; /* info record variable */
int ii;

for ( ii = 0; ; )
{
	if (ii == 0)
	{
		if (d_reclock(INFO, "r", task, CURR_DB) != S_OKAY)
		{
			... 		/* report locking error */
			break;
		}
	}
	if (d_keynext(ID_CODE, task, CURR_DB) != S_OKAY)
	{
		d_recfree(INFO, task, CURR_DB);
		break;
	}
	d_recread(&irec, task, CURR_DB);
	printf("id_code: %-20s title: %s\n", irec.id_code, irec.info_title);
	if (++ii == 20)
	{
		d_recfree(INFO, task, CURR_DB);
		ii = 0;
	}
}

The lock on the info record type will be freed every 20 records, allowing any queued write-lock requests access to the locked files.

This procedure is slightly more complicated when a set is scanned instead of a key. The problem occurs when an update is made to the set occurrence being scanned while the set lock is freed. If the current member at the time of the d_setfree is disconnected, then when the scan continues it will result in an error detected by RDM Embedded. If the set is timestamped, its status can be checked after the set type is locked to determine whether the set has been updated. If so, the operation can be terminated or restarted. This will often be acceptable when there are many occurrences of the scanned set type. If it is a system-owned set, however, there is only one occurrence, and if updates are common, timestamping the set will not help. In this situation, use of a record lock on the current member can be used to keep it from being disconnected. This is illustrated in the following code, which produces a report of authors:

d_csmset(AUTHOR_LIST, NULL, task, CURR_DB);
for (ii = -1; ;)
{
	if (ii <= 0)
	{
		while (d_setlock(AUTHOR_LIST, "r", task, CURR_DB) != S_OKAY)
			; if (ii == 0)
			d_rlbclr(task, CURR_DB);
		ii = 0;
	}
	if (d_findnm(AUTHOR_LIST, task, CURR_DB) != S_OKAY)
		break;
	d_crread(NAME, name, task, CURR_DB);
	printf("author: %s\n", name);
	if (++ii == 20)
	{
		if (d_rlbset(task, CURR_DB) == S_LOCKED)
			--ii; 		/* don't stop on a locked record */
		else
		{
			ii = 0;
			d_setfree(AUTHOR_LIST, task, CURR_DB);
		}
	}
}

The d_csmset call makes the current member of author_list null, so that the first call to d_findnm will return the first member. Integer ii starts at -1 to indicate that there is no current record lock to be cleared. When 20 records have been scanned, d_rlbset is called to set the record lock of the current record (which is also the current member). If the lock bit is already set (perhaps someone is updating it), ii is decremented so that the file lock is not freed until the current member can be locked. In order to work correctly, an author record should not be disconnected from the author_list set if its record lock bit is set.

7.12.6 Other Considerations

Use of static data can greatly improve database access performance. Static data can only be changed when a database is opened in exclusive access mode. Thus, to access static data in shared mode, no locks are required, and full virtual memory caching of the static data is performed. Things that only periodically change, such as keyed data validation tables and general ledger accounts, are good candidates for static definition.

Operations that involve processing all occurrences of a particular record type, but do not require that the records be ordered, should utilize the sequential record scan functions d_recfrst and d_recnext. These functions do not require the file to be locked.

7.13 One-User Mode

Two different one-user modes are available in RDM Embedded. A database can be opened in exclusive access mode by passing open type "x" as the second argument on the d_open call. A "one-user-only mode" is selected by passing open type "o" to d_open.

Exclusive-access mode requires the presence of the lock manager. If no other users have opened the selected database, the lock manager will grant exclusive access to the database and will prevent any other access to that database until it has been closed. Exclusive access is intended for use in a multi-user environment to update static tables or perform other critical update operations (for example, a year-end purge).

One-user-only access does not require the presence of the lock manager. Even if the lock manager is present, no interaction will occur. In one-user mode, it is the application's responsibility to make sure that one and only one user is accessing that database at a time. This mode is intended for use in a one-user environment where:1) the transaction processing and recovery features in the multi-user version are needed in a one-user application; or, 2) a one-user version of a multi-user application is to be distributed and a single copy of the source is desired.

A program running in either one-user or exclusive mode does not need to place any locks on any files. Any calls to the lock functions will immediately return (with no error). However, if transaction processing and logging are used, file locking will be fully operational, except that the runtime cache will never need to be cleared.


Copyright © 2008 by Birdstep Technology, Inc. All rights reserved.  RDM Embedded Product Family