RepositoryException: .lock is locked by another process

Symptoms

In a TarPM clustered CRX environment, following Exception is thrown when starting up additional CRX cluster nodes:

WARN com.day.crx.persistence.tar.ReplicatingTarSet.open - 
        Could not acquire the lock on /repository/shared/version/control 
 
javax.jcr.RepositoryException: The repository home /repository/shared/version/control 
        appears to be in use since the file named .lock is locked by another process. 

Cause

This exception is thrown due to a locking problem whose root cause can be one of the following:

• A CRX instance holding the lock on a shared folder (version, workspace, journal, or repository) was shutdown abnormally, leaving behind stale .lock files
• A CRX cluster node is currently holding the lock on a shared folder but other CRX cluster instances cannot connect to the master CRX cluster node

The first situation is usually non-problematic, the stale .lock file is removed and will be replaced. However, when using an older version of NFS, the file system may not release the file lock on this file after a crash.

The latter is a problem though as communication between a slave and master CRX cluster node is required to ensure proper synchronization of write access to the repository. If a slave cluster node is unable to connect to the master cluster node via the network, it assumes it is not running and tries to take over the master role.

Resolution

When using an older version of NFS, ensure that all processes are stopped, and manually delete all .lock files. If this does not solve the problem, check the address parameter of the listener.properties file within the shared folder that causes the issue.

Example:

address=192.168.1.22\:58128 

This file is read by all (slave) CRX cluster nodes. The IP address must therefore be reachable by all slave machines, otherwise said exception will occur.

Per default, CRX will use any available network interface and available port. It is also possible to explicitely assign an IP address and a port-range to be used. These settings have to be set on the TarPersistenceManager level in both repository.xml and workspace.xml:

<PersistenceManager class="com.day.crx.persistence.tar.TarPersistenceManager"> 
  ... 
  <param name="bindAddress" value="192.168.1.10" /> 
  <param name="portList" value="9100-9110" /> 
</PersistenceManager> 

Applies

CRX1.4.x (clustered)