Understanding Linux SCSI Reservation

SCSI Reservation is a mechanism which allows SCSI initiators to reserve a LUN for exclusive access and preventing other initiators from making changes. The feature is usually used in the cluster. This post elaborates the basic usage of SCSI reservation in CentOS/RHEL systems.

SCSI Reservation contains two stages: First, initiators must register a reservation key, then continue to reserve the device using the same reservation key when a host need exclusive access. Let’s see how we can use the SCSI reservation and necessary functions for SCSI Reservation.

 

Install the necessary utils

 

The sg_persist command provides all the necessary functions for SCSI Reservation, it’s contained in the package sg3_utils.

# yum install sg3_utils

 

View the registration

 

In the example shown below, no reservation key had been registered yet.

# sg_persist /dev/sdc
>> No service action given; assume Persistent Reserve In command
>> with Read Keys service action
IET VIRTUAL-DISK 0001
Peripheral device type: disk
PR generation=0x4, there are NO registered reservation keys

 

Register a reservation key

 

The reservation key must be hex strings, and up to 8 bytes long, here we use abc123 for example.

# sg_persist --out --register --param-sark=abc123 /dev/sdc

 

The output below show a SCSI LUN which have 2 reservation keys (or 2 hosts) registered.

# sg_persist /dev/sdc
>> No service action given; assume Persistent Reserve In command
>> with Read Keys service action
IET VIRTUAL-DISK 0001
Peripheral device type: disk
PR generation=0x6, 2 registered reservation keys follow:
0xabc123
0x123abc

 

Reserve a registered LUN on behalf of a given key

 

# sg_persist --out --reserve --param-rk=abc123 --prout-type=3 /dev/sdc
IET VIRTUAL-DISK 0001
Peripheral device type: disk

 

The –prout-type parameter specified the reservation type, from manpage, valid types including:

  • 1 : write exclusive
  • 3 : exclusive access
  • 5 : write exclusive – registrants only
  • 6 : exclusive access – registrants only
  • 7 : write exclusive – all registrants
  • 8 : exclusive access – all registrants

 

View the reservation

 

The output indicated the server was reserved by key abc123, with type 3 (exclusive access):

# sg_persist -r /dev/sdc
IET VIRTUAL-DISK 0001
Peripheral device type: disk
PR generation=0x6, Reservation follows:
Key=0xabc123
scope: LU_SCOPE, type: Exclusive Access

 

Verify the reservation

 

On node1 which have /dev/sdc1 reserved, it was able to mount the disk.

# mount /dev/sdc1 /mnt
# mount | grep mnt
/dev/sdc1 on /mnt type ext3 (rw,relatime,errors=continue,user_xattr,acl,barrier=1,data=ordered)
# umount /mnt

 

In the meanwhile, it was not able to access the same disk on node2

# mount /dev/sdc1 /mnt
mount: mount /dev/sdc1 on /mnt failed: Invalid exchange

 

The dmesg show reservation conflict

# dmesg | tail
[6902380.608058] sd 11:0:0:1: [sdc] tag#16 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK
[6902380.608060] sd 11:0:0:1: [sdc] tag#16 CDB: Read(10) 28 00 00 1f ff 80 00 00 08 00
[6902380.608061] blk_update_request: critical nexus error, dev sdc, sector 2097024
[6902380.608064] Buffer I/O error on dev sdc1, logical block 261872, async page read
[6902380.609007] sd 11:0:0:1: reservation conflict
[6902380.609011] sd 11:0:0:1: [sdc] tag#14 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK
[6902380.609013] sd 11:0:0:1: [sdc] tag#14 CDB: Read(10) 28 00 00 00 08 00 00 00 01 00
[6902380.609015] blk_update_request: critical nexus error, dev sdc, sector 2048
[6902380.609523] sd 11:0:0:1: reservation conflict
[6902380.609526] blk_update_request: critical nexus error, dev sdc, sector 0

 

Release the reservation

 

# sg_persist --out --release --param-rk=abc123 --prout-type=3 /dev/sdc

 

Unregister a reservation key

 

# sg_persist --out --register --param-rk=abc123 /dev/sdc

 

Was this answer helpful? 0 Users Found This Useful (0 Votes)