Table of Contents
This chapter outlines typical administrative tasks encountered during day-to-day operations. It does not cover troubleshooting tasks, these are covered in detail in Chapter 7, Troubleshooting and error recovery.
The most convenient way to look at DRBD’s status is the
drbd-overview
utility.
# drbd-overview 0:home Connected Primary/Secondary UpToDate/UpToDate C r--- /home xfs 200G 158G 43G 79% 1:data Connected Primary/Secondary UpToDate/UpToDate C r--- /mnt/ha1 ext3 9.9G 618M 8.8G 7% 2:nfs-root Connected Primary/Secondary UpToDate/UpToDate C r--- /mnt/netboot ext3 79G 57G 19G 76%
/proc/drbd
is a virtual file displaying
real-time status information about all DRBD resources currently
configured. You may interrogate this file’s contents using this
command:
$ cat /proc/drbd version: 8.4.0 (api:1/proto:86-100) GIT-hash: 09b6d528b3b3de50462cd7831c0a3791abc665c3 build by linbit@buildsystem.linbit, 2011-10-12 09:07:35 0: cs:Connected ro:Secondary/Secondary ds:UpToDate/UpToDate C r----- ns:0 nr:0 dw:0 dr:656 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0 1: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r--- ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0 2: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r--- ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
The first line, prefixed with version:
, shows the DRBD version used
on your system. The second line contains information about this
specific build.
The other four lines in this example form a block that is repeated for
every DRBD device configured, prefixed by the device minor number. In
this case, this is 0
, corresponding to the device /dev/drbd0
.
The resource-specific output from /proc/drbd
contains various pieces
of information about the resource:
cs
(connection state). Status of the network connection. See
Section 6.1.3, “Connection states”for details about the various connection
states.
ro
(roles). Roles of the nodes. The role of the local node is
displayed first, followed by the role of the partner node shown after
the slash. See Section 6.1.4, “Resource roles”for details about the possible resource
roles.
ds
(disk states). State of the hard disks. Prior to the slash the
state of the local node is displayed, after the slash the state of the
hard disk of the partner node is shown. See Section 6.1.5, “Disk states”for
details about the various disk states.
Replication protocol. Replication protocol used by the resource. Either A
, B
or C
. See
Section 2.3, “Replication modes” for details.
I/O Flags. Six state flags reflecting the I/O status of this resource. See Section 6.1.6, “I/O state flags” for a detailed explanation of these flags.
Performance indicators. A number of counters and gauges reflecting the resource’s utilization and performance. See Section 6.1.7, “Performance indicators” for details.
A resource’s connection state can be
observed either by monitoring /proc/drbd
, or by issuing the drbdadm
cstate
command:
# drbdadm cstate <resource> Connected
A resource may have one of the following connection states:
StandAlone
. No network configuration available. The
resource has not yet been connected, or has been administratively
disconnected (using drbdadm disconnect
), or has dropped its
connection due to failed authentication or split brain.
Disconnecting
. Temporary state during disconnection. The
next state is StandAlone
.
Unconnected
. Temporary state, prior to a connection
attempt. Possible next states: WFConnection
and WFReportParams
.
Timeout
. Temporary state following a timeout in the
communication with the peer. Next state: Unconnected
.
BrokenPipe
. Temporary state after the connection to
the peer was lost. Next state: Unconnected
.
NetworkFailure
. Temporary state after the connection to
the partner was lost. Next state: Unconnected
.
ProtocolError
. Temporary state after the connection to
the partner was lost. Next state: Unconnected
.
TearDown
. Temporary state. The peer is closing the
connection. Next state: Unconnected
.
WFConnection
. This node is waiting until the peer node
becomes visible on the network.
WFReportParams
. TCP connection has been established, this
node waits for the first network packet from the peer.
Connected
. A DRBD connection has been established,
data mirroring is now active. This is the normal state.
StartingSyncS
. Full synchronization, initiated by the
administrator, is just starting. The next possible states are:
SyncSource
or PausedSyncS
.
StartingSyncT
. Full synchronization, initiated by the
administrator, is just starting. Next state: WFSyncUUID
.
WFBitMapS
. Partial synchronization is just
starting. Next possible states: SyncSource
or PausedSyncS
.
WFBitMapT
. Partial synchronization is just
starting. Next possible state: WFSyncUUID
.
WFSyncUUID
. Synchronization is about to begin. Next
possible states: SyncTarget
or PausedSyncT
.
SyncSource
. Synchronization is currently running, with
the local node being the source of synchronization.
SyncTarget
. Synchronization is currently running, with
the local node being the target of synchronization.
PausedSyncS
. The local node is the source of an ongoing
synchronization, but synchronization is currently paused. This may be
due to a dependency on the completion of another synchronization
process, or due to synchronization having been manually interrupted by
drbdadm pause-sync
.
PausedSyncT
. The local node is the target of an ongoing
synchronization, but synchronization is currently paused. This may be
due to a dependency on the completion of another synchronization
process, or due to synchronization having been manually interrupted by
drbdadm pause-sync
.
VerifyS
. On-line device verification is currently
running, with the local node being the source of verification.
VerifyT
. On-line device verification is currently
running, with the local node being the target of verification.
A resource’s role can be observed either by
monitoring /proc/drbd
, or by issuing the
drbdadm role
command:
# drbdadm role <resource> Primary/Secondary
The local resource role is always displayed first, the remote resource role last.
You may see one of the following resource roles:
Primary
. The resource is currently in the primary role, and may be read from
and written to. This role only occurs on one of the two nodes, unless
dual-primary mode is enabled.
Secondary
. The resource is currently in the secondary role. It normally receives
updates from its peer (unless running in disconnected mode), but may
neither be read from nor written to. This role may occur on one
or both nodes.
Unknown
. The resource’s role is currently unknown. The local resource role
never has this status. It is only displayed for the peer’s resource
role, and only in disconnected mode.
A resource’s disk state can be observed either by monitoring
/proc/drbd
, or by issuing the drbdadm dstate
command:
# drbdadm dstate <resource> UpToDate/UpToDate
The local disk state is always displayed first, the remote disk state last.
Both the local and the remote disk state may be one of the following:
Diskless
. No local block device has been assigned to the
DRBD driver. This may mean that the resource has never attached to its
backing device, that it has been manually detached using drbdadm
detach
, or that it automatically detached after a lower-level I/O
error.
Attaching
. Transient state while reading meta data.
Failed
. Transient state following an I/O failure report
by the local block device. Next state: Diskless
.
Negotiating
. Transient state when an Attach
is carried out on
an already-Connected
DRBD device.
Inconsistent
. The data is inconsistent. This status occurs
immediately upon creation of a new resource, on both nodes (before the
initial full sync). Also, this status is found in one node (the
synchronization target) during synchronization.
Outdated
. Resource data is consistent, but
outdated.
DUnknown
. This state is used for the peer disk if no
network connection is available.
Consistent
. Consistent data of a node without
connection. When the connection is established, it is decided whether
the data is UpToDate
or Outdated
.
UpToDate
. Consistent, up-to-date state of the data. This
is the normal state.
The I/O state flag field in /proc/drbd
contains information about
the current state of I/O operations associated with the
resource. There are six such flags in total, with the following
possible values:
r
for running or s
for suspended
I/O. Normally r
.
resync-after
dependency, this flag becomes a
. Normally -
.
p
. Normally -
.
u
. Normally -
.
Locally blocked I/O. Normally -
. May be one of the following
flags:
d
: I/O blocked for a reason internal to DRBD, such as a
transient disk state.
b
: Backing device I/O is blocking.
n
: Congestion on the network socket.
a
: Simultaneous combination of blocking device I/O and network congestion.
s
. Normally -
.
The second line of /proc/drbd
information for each resource contains
the following counters and gauges:
ns
(network send). Volume of net data sent to the partner via the network connection; in
Kibyte.
nr
(network receive). Volume of net data received by the partner via the network connection;
in Kibyte.
dw
(disk write). Net data written on local hard disk; in Kibyte.
dr
(disk read). Net data read from local hard disk; in Kibyte.
al
(activity log). Number of updates of the activity log area of the meta data.
bm
(bit map). Number of updates of the bitmap area of the meta data.
lo
(local count). Number of open requests to the local I/O sub-system issued by DRBD.
pe
(pending). Number of requests sent to the partner, but that have not yet been
answered by the latter.
ua
(unacknowledged). Number of requests received by the partner via the network connection,
but that have not yet been answered.
ap
(application pending). Number of block I/O requests forwarded to DRBD, but not yet answered
by DRBD.
ep
(epochs). Number of epoch objects. Usually 1. Might increase under I/O load when
using either the barrier
or the none
write ordering method.
wo
(write order). Currently used write ordering method: b
(barrier), f
(flush),
d
(drain) or n
(none).
oos
(out of sync). Amount of storage currently out of sync; in Kibibytes.