LINBIT recently released their new solution for orchestrating and managing multiple DRBD arrays.
 
For example, you can have several nodes and each will have its own LVM or ZFS pool in which LINSTOR will automatically create new volumes and replicate them between nodes using the DRBD protocol.
 
LINSTOR supports thin-provisioning, snapshots and many other interesting things.
 
This solution is well suited for virtual machines and containers.
 
LINSTOR Satellites
 
I believe that you already have a Proxmox cluster, I have three nodes pve1 , pve2 and pve3 .
 

So the first step is to install drbd-utils and collect the core module is on each node.

Let's add the linstor repository and install kernel-headers , as well as all the necessary packages from it:

    wget -O- https://packages.linbit.com/package-signing-pubkey.asc | apt-key add -
echo "deb http://packages.linbit.com/proxmox/proxmox-5 drbd-9.0"
> /etc/apt/sources.list.d/linbit.list
apt-get update
apt-get -y install pve-headers
apt-get -y install drbd-dkms drbdtop    

After installation, let's check your kernel module version:

    modproble drbd
cat /proc /drbd    

If you see version ? then something went wrong and you booted the in-tree kernel module, check dkms status that would learn more about the assembly of the dkms module itself.
 
For LINSTOR, you must use the DRBD 9 version.

Next, on each node, set linstor-proxmox package:

    apt-get -y install linstor-proxmox
systemctl start linstor-satellite.service
systemctl enable linstor-satellite.service    

LINSTOR Controller

We will deploy the controller inside the LXC container.

Download the debian-template:

    wget http://download.proxmox.com/images/system/debian-9.0-standard_9.3-1_amd64.tar.gz -P /var /lib /vz /template /cache /  

Now create the container for the controller:

    pct create 100 local: vztmpl /debian-9.0-standard_9.3-1_amd64.tar.gz
--hostname = linstor-controller
--net0 = name = eth? bridge = vmbr? gw = ???.? ip = ???.123 /16    

Start the container and enter it:

    pct start 100
pct exec 100 bash    

Install the updates:

    apt-get update
apt-get -y upgrade    

Add the linstor repository and install linstor-controller and linstor-client :

    wget -O- https://packages.linbit.com/package-signing-pubkey.asc | apt-key add -
echo "deb http://packages.linbit.com/proxmox/proxmox-5 drbd-9.0"
> /etc/apt/sources.list.d/linbit.list
apt-get update && apt-get install -y linstor-server linstor-client
systemctl start linstor-controller.service
systemctl enable linstor-controller.service    

Linstor requires a configured locale. Set up the locale:

    sed -i '/en_US.UTF-8 UTF-8 /s /^ # //' /etc/locale.gen
locale-gen    

Let's immediately configure the time zone:

    dpkg-reconfigure tzdata    

Configuring the store

Let's create the nodes:

    linstor node create pve???.???r3r3468. linstor node create pve???.???r3r3468. linstor node create pve???.???r3r3461.  

For each node, we describe the additional interface that will be used for DRBD replication:

    linstor node interface create pve1 data ???.11
linstor node interface create pve2 data ???.12
linstor node interface create pve3 data ???.13    

The output of the command is linstor node list :

    ╭───────────────────────────────────────────────────────────────────────────────────────────────────
┊ Node ┊ NodeType ┊ Addresses ┊ State ┊
╞┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄ ┄┄┄┄┄┄┄┄┄┄┄┄┄╡
┊ pve1 ┊ SATELLITE ┊ ???.1????.11: 3366 (PLAIN) ┊ Online ┊
┊ pve2 ┊ SATELLITE ┊ ???.1????.12: 3366 (PLAIN) ┊ Online ┊
┊ pve3 ┊ SATELLITE ┊ ???.1????.13: 3366 (PLAIN) ┊ Online ┊
╰───────────────────────────────────────────────────────────────────────────────────────────────────    

Let's create the pools:

On each node you must prepare lvm thin-pool for drbd:

    lvcreate -L 800G --thinpool drbdpool pve    

Now you can add them to linstor:

    linstor storage-pool create lvmthin pve1 drbdpool pve /drbdpool
linstor storage-pool create lvmthin pve2 drbdpool pve /drbdpool
linstor storage-pool create lvmthin pve3 drbdpool pve /drbdpool    

We will configure the pools to use data interface:

    linstor storage-pool set-property pve1 drbdpool PrefNic data
linstor storage-pool set-property pve2 drbdpool PrefNic data
linstor storage-pool set-property pve3 drbdpool PrefNic data    

The output of the command is linstor storage-pool list :

    ╭─────────────────────────────────────────────────────────────────────────────────────────────────── ─────────────────────────────────────────────────────────────────────────────────────────────────── ┊ StoragePool ┊ Node ┊ Driver ┊ PoolName ┊ FreeCapacity ┊ TotalCapacity ┊ Support
╞┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄ ┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
┊ drbdpool ┊ pve1 ┊ LvmThinDriver ┊ hv1 /drbdpool ┊ 800 GiB ┊ 800 GiB ┊ true
┊ drbdpool ┊ pve1 ┊ LvmThinDriver ┊ hv1 /drbdpool ┊ 800 GiB ┊ 800 GiB ┊ true
┊ drbdpool ┊ pve3 ┊ LvmThinDriver ┊ hv3 /drbdpool ┊ 800 GiB ┊ 800 GiB ┊ true
╰─────────────────────────────────────────────────────────────────────────────────────────────────── ─────────────────────────────────────────────────────────────────────────────────────────────────  

Now we can add our new repository to the Proxmox config:

    cat /etc/pve/storage.cfg EOF
drbd: drbdpool
content rootdir, images
controller ???.123
controllervm 100
redundancy 3
EOF    

HA setting for the

Now we'll move our linstor container to linstor storage.

First we need to create a virtual disk for it:

    pct exec 100 bash
linstor resource-definition create vm-100-disk-1
linstor volume-definition create vm-100-disk-1 4G
linstor resource create vm-100-disk-1 --auto-place 3 -s pve    

The output of the command is linstor resource list :

    ╭────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ ResourceName ┊ Node ┊ Port ┊ State ┊
╞┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╡
┊ vm-100-disk-1 ┊ pve1 ┊ 7000 ┊ UpToDate ┊
┊ vm-100-disk-1 ┊ pve2 ┊ 7000 ┊ UpToDate ┊
┊ vm-100-disk-1 ┊ pve3 ┊ 7000 ┊ UpToDate ┊
╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯  

Default is linstor-satellite deletes all resource configurations at the time the service was started, then it expects a new configuration from linstor-controller , but if we keep linstor-controller in the same place as the data, we must make an exception for its resource by adding the option --keep-res = vm-100 for linstor-satellite on all nodes:

    echo -e "[Service]nExecStart = nExecStart = /usr /share /linstor-server /bin /Satellite -logs = /var /log /linstor-satellite --config-directory = /etc /linstor --keep-res = vm-100 "| SYSTEMD_EDITOR = tee systemctl edit linstor-satellite.service    

This action should prevent the resource from being deleted. vm-100-disk-1 at each start.

Now we will stop linstor-controller container and transfer all data from the local disk to the drbd-disk.

    pct shutdown 100
dd if = /var /lib /vz /images /100 /vm-100-disk-1.raw of = /dev /drbd /by-res /vm-100-disk-1/0 bs = 8M status = progress
. e2fsck -f /dev /drbd /by-res /vm-100-disk-1/0
resize2fs /dev /drbd /by-res /vm-100-disk-1/0    

Update the container configuration:

    sed -i '/^ rootfs: /d' /etc/pve/lxc/100.conf
echo 'rootfs: drbdpool: vm-100-disk-? size = 4G' /etc/pve/lxc/100.conf    

Run the container, and if everything is ok, delete the old disk:

    pct start 100
rm -f /var/lib/vz/images/100/vm-100-disk-1.raw    

Now we just need to add our container to the proxmox ha-manager :

    ha-manager add ct: 100 --max_relocate = 3 --max_restart = 3    

PROFIT

Performance and tuning

To automatically resolve the situation with the split-brain, add the following options to the controller:

    linstor controller drbd-options
--after-sb-0pri = discard-zero-changes
--after-sb-1pri = discard-secondary
--after-sb-2pri = disconnect    

For my 10G network, I found the following settings the most optimal for fast synchronization:

    linstor controller drbd-options
--max-buffers = 36864
--rcvbuf-size = 2097152
--sndbuf-size = 1048576
linstor controller drbd-options
--c-fill-target = 10240
- c-max-rate = 737280
-c-min-rate = 20480
--c-plan-ahead = 10