| Thanks For the swift reply Richard!
Ive just realised it’s you blog I’ve been reading! (Love the 'about the author' bit ; ) )
Hopefully get this working this weekend. Swamped this afternoon.
So if I want to export the device instead of separate directories.
Could I change the config file to something like this
[generic] # If you want to run everything as root rather than the nbd user, you # may either say "root" in the two following lines, or remove them # altogether. Do not remove the [generic] section, however. user = pi group = pi includedir = /etc/nbd-server/conf.d allowlist = true
# What follows are export definitions. You may create as much of them as # you want, but the section header has to be unique. [rpi-C3-food] exportname = /dev/sda transactionlog = /home/pi/media/500G_FAITH/logs/log_food readonly = false
Then on client > sudo nbd-client 192.168.1.14 -N rpi-C3-food /dev/nbd1 Then mount it. > mkdir /mnt/food > mount /dev/nbd1 /mnt/food
Thanks all really appreciate the help. Simon
PS I tried the list option - gave me an error : / nbdinfo: nbd_set_export_name: invalid state: READY: the handle must be newly created, or negotiating: Invalid argument
On Thu, Aug 26, 2021 at 07:49:33PM +0100, Simon Fernandez wrote:Hi folks,
thanks for the reply Wouter's reply was in junk only found it when I saw
Richards reply and fished it out.
I'm still getting
$ sudo nbd-client 192.168.1.14 10809 -N recipes /dev/nbd0
Negotiation: ..size = 0MB
Connected /dev/nbd0
Here are the changes I've made - first I built nbdinfo - thanks for the tip
Richard.
Build GnuTLS from
https://www.gnupg.org/ftp/gcrypt/gnutls/v3.6/gnutls-3.6.16.tar.xz
Build libnbd from
https://download.libguestfs.org/libnbd/1.8-stable/libnbd-1.8.3.tar.gz
After installing quite a lot of libraries . . .
----------------------------------------------------------------------
Thank you for downloading libnbd 1.8.3
This is how we have configured the optional components for you today:
Optional library features:
TLS support ............................ yes
NBD URI support ........................ yes
FUSE support ........................... yes
Manual pages ........................... yes
Bash tab completion .................... yes
Language bindings:
Go ..................................... no
OCaml .................................. no
Python ................................. no
nbdkit version note - during ./configure
checking for nbdkit... nbdkit
checking for nbdkit >= 1.12... no (1.10)
configure: WARNING: nbdkit is too old, some tests will be disabled
The 'make check' skipped a lot of tests but there were no FAILS
$ nbdinfo --version
nbdinfo 1.8.3
libnbd 1.8.3
== rpi-C3 (192.168.1.14) server side
Q: Wouter ( w@uter.be ) - Are they readable and writable by the "nbd" user?
R: I've changed the user & group to pi (the logged in user) so should be R/W
Note: Richard ( rjones@redhat.com ) - You don't need to serve each export on a
separate port. The NBD
protocol can serve multiple exports (disks) on the same port, with the
client requesting which one it wants.
R: Have remove the port addresses from config
pi@rpi-C3:~ $ cat /etc/nbd-server/config
[generic]
# If you want to run everything as root rather than the nbd user, you
# may either say "root" in the two following lines, or remove them
# altogether. Do not remove the [generic] section, however.
user = pi
group = pi
includedir = /etc/nbd-server/conf.d
allowlist = true
# What follows are export definitions. You may create as much of them as
# you want, but the section header has to be unique.
[images]
exportname = /home/pi/media/500G_FAITH/images/
transactionlog = /home/pi/media/500G_FAITH/logs/log_images
[recipes]
exportname = /home/pi/media/500G_FAITH/recipes/
transactionlog = /home/pi/media/500G_FAITH/logs/log_recipes
[movie]
exportname = /home/pi/media/500G_FAITH/movies/
transactionlog = /home/pi/media/500G_FAITH/logs/log_movie
One problem here is that NBD is a block device export protocol, not afile serving protocol (like eg NFS or SMB).You can however turn a directory into an NBD export usingnbdkit-floppy-plugin, eg: $ nbdkit floppy /home/pi/media/500G_FAITH/recipes/(https://libguestfs.org/nbdkit-floppy-plugin.1.html)That exports a VFAT filesystem as a block device which has to bemounted: # nbd-client 192.168.1.14 10809 /dev/nbd0 # mkdir /mnt/recipes # mount /dev/nbd0 /mnt/recipesQ: Wouter ( w@uter.be ) - Do the paths that you point to in the exportname
parameters exist?
R: YES its a directory with a bunch of images in it. YES - Non zero size.
pi@rpi-C3:~ $ ls /home/pi/media/500G_FAITH/recipes/
y000_Pork_Belly_Wontan____TOCOST.rtf y011_Rack_And_Black.rtf
y002_Salmon_Parfait_Forest_Rosemary_Bread_LE.rtf
y021_StoneAge_Beef_Taragon_Sauce____TOCOST.rtf
pi@rpi-C3:~ $ ls /home/pi/media/500G_FAITH/images
160g? of dough - 250C - 6m.jpg 20190228_163410_monkfish and red pepper
skewers.jpg
20190629_202814_couscous w apricots.jpg 672_V_cheddar_gorgeous.jpg
20181022_134259.jpg 20200124_152329_aubergine & pesto stuffing.jpg
25_androll_into_pancakes.jpg 665 Celeriac & Cockscombe Mint Soup.jpg
pi@rpi-C3:~ $ ls /home/pi/media/500G_FAITH/movies
NA #015 - Bread.mp4
NA #016 - Fisetin - Mayo Clinic Trials.mp4
NA #042 - Delta Variant COVID.mp4
NA #044 - Lex Fridman.mp4
pi@rpi-C3:~ $ ls -la /home/pi/media/500G_FAITH/logs
-rw------- 1 pi pi 56 Aug 26 17:00 log_images
-rw------- 1 pi pi 56 Aug 26 17:00 log_movie
-rw------- 1 pi pi 700 Aug 26 17:35 log_recipes
pi@rpi-C3:~ $ cat /home/pi/media/500G_FAITH/logs/log_images
%`?%`?
pi@rpi-C3:~ $ cat /home/pi/media/500G_FAITH/logs/log_movie
%`?%`?
pi@rpi-C3:~ $ cat /home/pi/media/500G_FAITH/logs/log_recipes
%`?`%`?a%`?b%`?c%`?d%`?e%`?f%`?g%`?h%`?i%`?j%`?k%`?l%`?%`?%`?%`?%`?%`?%`?%`?%`%
`? %`?
pi@rpi-C3:~ $ nbdkit --version
nbdkit 1.1.12
== client side
Note: Richard ( rjones@redhat.com ) - Use the 'nbdinfo' tool to examine what is
being served by an NBD
server. It can query all sorts of information as described in the manual:
R: Built the tool. - here is what it says : /
pi@rpi-C1:~ $ nbdinfo nbd://192.168.1.14:10809/recipes
protocol: newstyle-fixed without TLS
export="recipes":
export-size: 4096
uri: nbd://192.168.1.14:10809/recipes
is_rotational: false
is_read_only: true
can_cache: false
can_df: false
can_fast_zero: false
can_flush: false
can_fua: false
can_multi_conn: true
can_trim: false
can_zero: true
This looks like a good "recipes" export, although the size isunusually small (4K), which is probably because what you're exportinghere is the directory inode, not the directory - see above.pi@rpi-C1:~ $ nbdkit --version
nbdkit 1.10.3
pi@rpi-C1:~ $ nbd-client -l 192.168.1.14
Negotiation: ..
images
recipes
movie
nbdinfo can also list exports using the --list option which will tellyou a lot of detail about every export.pi@rpi-C1:~ $ sudo modprobe nbd
pi@rpi-C1:~ $ lsmod | grep nbd
nbd 49152 0
pi@rpi-C1:~ $ sudo nbd-client 192.168.1.14 10809 -N recipes /dev/nbd0
Negotiation: ..size = 0MB
Connected /dev/nbd0
pi@rpi-C1:~ $ sudo partx -a /dev/nbd0
partx: /dev/nbd0: failed to read partition table
pi@rpi-C1:~ $ sudo parted -l
Error: /dev/nbd0: unrecognised disk label
Model: Unknown (unknown)
Disk /dev/nbd0: 4096B
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:
pi@rpi-C1:~ $ dmesg | tail -n 20
[ 2138.454532] blk_update_request: I/O error, dev nbd0, sector 0 op 0x0:(READ)
flags 0x0 phys_seg 1 prio class 0
[ 2138.454547] Buffer I/O error on dev nbd0, logical block 0, async page read
[ 2138.457380] block nbd0: Other side returned error (22)
[ 2138.457497] blk_update_request: I/O error, dev nbd0, sector 0 op 0x0:(READ)
flags 0x0 phys_seg 1 prio class 0
[ 2138.457520] Buffer I/O error on dev nbd0, logical block 0, async page read
[ 2138.459238] block nbd0: Other side returned error (22)
[ 2138.459280] blk_update_request: I/O error, dev nbd0, sector 0 op 0x0:(READ)
flags 0x0 phys_seg 1 prio class 0
[ 2138.459297] Buffer I/O error on dev nbd0, logical block 0, async page read
[ 2138.460993] block nbd0: Other side returned error (22)
[ 2138.461023] blk_update_request: I/O error, dev nbd0, sector 0 op 0x0:(READ)
flags 0x0 phys_seg 1 prio class 0
[ 2138.461038] Buffer I/O error on dev nbd0, logical block 0, async page read
[ 2138.462649] block nbd0: Other side returned error (22)
[ 2138.462685] Buffer I/O error on dev nbd0, logical block 0, async page read
[ 2138.464344] block nbd0: Other side returned error (22)
[ 2138.466926] block nbd0: Other side returned error (22)
[ 2138.468506] block nbd0: Other side returned error (22)
[ 2138.470066] block nbd0: Other side returned error (22)
[ 2138.471442] block nbd0: Other side returned error (22)
[ 2138.472978] block nbd0: Other side returned error (22)
[ 2138.474448] block nbd0: Other side returned error (22)
pi@rpi-C1:~ $ sudo nbd-client 192.168.1.14 10809 -N movies /dev/nbd1
Negotiation: ..Error: E: server does not support NBD_OPT_GO and dropped
connection after sending NBD_OPT_EXPORT_NAME. Try -g.
Exiting.
I'm attempting to serve non empty directories so I don't know why it's saying
0Mb size?
Scratching my head here . . any thoughts? :/
Rich.Thanks in advance,
Simon
On 12 Aug 2021, at 22:32, Richard W.M. Jones <rjones@redhat.com> wrote:
On Thu, Jul 22, 2021 at 01:41:00PM +0100, Simon Fernandez wrote:
Hi folks,
I’m trying setup a client that connect to a 500Gb ext4 disk on another
machine
running nbd server.
Server: raspberry pi 3: uname -r 4.19.66+
Client: raspberry pi 3: uname -r 5.10.17-v7+
I’ve followed the information here [https://github.com/
NetworkBlockDevice/nbd]
and a lot of other sites but I think I’m doing something wrong on the
server
setup because I get this on the client:
$ sudo mount /dev/nbd2 /home/pi/Vols/nbd1-recipes/
mount: /home/pi/Vols/nbd1-recipes: can't read superblock on /dev/nbd2
Also it appears to be listening on default port - NOT ones specified in
config
file.
This is how I’ve set up the server & client, it’s probably something
obvious to
more experienced folk.
Wouter has answered this already, I just wanted to add a couple of
smaller points:
(1) You don't need to serve each export on a separate port. The NBD
protocol can serve multiple exports (disks) on the same port, with the
client requesting which one it wants.
(2) Use the 'nbdinfo' tool to examine what is being served by an NBD
server. It can query all sorts of information as described in the
manual:
https://libguestfs.org/nbdinfo.1.html
Rich.
== rpi-C3 (192.168.1.14) server setup
sudo apt-get update
sudo apt-get install nbd-server
sudo nano /etc/nbd-server/config
[generic]
# If you want to run everything as root rather than the nbd user, you
# may either say "root" in the two following lines, or remove them
# altogether. Do not remove the [generic] section, however.
user = nbd
group = nbd
includedir = /etc/nbd-server/conf.d
allowlist = true
# What follows are export definitions. You may create as much of them
as
# you want, but the section header has to be unique.
[images]
exportname = /home/pi/media/500G_FAITH/images
port = 510029
[recipes]
exportname = /home/pi/media/500G_FAITH/recipes
port = 510030
[movie]
exportname = /home/pi/media/500G_FAITH/movies
port = 510031
sudo modprobe nbd # NOT nbd-server!
lsmod | grep nbd
nbd 40960 0
sudo service nbd-server force-reload # if already running
== client side
sudo apt-get install nbd-client # done
sudo modprobe nbd
nbd-client 192.168.1.14 -l
Negotiation: ..
images
recipes
movie
sudo nbd-client 192.168.1.14 -N recipes /dev/nbd2
Negotiation: ..size = 0MB
Connected /dev/nbd2
sudo nbd-client 192.168.1.14 -N movie /dev/nbd3
Negotiation: ..size = 0MB
Connected /dev/nbd3
sudo nbd-client 192.168.1.14 -N images /dev/nbd1
Negotiation: ..size = 0MB
Connected /dev/nbd1
dmesg | tail -n 10
[ 1417.883364] Buffer I/O error on dev nbd1, logical block 0, async
page read
[ 1417.884614] block nbd1: Other side returned error (22)
[ 1417.884652] blk_update_request: I/O error, dev nbd1, sector 0 op
0x0:(READ)
flags 0x0 phys_seg 1 prio class 0
[ 1417.884673] Buffer I/O error on dev nbd1, logical block 0, async
page read
[ 1417.885977] block nbd1: Other side returned error (22)
[ 1417.886016] blk_update_request: I/O error, dev nbd1, sector 0 op
0x0:(READ)
flags 0x0 phys_seg 1 prio class 0
[ 1417.886036] Buffer I/O error on dev nbd1, logical block 0, async
page read
[ 1417.887276] block nbd1: Other side returned error (22)
[ 1417.888492] block nbd1: Other side returned error (22)
[ 1417.889751] block nbd1: Other side returned error (22)
sudo nbd-client 192.168.1.14 -d /dev/nbd1 # thought it might be
something to
do w/ nbd1
sudo nbd-client 192.168.1.14 -N images /dev/nbd4
Negotiation: ..size = 0MB
Connected /dev/nbd4
dmesg | tail -n 10
[ 1848.831897] block nbd4: Other side returned error (22)
[ 1848.833341] block nbd4: Other side returned error (22)
[ 1848.834651] block nbd4: Other side returned error (22)
[ 1848.836002] block nbd4: Other side returned error (22)
lsblk -f
NAME FSTYPE LABEL UUID
FSAVAIL
FSUSE% MOUNTPOINT
sda
├─sda1 vfat EFI 67E3-17ED
└─sda2 hfsplus time_box_2018 f6427a3d-363b-3284-88c4-03ce3493aeff
115.5G
94% /media/pi/time_box_2018
nbd2
nbd3
nbd4
mmcblk0
├─mmcblk0p1 vfat boot 5DE4-665C
203.4M
19% /boot
└─mmcblk0p2 ext4 rootfs 7295bbc3-bbc2-4267-9fa0-099e10ef5bf0
4.1G
37% /
dmesg | tail -n 10
[ 1848.836002] block nbd4: Other side returned error (22)
[ 1848.837430] block nbd4: Other side returned error (22)
[ 1848.838860] block nbd4: Other side returned error (22)
[ 1848.840224] block nbd4: Other side returned error (22)
[ 1848.841691] block nbd4: Other side returned error (22)
[ 1848.843077] block nbd4: Other side returned error (22)
[ 1848.844415] block nbd4: Other side returned error (22)
[ 2802.615037] block nbd3: Receive control failed (result -32)
[ 2802.625871] block nbd2: Receive control failed (result -32)
[ 2802.648444] block nbd4: Receive control failed (result -32)
ls /home/pi/Vols/
nbd1-recipes/ rpi-C2-nbd-500G/ tbx2018/ time_box_2018/
sudo mount /dev/nbd2 /home/pi/Vols/nbd1-recipes/
mount: /home/pi/Vols/nbd1-recipes: can't read superblock on /dev/nbd2.
sudo parted -l
Model: Seagate BUP Slim SL (scsi)
Disk /dev/sda: 2000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name
Flags
1 20.5kB 210MB 210MB fat32 EFI System Partition boot,
esp
2 210MB 2000GB 2000GB hfs+ time_box_2018
Error: /dev/nbd3: unrecognised disk label
Model: Unknown (unknown)
Disk /dev/nbd3: 4096B
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:
Error: /dev/nbd4: unrecognised disk label
Model: Unknown (unknown)
Disk /dev/nbd4: 4096B
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:
Error: /dev/nbd2: unrecognised disk label
Model: Unknown (unknown)
Disk /dev/nbd2: 4096B
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:
Model: SD SL08G (sd/mmc)
Disk /dev/mmcblk0: 7948MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 4194kB 273MB 268MB primary fat32 lba
2 273MB 7948MB 7676MB primary ext4
I’ve been trying to get them to talk for 2 days now
One thing I did notice is that the port numbers in the config file
don’t seem
to change the fact it’s listening on he default port??
$ sudo lsof -i -P -n | grep LISTEN
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
cnid_meta 312 root 3u IPv6 11170 0t0 TCP [::1]:4700
(LISTEN)
afpd 324 root 3u IPv4 11195 0t0 TCP *:548 (LISTEN)
sshd 426 root 3u IPv4 12126 0t0 TCP *:22 (LISTEN)
sshd 426 root 4u IPv6 12128 0t0 TCP *:22 (LISTEN)
dnsmasq 431 dnsmasq 5u IPv4 12046 0t0 TCP *:53 (LISTEN)
dnsmasq 431 dnsmasq 7u IPv6 12048 0t0 TCP *:53 (LISTEN)
nbd-serve 2026 nbd 3u IPv6 20100 0t0 TCP *:10809
(LISTEN)
I’d be very grateful for any pointers, or other resources that my be of
help.
Thanks in advance,
Simon
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/
~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjonesRead my programming and virtualization blog: http://rwmj.wordpress.comvirt-top is 'top' for virtual machines. Tiny program with manypowerful monitoring features, net stats, disk stats, logging, etc.http://people.redhat.com/~rjones/virt-top
|