Installer Bugs & Problems (especially MDADM+LUKS+LVM setups)

[maddes]

The Installer DVDs (here Multimedia 4.5) and the Installer's Expert Mode are intended for the advanced user but have several issues:

• Installer DVD does not ask for the wanted keyboard layout, e.g. which is necessary for entering passwords of users and/or encrypted volumes
  Workaround: right click task bar > Add/Remove Panel Items > add Keyboard Layout Handler > in preferences add wanted keyboard layout > move wanted layout to top
• Missing packages for advanced partition setups to be mounted under /target, e.g. MDADM+LUKS+LVM: mdadm, cryptsetup, lvm2
  Workaround: start root terminal, apt-get update, apt-get install mdadm cryptsetup lvm2 (has to be redone in chroot /target again)
• Example fstab is not created in /target/etc in Expert Mode
• Sparky plymouth theme doesn't allow for input of encryption password. Switch to text theme or any Debian theme (e.g. solar).

Wishes for upcoming release:

• Add packages mdadm, lvm2 and cryptsetup to all installer and rescue DVDs.
• Add functionality to choose /dev/mapper devices in normal mode.
• Install packages mdadm, lvm2 and cryptsetup in normal mode to /target.
• Create best-estimation /etc/fstab from mounts under /target.
• Sparky Plymouth theme that allows for input boxes.

This would allow for a much easier installation on MDADM+LUKS+LVM setups.
Advanced users would only have to care about crypttab and mdadm.conf plus update initramfs, just like it is with Ubuntu.



Further tips:

• (normally already present in Expert mode) DNS resolution in chroot /target: cp -v /etc/resolv.conf /target/etc/
• (normally already present in Expert mode) Necessary mounts for chroot /target:
  
  
  
  • mount -o rbind /dev /target/dev
  • mount -t proc proc /target/proc
  • mount -t sysfs sys /target/sys
• Additional necessary mounts for chroot /target after LVM is installed in chroot:
  
  
  
  • mount -o rbind /run/lvm /target/run/lvm
  • mount -o rbind /run/lock/lvm /target/run/lock/lvm
• (normally done in the end phase of Expert mode) Update initramfs after maintaining fstab, crypttab and/or mdadm.conf: inside chroot /target, update-initramfs -u -k all
• (normally not necessary) Force a keyboard layout in initramfs: inside chroot /target, nano /etc/default/keyboard (XKBLAYOUT="xx"), nano /etc/initramfs-tools/initramfs.conf (KEYMAP=y), update-initramfs -u -k all

[maddes]

Additionally the Installer also fails from time to time, especially on the Multimedia ISO 4.5, as described in the 4.5 release announcement under "Known Issues".
Although only the 32bit version is mentioned in the "Known Issues", this is also the case for the 64 bit versions.

Instead of asking if you want to reboot to the installed system, the installer just exits.
Just run the installer with the same options again, especially the indexing will be much faster.

This didn't help when installing the Multimedia Edition onto a LUKS+LVM setup, maybe the encryption increases the time too much to get through the installation process without hitting the crash.
My workaround was:

• install SparkyLinux Multimedia 4.5 to a normal partition with a separate boot partition
• boot SparkyLinux
• install cryptsetup+lvm2+mdadm
• prepare /etc/crypttab
• open LUKS partition
• format LVM rootfs volume with mkfs.ext4
• mount LVM rootfs volume to /target
• create /target/home and mount LVM homefs volume to /target/home
• create /target/boot and mount boot partition to /target/boot
• copy rootfs from separate temporary partition via cp -avx /. /target/
• adopt /target/etc/fstab to LVM volumes
• mount/bind necessary fs for chroot to /target (see previous post: /dev /proc /sys /run)
• chroot /target
• update-initramfs -u -k all
• update-grub
• reboot

To trace the live-installer, just call it from a root terminal.

Code Select Expand

root@live:/home/live# live-installer --sync
/usr/lib/live-installer/frontend/gtk_interface.py:58: GtkWarning: Unable to locate theme engine in module_path: "murrine",
  self.wTree = gtk.glade.XML(self.glade, 'main_window')
Error adding locale 'awk: warning: escape sequence `\.' treated as plain `.''
Error adding locale 'anp_IN'
Error adding locale 'ayc_PE'
Error adding locale 'ast_ES'
Error adding locale 'bem_ZM'
Error adding locale 'ber_DZ'
Error adding locale 'ber_MA'
Error adding locale 'bhb_IN'
Error adding locale 'bho_IN'
Error adding locale 'brx_IN'
Error adding locale 'byn_ER'
Error adding locale 'chr_US'
Error adding locale 'cmn_TW'
Error adding locale 'crh_UA'
Error adding locale 'csb_PL'
Error adding locale 'doi_IN'
Error adding locale 'fil_PH'
Error adding locale 'fur_IT'
Error adding locale 'gez_ER'
Error adding locale 'gez_ET'
Error adding locale 'hak_TW'
Error adding locale 'hne_IN'
Error adding locale 'hsb_DE'
Error adding locale 'kok_IN'
Error adding locale 'lij_IT'
Error adding locale 'lzh_TW'
Error adding locale 'mag_IN'
Error adding locale 'mai_IN'
Error adding locale 'mhr_RU'
Error adding locale 'mni_IN'
Error adding locale 'nan_TW'
Error adding locale 'nds_DE'
Error adding locale 'nds_NL'
Error adding locale 'nhn_MX'
Error adding locale 'niu_NU'
Error adding locale 'niu_NZ'
Error adding locale 'nso_ZA'
Error adding locale 'pap_AW'
Error adding locale 'pap_CW'
Error adding locale 'quz_PE'
Error adding locale 'raj_IN'
Error adding locale 'sat_IN'
Error adding locale 'sgs_LT'
Error adding locale 'shs_CA'
Error adding locale 'sid_ET'
Error adding locale 'szl_PL'
Error adding locale 'tcy_IN'
Error adding locale 'the_NP'
Error adding locale 'tig_ER'
Error adding locale 'unm_US'
Error adding locale 'wae_CH'
Error adding locale 'wal_ET'
Error adding locale 'yue_HK'
/usr/lib/live-installer/frontend/gtk_interface.py:656: Warning: Source ID 4294967295 was not found when attempting to remove it
  glib.source_remove(self.kbd_preview_generation)  # stop previous preview generation, if any
openjdk version "1.8.0_111"
OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-3-b14)
OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
Timezone: Europe/Berlin (UTC+0100) (402, 131)
Color: #552200, Image: timezone_1.0.png
/usr/lib/live-installer/frontend/gtk_interface.py:656: Warning: Source ID 30 was not found when attempting to remove it
  glib.source_remove(self.kbd_preview_generation)  # stop previous preview generation, if any
Excluding /dev/sr (detected as the live device)
Can't find model in blkid output: ['disk', 'fd0', '1', '4K']
Can't find model in blkid output: ['loop', 'loop0', '0', '2.4G']
Disks:  [('/dev/sda', 'Samsung SSD 750 (250 GB)'), ('/dev/sdb', 'SAMSUNG HD642JJ (640 GB)')]
mount: unknown filesystem type 'crypto_LUKS'
WARNING: Partition /dev/sda2 or type 0 failed to mount!
mount: /dev/sda3 is already mounted or /tmp/live-installer/tmpmount busy
       /dev/sda3 is already mounted on /target
No passwd entry for user 'sparky'
mount: wrong fs type, bad option, bad superblock on /dev/sdb1,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.
WARNING: Partition /dev/sdb1 or type 0 failed to mount!
## INSTALLATION
Template path: /usr/share/live-installer/slideshow/template.html
--> Installation started
umount: /target/dev/shm: mountpoint not found
umount: /target/dev/pts: mountpoint not found
umount: /target/dev/: not mounted
umount: /target/sys/: not mounted
umount: /target/proc/: not mounted
EXECUTING: 'mkfs.ext4 -F /dev/sda3'
--> Mounting partitions
------ Mounting /lib/live/mount/medium/live/filesystem.squashfs on /source/
EXECUTING: 'mount -o loop -t squashfs /lib/live/mount/medium/live/filesystem.squashfs /source/'
------ Mounting partition /dev/sda3 on /target/
EXECUTING: 'mount -t ext4 /dev/sda3 /target'
--> Indexing files
--> Copying files
/usr/lib/live-installer/frontend/gtk_interface.py:1012: GtkWarning: Failed to set text from markup due to error parsing markup: Error on line 1: Entity did not end with a semicolon; most likely you used an ampersand character without intending to start an entity - escape ampersand as &
  self.wTree.get_widget("label_install_progress").set_label(message)
The program 'main.py' received an X Window System error.
This probably reflects a bug in the program.
The error was 'BadDrawable (invalid Pixmap or Window parameter)'.
  (Details: serial 15328975 error_code 9 request_code 62 minor_code 0)
  (Note to programmers: normally, X errors are reported asynchronously;
   that is, you will receive the error a while after causing it.
   To debug your program, run it with the --sync command line
   option to change this behavior. You can then get a meaningful
   backtrace from your debugger if you break on the gdk_x_error() function.)


[maddes]

How to test this Calamares installer I read about? - Calamares is available in the Sparky testing builds.

Tested SparkyLinux 4.5-dev20161107 that uses Calamares, which doesn't support to install on a LUKS+LVM setup (or I just didn't find the trick to do it). Also mdadm, cryptsetup and lvm were not present after the installer DVD loaded.

Same wishes apply for Calamares release:

• Add packages mdadm, lvm2 and cryptsetup to all installer and rescue DVDs.
• Add functionality to choose /dev/mapper devices in installer.
• Install packages mdadm, lvm2 and cryptsetup in /target.

[pavroo]

Calamares 2.4.4 is pre-installed on Sparky dev:
https://sparkylinux.org/download/testing/

I've been working with Calamares 3.0.1 now, so if everything goes fine, a new dev iso should be ready soon.

[maddes]

@pavroo:
Tested everything possible and updated all above posts accordingly.
Maybe you can reply here when the new dev build is out, so I can immediately test it.

[pavroo]

I still fight with Calamares 3.0.1.
Works fine on Bios, some problems on UEFI and on encrypted disk.
Let you known when something be ready so you could test and suggest whatever you find.

[pavroo]

New development iso images 4.6-dev20170202 with Calamares 3.0.1 are already on sourceforge:
https://sourceforge.net/projects/sparkylinux/files/testing/
Work fine on Bios machines.
Known issues:
- problem with bootloader module when installing on UEFI machines
- installation on Bios machine with full disk encryption went fine. Reboot, unlock, the Grub starts, and that's it - something is still missing.
Testings and suggestions needed.

[maddes]

Test results with iso 4.6-dev20170202:

• Test on BIOS VM in ESXi 6.0, 50GB HDD, Hostname "sparky-test" (also used as LVM VG name = Debian standard)
• Use setxkbmap <language> (e.g. "de" for German) in a shell console to correctly enter passwords for existing encrypted setups.
• Open existing LUKS partition: cryptsetup luksOpen /dev/sdaX sdaX_crypt
• Activate existing LVM volumes of a volume group: lvm vgchange -a y <volume group>
• Bug: Cannot find a way to use an already existing LUKS+LVM setup:
  Accessing via console is possible, but the installer doesn't support to choose /dev/mapper devices for mounts.
  /dev/sda1 = 512M, ext2, /boot
  /dev/sda2 = 50G, LUKS, /dev/mapper/sda2_crypt
  /dev/mapper/sda2_crypt = LVM physical volume
  VG sparky-test = LVM volume group on /dev/mapper/sda2_crypt
  LV rootfs = 10G, ext4, /
  LV homefs = 10G, ext4, /home
  LV swap_1 = 10G, swap
• Installing an encrypted system via Erase Disk uses 2 encrypted drives for a) rootfs+bootfs and b) swap.
  /boot must not be on a separate unencrypted partition anymore as Debian testing includes latest GRUB 2 version. Except if you want to install dropbear to unlock the root partition remotely via SSH.
  Both encrypted partitions have two identical access keys: the password you entered and the generated binary file /crypto_keyfile.bin.
  Mapper name for encrypted partitions is prefix "luks-" plus the LUKS UUID of the partition.
  /etc/default/grub specifies additional parameters for GRUB_CMDLINE_LINUX_DEFAULT: cryptdevice=UUID=<luksUUID>:luks-<luksUUID> root=/dev/mapper/luks-<luksUUID> resume=UUID=<luksUUID of 2nd encrypted partition>.
  Bug: Encrypted setup doesn't boot, GRUB asks for the encryption password, but Linux and/or initramfs is not working:


• Wrong devices/UUIDs in several files. Normally use mapper names in them (recommended, see /etc/crypttab).
• for chrooting open encrypted partitions under the correct mapper name (also listed in rootfs /etc/crypttab) via:
  ~ MYDEV='/dev/sdaX' ; cryptsetup luksOpen ${MYDEV} luks-$(cryptsetup luksUUID ${MYDEV})
• mount partitions and all necessary stuff to /target (see also first post)
• chroot /target
• /etc/fstab
  ~ sed -i -e 's#^[[:space:]]*UUID=#/dev/mapper/luks-#' /etc/fstab
  (UUID way) ~ MYDEV='/dev/sdaY' ; MYLUKSUUID="$(cryptsetup luksUUID ${MYDEV})" ; MYFSUUID="$(blkid -o value -s UUID /dev/mapper/luks-${MYLUKSUUID})" ; sed -i -e "s#^[[:space:]]*UUID=${MYLUKSUUID}#UUID=${MYFSUUID}#" /etc/fstab
  
• /etc/initramfs-tools/conf.d/resume
  ~ MYDEV='/dev/sdaY' ; MYLUKSUUID="$(cryptsetup luksUUID ${MYDEV})" ; echo "RESUME=/dev/mapper/luks-${MYLUKSUUID}" >/etc/initramfs-tools/conf.d/resume
  (UUID way) ~ MYDEV='/dev/sdaY' ; MYLUKSUUID="$(cryptsetup luksUUID ${MYDEV})" ; MYFSUUID="$(blkid -o value -s UUID /dev/mapper/luks-${MYLUKSUUID})" ; echo "RESUME=UUID=${MYFSUUID}" >/etc/initramfs-tools/conf.d/resume
  
• /etc/default/grub
  You could also reduce GRUB_CMDLINE_LINUX_DEFAULT to just "quiet" as GRUB will find/set all the necessary stuff.
  ~ sed -i -e '/^[[:space:]]*GRUB_CMDLINE_LINUX_DEFAULT/ s#resume=UUID=#resume=/dev/mapper/luks-#' /etc/default/grub
  ~ MYDEV='/dev/sdaY' ; MYLUKSUUID="$(cryptsetup luksUUID ${MYDEV})" ; echo "cryptdevice=UUID=${MYLUKSUUID}:luks-${MYLUKSUUID} resume=/dev/mapper/luks-${MYLUKSUUID}"
  (UUID way) ~ MYDEV='/dev/sdaY' ; MYLUKSUUID="$(cryptsetup luksUUID ${MYDEV})" ; MYFSUUID="$(blkid -o value -s UUID /dev/mapper/luks-${MYLUKSUUID})" ; sed -i -e "/^[[:space:]]*GRUB_CMDLINE_LINUX_DEFAULT/ s#resume=UUID=${MYLUKSUUID}#resume=UUID=${MYFSUUID}#" /etc/default/grub
  ~ MYDEV='/dev/sdaY' ; MYLUKSUUID="$(cryptsetup luksUUID ${MYDEV})" ; MYFSUUID="$(blkid -o value -s UUID /dev/mapper/luks-${MYLUKSUUID})" ; echo "cryptdevice=UUID=${MYLUKSUUID}:luks-${MYLUKSUUID} resume=UUID=${MYFSUUID}"
  MYDEV='/dev/sdaX' ; MYLUKSUUID="$(cryptsetup luksUUID ${MYDEV})" ; MYFSUUID="$(blkid -o value -s UUID /dev/mapper/luks-${MYLUKSUUID})" ; sed -i -e "/^[[:space:]]*GRUB_CMDLINE_LINUX_DEFAULT/ s#root=/dev/mapper/luks-${MYLUKSUUID}#root=UUID=${MYFSUUID}#" /etc/default/grub
  
• swapoff -a ; swapon -s ; # disable swap
• swapon -a ; swapon -s ; # re-enable swap
• update-initramfs -u -k all
• update-grub
• exit from chroot and reboot
• Bug: Hibernate is not working. I only got this ever working on a LUKS+LVM setup with a separate /boot partition. Maybe the separate /boot partition is still or no more necessary.
• Bug: Defined user wasn't shown in user list of login screen
  Automatic logon was disabled during setup.
• Installing an encrypted system via Manual Partioning with separate /boot partition.
  /dev/sda1 = 512M, ext2, /boot
  /dev/sda2 = 20G, LUKS, /
  /dev/sda2 = 10G, LUKS, swap
  To define the LUKS partitions you have to choose the filesystem format, e.g. ext4 and linuxswap, and tick the checkbox for encrypt.
  Bug: Doesn't boot with a separate boot partition, no matter if the boot partition is encrypted or unencrypted. Tried to fix it afterwards but to no avail, maybe later when I know more about GRUB2/InitRAMFS handling in Debian testing (9 "Stretch")
  Bug: Re-using existing LUKS partitions looks impossible, you can say LUKS but not that it has to be formatted/used as ext4 under a given mount point or used as swap
  
• Installing an unencrypted system via Manual Partioning with separate /boot partition.
  Worked.
  

Much better experience than with the last development Calamares release from December.