Part-1 How To Obtain Success Using Sparky-Backup-Sys Tool (Forked Remastersys)

[sasdthoh]

Well,

It seems that work has subsided enough for me to keep a promise. Because I can be long winded (I think it necessary) when explaining how and why I perform particular tasks, I have decided to write several successive posts on this subject because I think it will be valuable, both to new Sparky Linux users and to experienced Linux veterans.

Taking the time to explain hings always makes sense to people learning but the Linux pros have a tendency to overlook the obvious because they work so fast. So, if anything I discuss interests developers, moderators, and hardened Linux users, then by all means check it out. At least it might be entertaining.

Before starting, I want to publicly declare my praise and appreciation by thanking pavroo and any other involved parties for their fine work in taking the old Remastersys project and reworking it using YAD to make it a valuable asset and useful timesaver for Sparky Linux users. I also declare that at no time do I take credit for any of their work. I simply spent many hours (and I do many many) attempting to use this tool as a means to help me save time in my charitable work. A long and drawn out number of posts have been previously provided explaining what was happening with me. Check out this area of the forum for those earlier posts. And now, let's get started.

One more thought before starting. There are several forks of Remastersys by other parties as well as similar tools that can be used with success (under limiting circumstances). I have used them as well as the complicated and time consuming live tools in Debian. I will save the uses and reasons of concern for those tools in another series of posts. After I finish outlining Sparky-Linux-Sys. Remember, I take no responsibility for mistakes, warranties, etc. Sparky Linux provides the proper disclaimers. Always make backups of any important files before starting.

The first question is as follows. Why do I want to use a tool like Sparky-Backup-Sys? Well, I rework older pc systems and donate them to anyone that needs them. Recipients are churches, schools, volunteer groups, rescue groups, public or private service organizations, and any individual(s) that may be short of funds to purchase new and more sophisticated hardware. In doing this work, I found myself spending a tremendous amount of time installing Sparky, Debian, etc., and configuring each machine. The age of the systems average between seven to twelve years old. I have even configured a few older than twelve years old but not many.

Based on those facts, the need to create a standard setup and install would save me a huge amount of time and time is the most valuable asset we all have. Additionally, my work demands that I work in a virtual environment to create a desktop configuration that is truly tailored for older pc systems, laptops, and notebooks. Once finalized and created, I use the tool to create a hybrid iso file, burn it to blank digital media or a usb thumb drive, then promptly boot, test, and install Sparky Linux / Debian with a minimal touch up at the end.

ACTUALLY, in the beginning, "failure" kicked me in the teeth almost every time. I was so mad because of so much lost time that I decided to experiment and test continually until I discovered what was causing my problems.

I discovered that there were two main causes of my failures. (and I bet everyone else as well) One was out of my control and caused by a newer version of Busybox that apparently caused grief for many other people just as it had caused for me. The good news is that Busybox no longer presents a problem, at least not for me in twenty four installations.

The other was completely caused by a simple issue that should have been obvious to me but I overlooked it many times. As I spent time configuring Sparky Linux to simplify its interface and reduce the software not needed for my purposes, I was constantly using root privileges to delete, remove, alter files and system settings as well as add themes, icons, artwork, etc. In all of my haste, I kept forgetting that the root user (su) and normal system user accounts store deleted files, etc. in trash folders (directories) until permanently removed. While I always made certain that the regular user system baggage was cleaned and sanitized, I ignored the root user's discarded system mess because it is hidden from the regular user. Remember, system wide directories, utilities, configurations, deleted files need admin access to perform system wide house keeping. The discarded system junk was piled high and deep.

Since hybrid iso files are read only squashed file systems compressed into a single file to be executed in memory, they are governed and restricted by the iso9660 IEEE specification and Sparky-Backup-Sys which do not allow a final iso size to exceed 3.99 gigabytes because of system overhead.

I found after testing over and over that exceeding 3.5 gigabytes sometimes caused problems. What's the root cause of this fact? Invisible code execution? Who knows except to say that software can be peculiar in its behavior in real world conditions. So rather than experiencing more failures, I now limit the FINISHED file size accordingly:  3.5 gigabytes or smaller.

The simple rule is don't install and configure everything including the kitchen sink. Just install and configure the utilities, services, and applications needed for regular usage and installation. More can be added when and if the iso is used to install on hardware.

Continuing, because there exists some inconsistencies and incompatibilities between similar tools created and used by other parties, I recommend that you completely uninstall those tools. As an example for instruction only the following is what I use to perform the task.

sudo apt update
sudo apt purge theothertoolname

then the following

sudo apt-get autoremove
sudo apt-get autoclean

NOTE: Caution is in order when using autoremove because of unintended results with xorg and system wide connected components. An example would be anything related to X. Use the man pages for apt, apt-get options to read the commands I listed and their functions in housecleaning. Next, I installed the following simple utility available in the repos.

sudo apt install trash-cli

Once installed, use the following command to clear out the hidden trash leftovers from the root user's account.

sudo trash-empty

We need to keep in mind that the Sparky-Backup-Sys tool will include system directories and their contents. If the root user's trash and discarded junk isn't removed permanently, it will cause the the hybrid iso file to exceed its maximum allowed size. Although the regular user's home directories and their files are not included in the iso build, the following will clear out and remove all related trash as well.

trash-empty (sudo is not needed) for the regular user.

(part-2 will be continued on the next forum post and continue where I left you)

[pavroo]

As an addition, I suggest to replace:

Code Select Expand

sudo apt-get autoclean
with:

Code Select Expand

sudo apt-get clean

The 'autoclean' removes older version of deb packages if exist, the latest ones won't be removed.
The 'clean' removes all downloaded by APT deb packages from the system.

[sasdthoh]

If you read my post to the end, you will see where I include autoremove and autoclean in the house keeping duties.

Thanks for your diligence.

Paul ( sasdthoh )

[pavroo]

Yes, I did, but I only suggest to change the apt command of "autoclean" to "clean".

[sasdthoh]

Ok, I have no problem with using "clean" instead of "autoclean." Can you enlighten me as to the reason you make that suggestion, based upon the man page options and explanations?

Just curious.

Thanks!

Paul

[pavroo]

I already did:

The 'autoclean' removes older version of deb packages if exist, the latest ones won't be removed.
The 'clean' removes all downloaded by APT deb packages from the system.

Clean removes all debs making the target iso smaller.
Autoclean always keeps the latest versions of debs of installed packages, you do not need in the target iso.

[sasdthoh]

I'm sorry for overlooking a clearly stated explanation that already answered my question. I am seriously tired and it was a classic "DUH" situation on my part.

I wasn't questioning your wisdom, just asking to clarify.

In truth, you are correct to remove ALL deb files to make the smallest possible iso.

Can I assume I was basically spot on about the rest?

Thank you again!

Paul ( sasdthoh )

[pavroo]

Of course, feel free to do so.

[sasdthoh]

Part-3 How To Obtain Success Using Sparky-Backup-Sys Tool (Forked Remastersys)

Continuing from my last post, if you have completed the steps and considered the suggestions I detailed in my muti-part posting about this tool, you should now have a valid hybrid iso file, ready to burn to a usb thumb drive or a digital media blank disc.

The first thing I do is to make certain the usb thumb drive is not defective so I use the USB disk Formatter Tool available in Sparky Linux to format the USB thumb drive. If it is not installed in your system, use the Sparky-Aptus-Extra tool to install it as well as the Live USB Creator.

As always, you can use the lsblk command in the terminal to identify the usb thumb drive.

After a successful format, use the useful Live USB Creator to write your newly created iso to the thumb drive. For you terminal lovers, use the following dd command to write the same iso file.

sudo dd if=/home/filepathtoyourisofile/yourisoname.iso of=/dev/sdb bs=4096

This assumes your iso is located in the default location where the Sparky-Backup-Sys tool created the file.
Of course, the location of file will determine the path in the statement. For the purpose of an example, I used sdb as the system identifier of the usb thumb drive. Sparky Linux makes this process very easy but all that you need can be accomplished using the terminal.

In the Windows Desktop world I use either Rufus or Universal USB Installer to burn the iso to the thumb drive. The tools and methods I have described have never failed me as long as the iso is valid and the usb thumb drive is not defective.

Once the processed has finished, you can test the usb in any pc or laptop that supports booting from external devices like the thumb drive.

I am interested in your success so let me know whether or not you have problems following the posted instructions. I created another one earlier today using the exact same steps and it worked without issues.

I have more posts covering saving user configurations and my experimenting with using persistence. I will cover those topics when enough time permits.

Best Wishes and Good Luck!

Paul ( sasdthoh )

[sasdthoh]

Part-2 How To Obtain Success Using Sparky-Backup-Sys Tool (Forked Remastersys)

Well, I wrote a very lengthy post for part-2 yesterday detailing the steps I take to prep and create the hybrid iso file. For some reason, I do not see it listed here. I can't explain it so I will take some time to write another post hoping that I will not have to repeat the process.

After explaining the procedures and issues in part-1, I take the following steps.

I use the graphical front end from the menus for Sparky-Backup-Sys to edit the Backup config file and the Installer config file.

In the Backup config file, the WORKDIR label determines the name and location of the completed hybrid iso file. While learning, I suggest to leave it alone. It will create the following folder (directory) by default where all the of temp work files do their magic.

/home/sparkybackup-sys

I leave the LIVEUSER label the same. It serves my purposes because when I create the final iso file, it will use the username "live" and the password "live" until installing to actual hardware.

LIVECDLABEL is what appears at the top of the iso boot manager's initial display.  I change it to something simple nut it's not required.

The CUSTOMISO label entry is where you actually name the iso file created. I always chnage this to something short and descriptive. Remember to keep it within quotes. In fact, make certain that the quotes never get accidently removed when changing labels.

Save and Exit.

Open the Installer config file. There is really only one label to contend with, the NewGrubDefLine. I remove the entry and leave just the quotes with no space between them but it's up to the user and not required. I have tried making changes to this option and the results are obvious. This one will be up to the user but you can ignore this if you choose. If you make any change on this option, don't forget to save before exiting.

Lastly, the Clean_Cache option removes all deb files in the system to reduce the final iso file size. I use the command line option to perform this task and will shot it momentarily.

DO NOT USE THE GRAPHICAL GUI TO CREATE THE ISO. For some reason, I have had mixed problems using the gui to create the iso. I have no idea why it doesn't consistently work for me so I now ONLY use the command line commands. Exit the gui and open a terminal.

Remember to use the house keeping commands and tools to make certain you system has removed all unnecessary junk and system leftovers.

A WORD ABOUT CLEAN AND AUTOCLEAN. I have always used AUTOCLEAN as a system house keeping command but pavroo is absolutely correct by stating the use of CLEAN instead. It removes all deb files while autoclean only removes deb files that can no longer be used by any package leaving current deb files remaining.

There is a case for using autoclean on installed Sparky Linux systems but creating the hybrid iso requires the smallest file size possible. It gives the user some leeway in adding other software to their build because more space is available assuming the user wishes to do so. Use the man pages to read a detailed explanation of the differences. It's quite educational.

OK, now is the time to create the iso.

Open the terminal, make certain to review you use of the house keeping commands, then issue the following command.

sudo sparkybackup clean (only use this command to remove any previous builds)

sudo sparkybackup dist

This will start the process and takes varying amounts of time, depending on hardware and system resources. Be patient. Also, do not use the system for other tasks while building the iso file. Get yourself something to eat, a favorite drink, read a book, whatever helps to pass the time until the process has completed.

Once the process has completed, use your favorite file manager and check the hybrid iso's properties to make certain the file size appears to be normal. My builds usually range between 800-megabytes and 1.1 gigabytes which depends on the system software and configuration choices. Yours could be slightly smaller or larger based on personal choices.

If all appears to be normal, read my post part-3 for instructions to burn this newly created iso file to blank digital media or a usb thumb drive.

Thanks!

Paul ( sasdthoh )

[armakolas]

Question #1: Having in mind that in Sparky Backup System's guide Part-1 it is written that "NOTE: Caution is in order when using autoremove because of unintended results with xorg and system wide connected components." then why does the Clean_Cache command of Sparky Backup Sys uses the "sudo apt autoremove -Y" command?

Question #2: Isn't it even more dangerous and destructive once it uses the "-Y" option?

Question #3: Since the title of Sparky Backup System's command is "Clean_Cache", wouldn't it be more appropriate to either change the name to something like "Remove all orphaned" or to attach a command that really cleans the cache, like "sudo apt clean"?

Question #4: In addition, since in the last post it is stated that "Lastly, the Clean_Cache option removes all deb files in the system to reduce the final iso file size. I use the command line option to perform this task...", isn't this a clear sign that we all do the same mistake when we trust the Clean_Cache option of the gui? Especially when the exact command that is used by the user sasdthoh is not clearly stated. I guess that it is the "sudo apt clean".

Question #5: Furthermore, in the next sentence, we are advised not to use the gui to create the iso. Could the "Clean_Cache" problem be the reason for gui's misfunction?

May I guess that the Clean_Cache command, which uses the "sudo apt autoremove -Y" command, has destroyed some installations of some unlucky Sparky friends? In my installation, it deleted "grub-pc-bin" and "grub2-common", but I was lucky enough to boot again without having to repair anything. I later executed "sudo apt install grub2
" so as to re-install the missing files.

[pavroo]

That's good point.
Removed the "-y" option from autoremove so it will ask for your permission now.

Anyway it has to be used to remove all orphaned packages.

[armakolas]

I am happy that I helped. I think that nobody would deny the need to use it and it's effectiveness in removing orphaned packages. Though, the name Clean_Cache is misleading. Please consider changing the name in the gui and it's description.

[armakolas]

I have noticed that the command "sudo sparkybackup clean" does not delete the same files as Sparky Backup System does when we double click on Clear.
The command "sudo sparkybackup clean" only deletes the folder /home/sparkybackup-sys/sparkybackup-sys, which hosts the previous builds. It does not delete the folder /home/sparkybackup-sys, nor the files inside /etc/skel/.
On the other hand, running Sparky Backup System and double clicking on Clear, deletes the folder /home/sparkybackup-sys, not only the sparkybackup-sys folder inside it, and the files that exist in /etc/skel folder. The gui option is more thorough than the cli one.
Shouldn't these commands be totaly equivalent?

[pavroo]

Made all the changes and improvenments you suggested:
https://sparkylinux.org/sparky-backup-sys-20180607/
Test and report whatever you find, please.