Success! How To Create Persistence To Work With Your Hybrid ISO File

[sasdthoh]

I have successfully created persistence to work with our Hybrid ISO files. Three different tests were performed making various changes to my live hybrid iso's and it works without any problems or errors. I would like to perform a number of additional tests to make certain there is 100% satisfaction for anyone wishing to use persistence in their usb thumb drives.

After a short break and after my tests have been completed, I will post here as a reply to this message to show how it is done. There are a few steps to be taken but is very easy and everyone will understand my guide.

This method should work with default Sparky Linux iso's as well but I haven't had enough time to test with Sparky downloads. Logically, it should work very well.

Stay Tuned For More Fun!

Paul ( sasdthoh )

[sasdthoh]

As promised, this is the process I use to create "persistence", or rather the ability to save changes while using your live hybrid iso created using the Sparky-Backup-Sys tool when written to usb thumb (pen) drives and / or similar devices. Before we start, I have a few thoughts I would like to express concerning this subject that I feel is important. The usual disclaimers apply. You and you alone are responsible for the backup and integrity of your important data. I offer no guaranty or warranty of any kind. This is for instruction purposes only and Sparky Linux has no responsibility for my posts.

The number one issue that I found to be an obstacle when learning to create / use persistence is misinformation, poorly written documentation, and completely / partially false documented procedures and instruction concerning this subject. Searching this subject over the Internet will yield so many versions of this subject that confusion becomes the "order for the day." Simply stated, you have no idea what documented procedures and information is correct or incorrect. Basically, what you find will generally SUCKS! That's right, I said it!

TRIAL and ERROR, over and over again will eventually lead you to "persistence nirvana" but not before frustration and disgust takes its toll. I made a promise in my previous posts that I would work on the "TRINITY" of squashfs systems until was happy with the results. I believe I have finally breached the last obstacle, persistence.

The following is how I am able to implement persistence while using the hybrid iso files  created and written to a usb thumb (pen) drives or similar devices. I have yet to perform this method on other distros, Sparky default iso's, or even Debian iso's. When time affords me the opportunity, I will report my findings concerning those areas of interest. Now let's get to the "meat and potatoes."

!) Why are hybrid iso's such a problem as it relates to persistence? It's because they are compressed read only single file versions of an operating system that are executed in ram and cannot be altered on the usb device itself. The only changes possible are some that take place in memory during use. If someone opens a browser and finds an interesting web site, they bookmark that site for later access and it works fine until such time that you wish to shut down the pc or laptop hosting the live OS. Because the live OS is a read only file system which occupies the entire physical area of the disk (partition size), the changes cannot be saved and / or physically written to the usb device for later access / retrieval. In my work billet, this is not an issue because of how I use a live OS on usb devices but many users would like to carry a miniature version of their OS configuration with them for the purpose of mobility.

In order to achieve our goal of a working persistence configuration we have to achieve the following.

1) Change the actual existing partition by reducing its size on the usb device in order to create the space necessary to save changes. Then create a new partition with the unused space for the purpose of storing data and / or changes

2) Use a terminal to issues some very simple commands so that Linux will know of the existence of the newly created space and how to use it.

3) Pass boot parameters to the kernel when starting the live OS so it knows to save changes stored in memory and do it automatically.

This is how you to accomplish the entire process.

First, successfully write and test a hybrid iso to a usb thumb (pen) drive or device to confirm it works correctly. Then use a tool in you existing hardware installation of Linux, virtual machine installation of Linux, or a similar tool in Windows to re-size the partition to make room on the thumb drive. Gparted works as advertised. I advise that you use the terminal and issue the   lsblk   command and identify the device as listed. In my instruction here, I will pretend the device id listed as

/dev/sbd (physical device)         (example for instruction purposes)
        /dev/sdb1 (actual partition)

You must substitute any identifier that actually lists your usb device.

Open gparted with admin privileges (install if needed) and select the usb device from the drop down list. Then RIGHT-CLICK on the listed partition and UNMOUNT it. Then select it once more and resize the partition to a size larger than the hybrid iso file that was originally written to the device. I think you should reduce it by half of more the entire physical size of the usb device. Remember, the more storage space available for persistence, the more changes you will eventually be able to save. Then click apply to perform the partition re-size procedure.  Be patient.  It only takes a few moments time to complete. You will then be presented with a refreshed view of the re-sized partition and newly created unused space.

Next, slect the unused space and RIGHT-CLICK on it, select new, then create a partition using the entire size of newly created space with the ext4 file system and a partition label

persistence    IMPORTANT: you MUST use the label spelled exactly as presented lower case

Save and select apply once more for the newly created changes to take effect. If you are using Windows for this part of the procedures, perform an Internet search for a tool called
Mini Partition Wizard Tool Free version. Download and install it. It will essentially perform the same tasks as Gparted does in Linux. BE CAREFUL NOT TO BLOW OUT YOUR WINDOWS PARTITIONS! Only work with the usb device in question. DOUBLE CHECK AND TRIPLE CHECK before making changes. You are responsible for any mistakes, not Sparky Linux or me.

Once the changes have been made to partitioning, close Gparted (or other used tools) and enter a Linux terminal.

DO NOT use the live usb for this part of the proceures. Use the regular installed Linux terminal, either in a virtual machine or on real hardware.

Open the terminal, switch to su, enter your password, then you will be presented with a # prompt. Issue the following simple commands, remembering the lsblk command to identitfy the newly created partition.

NOTE: sometimes you have to unplug and plug in again the usb device for Linux to recognize the changes. Also remember to use the identified device assignments in your situation.

#   lsblk

#   /dev/sdb                                (examples for instruction)

             /dev/sdb1 (original partition)
             /dev/sdb2 (newly created partition with the label: persistence)

Issue the following four commands.

#   mkdir -p /mnt/usb

#    mount /dev/sdb2 /mnt/usb

#    echo "/ union" >/mnt/usb/persistence.conf     (characters and spaces must be exact)

#    umount /dev/sdb2 /mnt/usb

You will be presented with a message that /mnt/usb is not mounted. Note the name of the persistence.conf configuration file. It matches the partition label exactly. All is well if you followed my instructions. Close the terminal. It is now time to test persistence in the real world.

Reboot your actual computer / laptop and select your usb device from the boot menu. You will be presented with a Sparky Linux Grub boot menu with several options. Make certain the top option is selected and press the   ESC   key.

You will be presented with the following dark screen prompt. Type two words at the prompt as follows.

BOOT:   live persistence

Press ENTER to begin the boot process. NOTE: The first time you boot with this option, the speed of the boot time is reduced. Be patient. System overhead is being performed. It will boot much faster on subsequent boots. When you are finally presented with your desktop, makes some changes such as a recognizable wallpaper, create a folder on your destop with any recognizable label, open a browser and add some bookmarks or any change that you would recognize when testing to see if changes are being saved.

NOTE: You may notice that your system seems to be sluggish during the initial use on this first time. That is because it is busy with system housekeeping to the persistence.conf file, etc. It sluggishness will pass and it will work much faster in subsequent boots.

Once you have made some obvious changes and the system has finished with its apparent slow behavior, reboot your system and perform the very same procedures as before from the boot parameters screen.

BOOT:   live persistence

Once the boot process has finished, check to see if your changes have been saved. If yes then CONGRATULATIONS! All you have to do from this time forward is to issue those two words if you want your changes to be saved and recognized on subsequest boots.

If no success was achieved, then something was not done correctly or some other issue has raised its "ugly head.". Retrace your steps and try again. The process is worth repeating, if for no other reason, just for learning.

I have repeated this exact process five or six different times and with multiple usb thumb (pen) drives and never failed to create a working persistence.

PLEASE ......... I have dedicated a significant amount of personal time on this issue. I would like to hear from the Sparky Linux community on my procedures and whether or not it worked for you personally. If I am able to help just a couple of people, then I will be pleased that my work was not in vain.

Good Luck! (and PLEASE let us know of your success or failures)

Paul ( sasdthoh )

[sasdthoh]

GOOD NEWS FOLKS!

I just finished two separate tests creating persistence using the default Sparky Linux MinimalCLI iso (non-gui, no codecs) downloaded from the Sparky Linux repos.

My procedures listed in the previous post worked without errors or issues. I will continue testing each type of Sparky Linux iso to confirm success with persistence creation using my procedures previously listed.

Stay Tuned!

Paul ( sasdthoh )

[sasdthoh]

I notice that my posts covering persistence, saving user settings, and successful creation of hybrid iso files are being read by a large number of people.

Yet, nobody has written any messages stating their success or failure by following my instructions. I have helped several people over the telephone with success but I would really like to hear from anyone that has successfully created persistence following my instructions.

If anyone has problems, I'll be happy to clarify or assist you to achieve persistence success.

Paul ( sasdthoh )

[LoganFM]

Hi @sasdthoh,

I need some help here.  I'm using 4.5 GameOver edition as live on my 8Gb Sandisk Flash Drive. I've installed it using RUFUS in DD mode.

The problem within the process shows up after I open GParted. After choosing sdb on the drop down it shows me a /dev/sdb2 fat16 with 2.81 MiB and another unallocated partition with 7.45 GiB. I know it isnt possible to have 99% of this drive unallocated since the ISO consumes more than half of its size, but so, how would I proceed to separate the unused space?


Thanks in advance and kind regards!

Logan.

[sasdthoh]

Hello LoganFM and thank you for posting, even with issues to resolve. I believe I can help you. But first, let's start from the very beginning so that I can control the entire process from the beginning to the end, OK?

Are you ONLY working from a Linux installed system on real hardware, a virtual machine running Linux, or are you using a Windows setup along with Linux? I need to know specifically. I do not want to use the live os for the setup work. The use of Rufus makes me believe that you are using a Windows environment and also working with Linux.

If you are indeed in Windows using Rufus, I want you to download two different tools for learning purposes. Once we finally have success, you will have the freedom to use other tools as well, ok? The following are the two tools I use when in a Windows install.

1) Mini Tool Partition Wizard Free --->

                                                 http://download3.minitool.com/pw10/pw10-free.exe
2) Universal USB Installer

https://www.pendrivelinux.com/downloads/Universal-USB-Installer/Universal-USB-Installer-1.9.7.7.exe

Once you have the two utility programs downloaded, send me a private message here then I will post openly here on the forum the entire process for you, step by step. Make certain to tell me the OS you are working in while preparing the USB thumb drive, ok?

Don't worry, I'll get you there.

If you feel comfortable, send me a private message here on the forum to my username,     sasdthoh   , and provide a regular email address so that we can communicate on a more personal basis.

I'll keep an eye for your response here.

Paul ( sasdthoh )

[sasdthoh]

This posting is more for the benefit of less experienced users. The seasoned Linux user / veteran is not likely to be confused by the lengthy explanations I provided on persistence. The following clarifications might serve to make the process easier.

The first issue you need to determine is whether you are working in a LINUX ONLY installation on real hardware (virtual machine etc.) or whether you are a user that uses Windows as their primary OS and want to use a Linux Live OS on a usb thumb drive with persistence. Most inexperienced Linux users that I have met still use Windows OS desktops and / or laptops and sometimes use Linux in mixed installations and hardware. For this specific clarification, I will assume the user uses Windows as their primary OS and also wants to use Linux wit persistence as a LIVE option.

For learning purposes, I recommend using two specific programs to assist you in a Windows environment. Once you learn the overall process, then you will be free to use other similar tools to accomplish the same tasks. The following two Windows applications are the ones I recommend and successfully prepare the usb thumb drive. Both programs are free of cost.

1) Universal USB Installer

https://www.pendrivelinux.com/downloads/Universal-USB-Installer/Universal-USB-Installer-1.9.7.7.exe

This utility program runs in memory without having to perform an installation.

2) Mini Tool Partition Wizard Free

http://download3.minitool.com/pw10/pw10-free.exe

This utility program performs the same tasks that Gparted does in Linux and its interface looks like other third party Windows Partition Managers. This programs MUST be installed in Windows before actually using it. Just accept the defaults during installation.

Once you have obtained both programs, make certain you have a usb thumb drive (device) with the minimum capacity of eight gigabytes. Anything smaller will not allow enough free space to use with persistence in a practical way.

Before we begin, I recommend that you disconnected, either by using Windows to "safely disconnect" any usb devices that are not being used for this tutorial. If necessary, just physically disconnect them until you have completed the procedures. This will serve to eliminate any confusion when identifying the thumb drive to be used and will also make it impossible to make unwanted changes to the disconnected usb devices.

Also remember that Sparky Linux and me are NOT responsible for any mistakes you make that cause problems. It's your responsibility alone when working with this tutorial.

I will use the desktop system available to me for this detailed explanation. It has a hard drive with Windows 7 installed and I have a working Sandisk usb thumb drive with the capacity of 8-gigabytes. No other usb storage device is attached to the computer used by me.

Open Windows File Manager (called Windows Explorer) and identify the drive letter assigned to the usb thumb drive. In my working system, it is identified as device E: Just make certain to stay away from device C: !

Next, launch the Mini Tool Partition Wizard. You will be presented with small graphical window. Select the upper left section to launch the utility program. The program will open with a display showing the drives in your system. Two main areas are provided. The upper section first lists the Windows partitons on Drive C and below will display the applicable partitions on the other attached storage devices. At the bottom you will see the description and partition of the usb thumb drive. Remember, my usb device is identified as Drive E:

Normally, this first procedure would not be necessary but it is a good idea to make certain there are no defects with the usb device so I am doing the following. Select the identified partition at the bottom. The display above will be highlighted with a yellow / orange color indicating which device and partition you have currently chosen to carry out procedures.

RIGHT-CLICK on the usb tumb drive's partition and select DELETE from the options menu. Once DELETE has been selected and you have identified that your selection is correct, the locate and select the APPLY toolbar icon.

WARNING! Make absolutely certain that you are working with / selecting the usb thumb drive partiton and NOT your Windows partitions.

Once you have deleted the existing partition on the usb thumb drive, it will now display and indicator of UNUSED SPACE where the partition was before. Highlight it once more, RIGHT-CLICK, select NEW and confirm your desire to continue. First, use the drop down box to select FAT32 as a file type. It defaults to the entire size of the usb thumb drive. Accept the remaining defaults and click the APPLY toolbar icon once more. The process is swift. Once completed, close the utility program. The UNPLUG then reconnect the usb thumb drive. A few moments later Windows will open a dialogue asking if you wish to view the contents. Double check the drive identification and close any windows that are open.

At this point, you have already created a hybrid iso or downloaded a distro iso file. If not, do so now. I advise you to select one that isn't too large, at least while learning. My instructions were originally for the purpose of adding persistence to the HYBRID ISO files created using Sparky-Backup-Sys tool or the Sparky-Backup-Core tool. I have tested this process and it works with Sparky default Distro ISO files.

Once downloading is complete, launch the Universal USB Installer, accept the disclaimer and open to the main screen. The top line is a drop down box. RESIST the temptation of selecting a specific name from the list. Scroll to the very bottom of the entire list. Locate and select the option   Try Unlisted Linux ISO. The next line allows you to navigate and select the downloaded iso file you wish to use. The bottom line provides another drop downlist to select the usb thumb drive in question Mine was identified as Drive E: There is no need to check the Format option. This was taken care of in the Mini Tool. Simply click CREATE and confirm the procedure. Now wait until the process has completed. Please note: once the screen has indicated the process has completed, wait a short time to make certain the entire contents of data has been written to the thumb drive. Close the application, unplug and reconnect the usb device once again. Reboot your actual hardware to test your usb to make certain it boots as a Live OS without errors.

Until now, this process seems to have take a lot of time but the reality is that it was just because you were reading my post and following my instructions. The actual time taken so far has been brief. Remember, if your usb doesn't boot without issues, then persistence will never work. If if booted without any problems, restart your system once more and return to you regular desktop.

Launch the Mini Tool once more. Select the usb thumb drive partition listed at the bottom. RIGHT-CLICK to open the options menu and select RE-SIZE. When the action screen appears, use the mouse to select the slider and drag it to the appropriate size desired. For learning purposes, change it to 4.00 gigabytes. Then confirm and select the APPLY toolbar icon to perform the required procedure. after finishing, you will notice the unused space at the bottom created by the re-sizing procedure. Select the unassigned space, RIGHT-CLICK to open the options menu, the select NEW. Next select EXT4 from the file type drop down listing and then type the following label name in lower case. Make certain you spell it correctly as listed.     persistence    Accept your selections and APPLY once more to carry out the procedure. This particular step is the longest in the entire process. Be patient. I will happen in short order. Pnce completed, you will be presented with your usb device with two partitions, the re-sized FAT32 partition and the newly created EXT4 partition with the label name of     persistence     Close the Mini Tool. The hard work is now complete. (and that wasn't REALLY HARD, was it?) The final preparatory procedure is to use the Linux terminal to issues four easy commands. Once completed, then all you have to do is test the results.

I know I told everyone in my previous instructions NOT to use the LIVE OS to perform this next step but I have tested and confirmed that it works fine. Reboot your system and select your usb device from your boot menu. When you are presented with the Sparky Linux Grub Boot Loader Menu, make certain the top option is highlighted and then press your ESC key. You will be presented with a black screen with the following prompt. Enter the two words in lower case exactly as provided.

BOOT: live persistence  (the words are separated by a space)  Then press your ENTER key to start the boot process. The boot process will seem sluggish. This will pass later. Once you are presented with a working desktop, keep in mind this is a LIVE OS with SUDO privileges. First disable the screen saver in the Preference menu. Then open a terminal and make it full screen to eliminate distractions. Enter the following at the prompt.

$ sudo passwd root   (press enter key)   Type an easy to remember password for the ROOT user and press ENTER to confirm your choice. Once you have a new password for the ROOT user, enter the following.

$ su   Press the ENTER key and type the newly created ROOT user password)

Now you are logged in as the super user and sudo is no longer needed. Type the following five commands, one at a time taking care to type them exactly as provided, space for space, character for chacter, lower case.

# lsblk   (this command will identify the usb device identifier, mine was /dev/sdb)

/dev/sdb
              /dev/sdb1   (partitions 1 & 2 created on the usb thumb drive)
              /dev/sdb2   persistence   (substitute your identifier if necessary)

# mkdir -p /mnt/usb

# mount /dev/sdb2 /mnt/usb   (mounting EXT4 persistence partition to /mnt/usb)

# echo "/ union" >/mnt/usb/persistence.conf   (persistence config file matches label name)

# umount /dev/sdb2 /mnt/usb   (umount does NOT have an N in the command)

If you receive a message stating that the partition is not mounted, all is well in persistence land!

Close and reboot your system once more making certain to follow the previous instructions to press the ESC key and type   live persistence   at the boot options screen, same as the previous boot. You will notice some sluggishness because of initial system overhead communicating with the persistence.conf config file. Make some changes, add wallpaper, create a folder on your desktop, maybe launch the browser and bookmark web sites, change the home page etc., anything where your changes are different from the initial LIVE boot. Once done, wait a few moments to allow the system to save your changes, then reboot once again and doing the same as before by entering   live persistence    The system will start to perform more robustly. Once you are presented with the desktop, check to see if your changes are still there. If yes then

CONGRATULATIONS! Now all you have to do from this point forward is to boot with the   live persistence   boot parameter and your changes will be saved as long as the changes do not exceed the partition size allocated. Remember that your are using a usb thumb drive, not a hard drive so its size is limited.

ENJOY Persistence Nirvana!

Paul ( sasdthoh )