Allow user data partition to be included for silentinstall

[meltwater2000]

Only a single OS/flavour can be selected for silentinstall, this includes the user data partition.
Can this be amended?

[lurch]

Any suggestions as to how you'd like to indicate that silentinstall should install both the OS and the data_partition? :)
Or should silentinstall be modified to always automatically install all non-bootable "distros" alongside the single bootable distro?

[meltwater2000]

Currently what happens if more than one OS is present?
(If I recall correctly nothing is installed).
Would there be an issue if it installed any which are present?
Thanks
TimC

Andrew Scheller notifications@github.com wrote:

Any suggestions as to how you'd like to indicate that silentinstall
should install both the OS and the data_partition? :)
Or should silentinstall be modified to always automatically install all
non-bootable "distros" alongside the single bootable distro?

---

Reply to this email directly or view it on GitHub:
#196 (comment)

[lurch]

IIRC if more than one OS is present, it simply 'aborts' the silentinstall and simply loads the NOOBS GUI as normal.

I'd be very hesitant for silentinstall to simply install all OSes present on the card, because it would likely lead to users not understanding what it's intended for, and trying to trigger silentinstall without actually deleting any OSes from the SD card, which is likely to them then running out of diskspace?

The primary goal is for NOOBS to be noob-friendly, not necessarily be an auto-installer for lazy advanced users ;-)

[meltwater2000]

Yes that is fair enough, want to keep the tool easy to use and as clear as possible.

Certainly the ability to install any non-OS partitions would be very helpful.
Would an additional command line option be a way to do it, --forceinstallall or something like that?

Or a command line option to specify which OS is installed during a silent install (if option not present on the commandline it installs as before if only one OS is present)? In the case of non-silent installs this command line could pre-select (tick) the specified items (allowing a preselected recommended flavour or setup without forcing the user).
eg.
--install Raspbians XBMC Raspbian_Scratch

[lurch]

Yeah, I guess the latter option may work, and would potentially even allow silentinstall to work with NOOBS lite :-)

This is using kernel-command-line syntax (see https://github.com/raspberrypi/noobs/blob/master/recovery/cmdline.txt ), rather than application-command-line syntax, so it could be something like install_os_list=raspbian,data_partition,openelec. Or perhaps install_flavour_list=Raspbian_-_Boot_to_Scratch,Data_Partition,OpenELEC given that each os subdirectory can contain multiple flavours.

Although maybe there also needs to be another cmdline.txt option which specifies whether to install the OS contained on the card, or the latest OS available from the internet...? (silentinstall_use_internet=1?)
Always a bit of a balancing act between simplicity & flexibility ;-)

[maxnet]

Certainly the ability to install any non-OS partitions would be very helpful.

Just open the partitions.json in the folder of the OS being silent installed and add to the list:

    ,{
      "label": "mypartition",
      "filesystem_type": "ext4",
      "partition_size_nominal": 512,
      "empty_fs": true
    }

empty_fs is an undocumented option that was used during testing to indicate a partition does not have a corresponding tarball with the contents of the partition.

[lurch]

Aaah, handy :)
I guess if you wanted to you could also modify partition_setup.sh to automatically add this new partition to /etc/fstab too - e.g. you could add something like this for Raspbian (between the sed and umount lines) - but beware that this is untested

mkdir /tmp/2/mnt/mypartition
echo "$part3 /mnt/mypartition ext4 defaults 0 0" >> /tmp/2/etc/fstab

[meltwater2000]

The flexibility selecting the distros has is excellent, and I think avoiding the need to take drastic action of deleting stuff off the card is a good plan (those who have brought pre-loaded cards won't be keen to remove things off it - I know it is easily replaced).

"Silent install latest from the internet"...now that IS fancy!
In fact, I think that might become a very popular way to do it once you've decided what kind of setup you want, avoiding the need to continuously check if the SD card is the latest, while being able to pop the card in and let it run (good for schools/workshops). Just need the satisfying "Wall-E charged" sound when it is cooked!

maxnet - empty_fs, that is very useful and also solves the original question for me. I'd not looked too closely at that file, and that is perfect for what I wanted.

[lurch]

Hmmm, the other advantage of silentinstall is that it auto-boots that OS when the installation is finished - so I guess if we allowed silentinstall to install multiple OSes, we'd also need another option to tell it which of those OSes to boot once it'd finished installing everything? :-S

Maybe it'd be better to take a different tack and ask why you think editing recovery.cmdline to add half a dozen new options (before booting NOOBS for the first time), would be any easier than just letting NOOBS load as normal and using the GUI to select which OSes you want to install? ;-)
(i.e. what's the use-case that the current system doesn't solve?)

[meltwater2000]

That is a good point - these are my use-cases, hopefully you can see my reasons (hopefully they make good sense). I am happy to accept NO, since I fully understand NOOBS is there to keep it simple not make it more complicated. Just focusing on ways to improve NOOBS and make it easier to manage setups.

USE-CASE 1 Flavours:
The route to this was being able to setup a magic NOOBS install which is configured for whatever purpose needed (workshop setup, automated demo etc etc). That is covered by noobs-config very nicely, the part of the puzzle here is the automated install part which would allow a card to be preconfigured and copied to an SDCard, inserted and powered on (without the need to setup the display or interact with it).
So far I've seen schools having a Master SDCard which they clone for everything, which is fine, except you run into issues with needing different versions for different things, with different sized SDCards, when upgrading and if it fails or needs to change, managing multiple 8Gb images etc etc. There are lots of issues related to this which can be avoided.
The way I have it working currently, is I can setup a different flavour for each particular setup I want, right down to:

• A flavour which auto-runs a traffic light sequence on the pins when it boots
• A pre-configured setup with network shares, user files and wifi configuration
• A setup specifically configured for a workshop

Now you might say this is an advanced use, but this would allow the configuration to be set-up once and shared as a NOOBS setup. As always the hard part is creating the setup, but that would need to be done anyway, the difference is you do it once, rather than each time you do a new install, also you can include/exclude it if you want by selecting the flavour you need. So you end up with one setup which you can manage a lot easier (it would even allow you to manage the setup via Github if you wanted).

The "advanced" part for a typical user is copying it to an SDCard and switching the RPi on, but it allows the complex parts to be done beforehand...as NOOBS should be!

USE-CASE 2 Dual Boot:
It is not uncommon to have a RPi dedicated to a specific task, be it a media center or otherwise. It would make sense for these to also be able to have a Raspbian install on (or preferred OS) so that it can be also used for other purposes. My particular case I am thinking of is the RPi which is in a kids bedroom serving video via XBMC, it would make sense to have Raspbian available to switch to by the kid without messing around with the RPi itself. However, you can imagine if that card gets messed up, you want it up and running quickly (typically the time you have available to redo things is when the kids want to use it), so an automated NOOBS install that allows multiple OS selection would be very helpful. Auto-Boot selection, can simply be based on first OS listed.
[A side thing here would be boot selection via GPIO...but that is another project I've not looked at yet]

USE-CASE 3 Data-Partition:
A side use-case, is the data-partition which is a good way to help manage user files on an SD-Card, it provides a clear place to put files which you know you want to keep/backup etc etc. Also as a common drive between distros (if you have more than one). I like this idea, I use it all the time and it should be encouraged as a general practice (linux is too easy to spread your user data all over the place and it can quickly start to cause issues for beginners not used to it - particularly if they start fighting user permissions etc).

[To play advocate here (not suggesting this should be done)...once NOOBS has been put on a card, there should be no reason for a "NORMAL" user to touch it (and that appears to be generally preferred thing so we "could" move it).
Q: How feasible would it be to have the user data partition, as FAT32 and the 1st partition on the card (hiding the Recovery partition away)? I assume it would still need at least one file in the 1st partition to boot initially, but it would make the user data FAR easier to backup and manage (since it would be viewable on any computer including windows). I always prize the user-data as the most important information since everything else can be replaced, so making it easy to backup and keep is paramount. Also by allowing scripts to be edited on any computer means you don't need a Pi setup for every single child, they can edit their code and try it out when ready (even if they end up fine tuning it directly on the Pi after). With the right setup, the Pi could be used headless and used as more of an embedded controller too, without the hassle of switching cables etc.]

[procount]

I'm not sure it's possible to put a "pure" data partition in Partition 1 because you need the kernel files there to boot from at least, and moving the recovery images to another "hidden" partition would add to the complexity and make customisations of the images (and noobs-config) harder to manage.

However an alternative might be to have an option for NOOBS to reserve a specified amount of space on the recovery partition for a /USR folder before it resizes the partitions. This could be mounted (-o bind) by a distro later. This makes a data folder partition available on the FAT32 recovery partition to be pre-written onto the card easily from Windows beforehand, plus maintains the customisation of the images using noobs-config or by adding/replacing other distro images or flavours as required by advanced users (for the use of noob-users).

[meltwater2000]

Probably best keep the 1st partition User partition as a side comment, as it is outside the original goal of the silent install OS options. I like the idea of the user folder (linux is so flexible!), and that is something I can experiment with without polluting NOOBS with these dark ideas [yet]. Is there a way to specify how what size the recovery partition ends up as, or is that hardcoded (size of files + xMb)?

[lurch]

First off the bat: writing to Partition 1 is never going to happen, because of the NOOBS prime directive https://github.com/raspberrypi/noobs/wiki/NOOBS-partitioning-explained#noobs-partitioning

USE-CASE 1 Flavours:

Sounds like you basically want to be able to silentinstall a particular flavour (i.e. just one), whilst still leaving all other OSes/flavours available on the card for later (re)installation? Sounds like a reasonable request.
(and this could possibly also cover the NOOBS-Lite case - perhaps the logic could be that it'll only download from the internet if the OS isn't already available on the SD card.)

USE-CASE 2 Dual Boot:
if that card gets messed up, you want it up and running quickly

I still don't see how pre-customising recovery.cmdline is any quicker than just using the regular NOOBS installation GUI in this scenario. It's not as if the NOOBS GUI is particularly complicated...

[A side thing here would be boot selection via GPIO...but that is another project I've not looked at yet]

Yeah that's entirely possible (http://www.raspberrypi.org/forums/viewtopic.php?f=91&t=71107&p=568213#p568213 may give you a starting point) - but if this is a feature you really want please open a separate issue for it.

USE-CASE 3 Data-Partition:

@maxnet has already showed one easy way to add this, but note that @maxnet 's method creates the data-partition as being logically 'owned' by Raspbian, rather than logically being a standalone partition (not that it makes much difference in practical terms!).
And don't forget that every time you install/re-install an OS, the Data-Partition gets wiped too.

One thing I've been considering (but not in great depth yet) is some way to add the Data-Partition (if selected) to the partition_setup.sh call, so any OSes installed at the same time could automatically mount it (via /etc/fstab) if available, rather than leaving this as a manual task for the user.

linux is too easy to spread your user data all over the place and it can quickly start to cause issues for beginners not used to it

Hmm? There shouldn't be any reason for them to store files outside of their /home/pi user folder? Unless they get too carried away with using sudo to store files in places they're not supposed to...?

How feasible would it be to have the user data partition, as FAT32 and the 1st partition on the card

Never gonna be the first partition on the card, but you're welcome to switch it (for your local setups) to FAT32 if you want.
I wonder if using the Windows "Disk Management" GUI allows you to (manually) mount FAT partitions on an SD card beyond the first one?

In summary: I'm happy to consider options for silent-installing a single flavour/OS out of all those available, and I'm happy to also consider an option for installing non-bootable OSes (i.e. Data Partition) at the same time, but IMHO installing multiple bootable OSes is beyond the scope of silentinstall, and is a level of complexity that should be left to the normal NOOBS GUI.

[procount]

Agreed.
x=100MB, but should be easy to make it settable.