doc updates... meanwhile, it doesn't boot, complaining about not finding the rootfs in the initrd. working on that in the background.

docs/TODO

@@ -22,6 +22,7 @@
 -There *has* to be a better way of handling package installation in the chroots.
 -maybe remove lxde, firefox, chrome and replace with enlightenment/midori?
 -custom repo? https://brainwreckedtech.wordpress.com/2013/01/27/making-your-own-arch-linux-repository/
+-implement better "additional" packages list. specify for path in build.ini- these should be more easily changed by end users. DON'T TOUCH iso.pkgs.lst since those are necessary for booting.
 
 
 ## NETWORKING ##

docs/manual/HEAD.adoc

@@ -8,7 +8,7 @@ v1.0, 2016-12
 :toc: preamble
 :toc2: left
 :sectnums:
-:toclevels: 5
+:toclevels: 2
 // So there's currently a "bug" in that the TOC will display with continued numbering across parts.
 // i essentially want the opposite of https://github.com/asciidoctor/asciidoctor/issues/979 TODO
 
@@ -28,7 +28,7 @@ A lot of research went into how low-level boot operations take place when writin
 === What is BDisk?
 BDisk refers to both a live distribution I use in my own uses (for rescue situations, recovery, etc.) but foremost and most importantly, it also refers to the tool I use for building that distribution. In other words, it's both a complete GNU/Linux distribution you can run entirely from USB/CD/DVD/etc. (without needing to install it to your hard drive)... and also the name of a tool to create a custom GNU/Linux install. The latter is what this project and documentation refer to when the word “BDisk” is used.
 
-When I rewrote BDisk in Python 3.x (Version 0.x-2.x of BDisk was written in Bash, and horribly inelegant and rigid. I should take the time to note that I am still quite new to Python so expect there to be plenty of optimizations to be made and general WTF-ery from seasoned Python developers), one of my main goals was to make it as easy to use as possible. This is surprisingly hard to do- it’s quite challenging to try to approach software you’ve written with the mindset of someone other than you.
+This documentation was started when I rewrote BDisk in Python 3.x; versions 0.x-2.x of BDisk were written in Bash, and horribly inelegant and rigid. footnote:[I should take the time to note that I am still quite new to Python so expect there to be plenty of optimizations to be made and general WTF-ery from seasoned Python developers. If you encounter any bugs or improvements, please <<FURTHER.adoc#_bug_reports_feature_requests,report them>>! It'd be much appreciated.] One of my main goals was to make BDisk as easy to use as possible. This is surprisingly hard to do- it’s quite challenging to try to approach software you’ve written with the mindset of someone other than you.
 
 It’s my hope that by releasing this utility (and documenting it), you can use it and save some time for yourself as well (and hopefully get the chance to learn a bit more in the process!). 
 

docs/manual/USER.adoc

@@ -20,4 +20,5 @@ This manual will give you the information you need to build your very own live G
 include::user/GETTING_STARTED.adoc[]
 include::user/IMPORTANT_CONCEPTS.adoc[]
 include::user/PROJECT_LAYOUT.adoc[]
+include::user/BUILDINI.adoc[]
 

docs/manual/further/BUGS.adoc

@@ -1,4 +1,6 @@
 == Bug Reports/Feature Requests
+NOTE: It is possible to submit a bug or feature request without registering in my bugtracker. One of my pet peeves is needing to create an account/register on a bugtracker simply to report a bug! The following links only require an email address to file a bug (which is necessary in case I need any further clarification from you or to keep you updated on the stats of the bug/feature request -- so please be sure to use a valid email address).
+
 === Bugs
 If you encounter any bugs in *BDisk*, you can file a bug report https://bugs.square-r00t.net/index.php?do=newtask&project=2&task_type=1[here^].
 
@@ -10,7 +12,7 @@ If you have any features you'd like to see or you think would help *BDisk* becom
 If you have any suggestions on how to improve *this documentation* or feel it's missing information that could be useful, please file a feature request https://bugs.square-r00t.net/index.php?do=newtask&project=8&task_type=2[here^].
 
 === Patches
-I gladly welcome patches, but I deplore using GitHub (even though I https://github.com/johnnybubonic/BDisk[have a mirror there^]). For this reason, please follow the same https://www.kernel.org/doc/Documentation/SubmittingPatches[patch/pull request process] for the Linux kernel and email it to bts@square-r00t.net.
+I gladly welcome https://www.gnu.org/software/diffutils/manual/html_node/Unified-Format.html[patches^], but I deplore using GitHub (even though I https://github.com/johnnybubonic/BDisk[have a mirror there^]). For this reason, please follow the same https://www.kernel.org/doc/Documentation/SubmittingPatches[patch/pull request process] for the Linux kernel and email it to bts@square-r00t.net.
 
-Alternatively, you may attach a patch to a <<_bugs,bug report>>.
+Alternatively, you may attach a patch to a <<_bugs,bug report>>/<<_feature_requests,feature request>>.
 

docs/manual/user/BUILDINI.adoc

@@ -0,0 +1,75 @@
+== The `build.ini` file
+This file is where you can specify some of the very basics of BDisk building. It allows you to specify/define certain variables and settings used by the build process. It uses https://docs.python.org/3/library/configparser.html[ConfigParser^] for the parsing engine, and you can do some https://wiki.python.org/moin/ConfigParserExamples[more advanced^] things with it than I demonstrate in the default.
+
+It's single-level, but divided into "sections". This is unfortunately a limitation of ConfigParser, but it should be easy enough to follow.
+
+Blank lines are ignored, as well as any lines beginning with `#` and `;`. There are some restrictions and recommendations for some values, so be sure to note them when they occur. Variables referencing other values in the `build.ini` are allowed in the format of `${value}` if it's in the same section; otherwise, `${section:value}` can be used.
+
+We'll go into more detail for each section below.
+
+=== Example
+ [bdisk]
+ name = BDISK
+ uxname = bdisk
+ pname = BDisk
+ ver =
+ dev = A Developer
+ email = dev@domain.tld
+ desc = A rescue/restore live environment.
+ uri = https://domain.tld
+ root_password =
+ user = yes
+ [user]
+ username = ${bdisk:uxname}
+ name = Default user
+ password = $$6$$t92Uvm1ETLocDb1D$$BvI0Sa6CSXxzIKBinIaJHb1gLJWheoXp7WzdideAJN46aChFu3hKg07QaIJNk4dfIJ2ry3tEfo3FRvstKWasg/
+ [build]
+ mirror = mirror.us.leaseweb.net
+ mirrorproto = https
+ mirrorpath = /archlinux/iso/latest/
+ mirrorfile =
+ mirrorchksum = ${mirrorpath}sha1sums.txt
+ mirrorgpgsig =
+ gpgkey = 7F2D434B9741E8AC
+ gpgkeyserver =
+ gpg = no
+ dlpath = /var/tmp/${bdisk:uxname}
+ chrootdir = /var/tmp/chroots
+ basedir = /opt/dev/bdisk
+ isodir = ${dlpath}/iso
+ srcdir = ${dlpath}/src
+ prepdir = ${dlpath}/temp
+ archboot = ${prepdir}/${bdisk:name}
+ mountpt = /mnt/${bdisk:uxname}
+ multiarch = yes
+ ipxe = no
+ i_am_a_racecar = no
+ [gpg]
+ mygpgkey =
+ mygpghome =
+ [sync]
+ http = no
+ tftp = no
+ git = no
+ rsync = no
+ [http]
+ path = ${build:dlpath}/http
+ user = http
+ group = http
+ [tftp]
+ path = ${build:dlpath}/tftpboot
+ user = root
+ group = root
+ [ipxe]
+ iso = no
+ uri = https://domain.tld
+ ssldir = ${build:dlpath}/ssl
+ ssl_ca = ${ssldir}/ca.crt
+ ssl_cakey = ${ssldir}/ca.key
+ ssl_crt = ${ssldir}/main.crt
+ ssl_key = ${ssldir}/main.key
+ [rsync]
+ host =
+ user =
+ path =
+ iso = no

docs/manual/user/GETTING_STARTED.adoc

@@ -45,8 +45,6 @@ These are needed for using BDisk.
 
 * https://www.python.org/[Python^] (>=3.5)
 * https://github.com/dosfstools/dosfstools[dosfstools^]
-* http://gcc.gnu.org[gcc (multilib)^] (>=6.x)
-* http://gcc.gnu.org[gcc-libs (multilib)^] (>=6.x)
 * http://libburnia-project.org[libisoburn^]
 * http://squashfs.sourceforge.net[squashfs-tools^] (>=4.2)
 
@@ -62,6 +60,10 @@ While not strictly necessary, these will greatly enhance your BDisk usage. I've
 
 NOTE: If you do not wish to install any of these or cannot install them, be sure to disable the relevant options in the `build.ini` file (we'll talk about that later). The default `extra/dist.build.ini` should be sane enough to not require any of these.
 
+* http://gcc.gnu.org[gcc (multilib)^] (>=6.x)
+** Needed for building iPXE.
+* http://gcc.gnu.org[gcc-libs (multilib)^] (>=6.x)
+** (Same as gcc.)
 * https://git-scm.com/[git^]
 ** For autodetection of version, automatically making commits for your project, etc.
 * https://www.gnupg.org/[gpg/gnupg^] (>=2.1.11)

docs/manual/user/IMPORTANT_CONCEPTS.adoc

@@ -16,6 +16,10 @@ A *live distro*, *live CD*, *live DVD*, *live USB*, and the like are a way of bo
 
 *Chroots*, *chrooting*, and the like are variants on the word *chroot*. A *chroot* is a way of running a GNU/Linux install "inside" another GNU/Linux distro. It's sort of like a virtual machine, or VM, except that it's a lot more lightweight and it doesn't do any actual virtualization- and uses the host's kernel, memory mapping, etc. It's very useful for development of operating systems.
 
+*PXE*, or Pre-boot eXecution Environment, is a way of booting operating systems over a local network.
+
+*iPXE* is a http://ipxe.org/[project^] that builds a very small Linux kernel, UNDI (traditional PXE) images, and the like that allow you to essentially use PXE over the Internet. It's very flexible and customizable, and supports a custom scripting engine and such.
+
 === Why live media is necessary/Why you might want BDisk
 "But Brent," I hear you ask in a voice which most likely is nothing close to what you actually sound like and entirely in my head, "Why would I need a live CD/USB/etc.? And why BDisk?"
 
@@ -26,7 +30,7 @@ Elementary, my dear imaginary reader! I touch on some reasons why one might want
 * It performs optimizations and compression to help you get the smallest ISO possible.
 * In addition to building hybrid ISOs, it supports building iPXE hybrid ISOs (meaning you only need a very small file; the rest of the operating system boots over the Internet).
 * It supports both BIOS and UEFI systems- both the full image and the iPXE images.
-* It supports multiple architectures (x86_64, i686, possibly IA64 -- untested).
+* It supports multiple architectures (x86_64, i686, possibly IA64 -- untested) on the same ISO.
 * It supports automatically syncing to a web mirror, PXE boot server, etc. via rsync upon successful build.
 * It supports SecureBoot.
 

docs/manual/user/PROJECT_LAYOUT.adoc

@@ -1,4 +1,73 @@
 == Project Structure
+The following is a tree of files and directories in a BDisk root directory. Note that yours may not look quite like this, as BDisk supports some directory relocation to aid in packaging for distros. These will be examined in-depth in the coming sections.
+
+ <BDisk root directory>
+ ├── bdisk
+ │   ├── bchroot.py
+ │   ├── bdisk.py
+ │   ├── bGPG.py
+ │   ├── bSSL.py
+ │   ├── bsync.py
+ │   ├── build.py
+ │   ├── host.py
+ │   ├── ipxe.py
+ │   └── prep.py
+ ├── docs
+ │   ├── COPYING
+ │   ├── LICENSE -> COPYING
+ │   ├── manual
+ │   │   └── (...)
+ │   ├── README
+ ├── examples
+ │   └── HTTP
+ │       └── (...)
+ ├── extra
+ │   ├── bdisk.png
+ │   ├── bin
+ │   │   └── (...)
+ │   ├── dist.build.ini
+ │   ├── external
+ │   │   └── (...)
+ │   ├── mirrorlist
+ │   ├── pre-build.d
+ │   │   ├── (...)
+ │   │   ├── i686
+ │   │   │   └── (...)
+ │   │   └── x86_64
+ │   │       └── (...)
+ │   └── templates
+ │       ├── BIOS
+ │       │   ├── isolinux.cfg.arch.j2
+ │       │   └── isolinux.cfg.multi.j2
+ │       ├── EFI
+ │       │   ├── base.conf.j2
+ │       │   ├── loader.conf.j2
+ │       │   ├── ram.conf.j2
+ │       │   ├── uefi1.conf.j2
+ │       │   └── uefi2.conf.j2
+ │       ├── GPG.j2
+ │       ├── iPXE
+ │       │   ├── BIOS
+ │       │   │   └── isolinux.cfg.j2
+ │       │   ├── EFI
+ │       │   │   ├── base.conf.j2
+ │       │   │   └── loader.conf.j2
+ │       │   ├── EMBED.j2
+ │       │   ├── patches
+ │       │   │   ├── 01.git-version.patch.j2
+ │       │   │   └── 02.banner.patch.j2
+ │       │   └── ssl
+ │       │       └── openssl.cnf
+ │       ├── VARS.txt.j2
+ │       └── VERSION_INFO.txt.j2
+ └── overlay
+     ├── (...)
+     ├── i686
+     └── x86_64
 
 include::fslayout/BDISK.adoc[]
+include::fslayout/DOCS.adoc[]
+include::fslayout/EXAMPLES.adoc[]
+include::fslayout/EXTRA.adoc[]
+include::fslayout/OVERLAY.adoc[]
 

docs/manual/user/fslayout/BDISK.adoc

@@ -1,6 +1,30 @@
-== bdisk/
-This directory contains the "heart" of BDisk. It essentially is a Python module package. It contains several python "subpackages" split into different files that provide different functions for BDisk.
+=== bdisk/
+This directory contains the "heart" of BDisk. It essentially is a Python module package. It contains several python "subpackages" split into different files that provide different functions for BDisk. Chances are you won't ever need to touch anything in here.
 
-=== bchroot.py
+==== bchroot.py
 This file controls creation of the chroots -- the directories in which BDisk builds the actual system that is booted into.
 
+==== bdisk.py
+This file is a sort of "wrapper" -- it pulls all the other files in this directory together into a single usable python script. In other words, to build a BDisk distribution, you would simply run `bdisk/bdisk.py` -- that's it!
+
+==== bGPG.py
+This contaions functions having to do with GPG -- signing files, verifying other signatures, generating a key (if one wasn't specified), using a key (if one was specified), etc.
+
+==== bSSL.py
+Functions having to do with OpenSSL are stored here. This is used primarily for "mini" builds (via iPXE), they let you boot your BDisk distribution over the Internet. If an SSL key, CA certificate, etc. weren't defined and you want to build a mini image, this file contains functions that will build an SSL PKI (public key infrastructure) for you automatically.
+
+==== bsync.py
+This file has functions relating to copying your BDisk build to various resources. For instance, if you want your ISO available to download then this file would be used to copy your finished build to an HTTP server/root you specify.
+
+==== build.py
+This is responsible for building the "full" ISO, building UEFI support, etc.
+
+==== host.py
+These functions are used to perform "meta" tasks such as get information about the build host, find the `build.ini` file, and parse your configuration options.
+
+==== ipxe.py
+This file handles building the "mini" ISO via iPXE.
+
+==== prep.py
+This contains functions that download the base tarball releases, preps them for `bchroot.py`, builds necessary directory structures, and performs the overlay preparations.
+

docs/manual/user/fslayout/DOCS.adoc

@@ -0,0 +1,15 @@
+=== docs/
+This directory contains various documentation and other helpful text.
+
+==== COPYING
+This contains BDisk's license, the GPLv3.
+
+==== LICENSE
+This is simply a link to `COPYING`.
+
+==== manual/
+This directory contains the documentation source you're reading right now! It's written in http://asciidoc.org/[asciidoc^] (well, to be more precise it's written in/has some http://asciidoctor.org/[asciidoctor^]-isms). I'd recommend reading the rendered version, as the source (while perfectly human-readable) is written in a very modular fashion so it may be inconvenient to read each source file and following include directives.
+
+==== README
+This is a placeholder for common convention; it simply tells you to read the manual (and where to find it/build it).
+

docs/manual/user/fslayout/EXAMPLES.adoc

@@ -0,0 +1,3 @@
+=== examples/
+This directory contains example filesystem layouts for e.g. your webserver (for iPXE), or your PXE server via TFTP.
+

docs/manual/user/fslayout/EXTRA.adoc

@@ -0,0 +1,21 @@
+=== extra/
+This directory contains multiple "support files" for BDisk building.
+
+==== bdisk.png
+This file is used for bootloader graphics. If you change the name of the project, this can be named something different -- see <<_uxname,the section on uxname>>.
+
+==== bin/
+This directory contains sample code or extra tools that don't have anything to do with BDisk normal operation but are useful in building a BDisk distribution.
+
+==== dist.build.ini
+This is the "source-provided"/upstream example `build.ini`. It will be sourced for any missing configuration options or the like.
+
+==== external/
+This directory contains external source code for use with extra features in BDisk that would otherwise be inconvenient to fetch and build dynamically.
+
+==== pkg.build.ini
+This is the recommended default `build.ini` file for packagers of distro repositories to use when packaging BDisk for inclusion in a package manager.
+
+include::PREBUILD.adoc[]
+
+include::TEMPLATES.adoc[]

docs/manual/user/fslayout/OVERLAY.adoc

@@ -0,0 +1,13 @@
+=== overlay/
+This directory follows similar rules to the <<_pre_build_d,pre-build.d/>> directory, except it is applied *after* the chroots are prepared (as it is designed to be user-centric rather than core functionality). We'll go more into this later in-depth, as this is where most of your customizations will be done.
+
+For files that should be included in both chroots, simply recreate the path with the desired file. For instance, if you want a file `/etc/foo/bar.conf` to exist in both i686 and x86_64 versions, it would exist as the path `overlay/etc/foo/bar.conf`.
+
+It follows the following structure:
+
+==== i686/
+This contains modifications that should be applied to the i686 version only. If you wanted a file to exist only in the i686 version at `/etc/a/b.conf`, it would be placed in `overlay/i686/etc/a/b.conf`.
+
+==== x86_64/
+This contains modifications that should be applied to the x86_64 version only. If you wanted a file to exist only in the x86_64 version at `/etc/z/y.conf`, it would be placed in `overlay/x86_64/etc/z/y.conf`.
+

docs/manual/user/fslayout/PREBUILD.adoc

@@ -0,0 +1,13 @@
+==== pre-build.d/
+This file contains a "core" overlay. Generally these files shouldn't be modified unless you know what you're doing, but there are some interesting things you can do in here. Generally speaking, though, you'll want to place your modifications in the `overlay/` directory.
+
+For files that should be included in both chroots, simply recreate the path with the desired file. For instance, if you want a file `/etc/foo/bar.conf` to exist in both i686 and x86_64 versions, it would exist as the path `pre-build.d/etc/foo/bar.conf`.
+
+It follows the following structure:
+
+===== i686/
+This contains modifications that should be applied to the i686 version *only*. If you wanted a file to exist only in the i686 version at `/etc/a/b.conf`, it would be placed in `pre-build.d/i686/etc/a/b.conf`.
+
+===== x86_64/
+This contains modifications that should be applied to the x86_64 version *only*. If you wanted a file to exist only in the x86_64 version at `/etc/z/y.conf`, it would be placed in `pre-build.d/x86_64/etc/z/y.conf`.
+

docs/manual/user/fslayout/TEMPLATES.adoc

@@ -0,0 +1,45 @@
+==== templates/
+This directory contains dynamic templates used for dynamic configuration building and other such things. They are written in http://jinja.pocoo.org/[Jinja2^]. If you haven't used Jinja2 before, the http://jinja.pocoo.org/docs/dev/templates/[templating documentation^] will prove to be very useful.
+
+This allows you to customize low-level behaviour of BDisk without modifying the source.
+
+===== BIOS/
+The `isolinux.cfg.arch.j2` template controls boot options for the single-arch versions of BDisk. In other words if you only build an i686 or only an x86_64 version, this is the template that would be used for BIOS boot mode.
+
+The `isolinux.cfg.multi.j2` is used for multi-arch. It manages booting for both i686 and x86_64 versions.
+
+These files will let you change the behaviour of booting in BIOS systems. The menu colour, the menu entries, the menu default, etc.
+
+===== EFI/
+The files in here are https://www.freedesktop.org/wiki/Software/systemd/systemd-boot/[systemd-boot^] configurations. The distributed defaults are:
+
+`base.conf.j2`, which controls the "main"/default entry.
+
+`loader.conf.j2`, the meta configuration file which tells the loader which entry to load by default and which entries to include.
+
+`ram.conf.j2` which allows BDisk to run entirely from RAM.
+
+`uefi1.conf.j2` which provides a UEFI shell (for older UEFI systems).
+
+`uefi2.conf.j2` which provides a UEFI shell (for newer UEFI systems).
+
+===== GPG.j2
+This file contains default parameters for the https://www.gnupg.org/documentation/manuals/gnupg/Unattended-GPG-key-generation.html[GPG key generation], if we need to automatically generate a key.
+
+===== iPXE/
+This directory holds templates for iPXE/mini builds.
+
+The `BIOS/` directory is similar to <<_bios, BIOS/>> mentioned above, but it only needs one configuration file and is a much more minimal design (since its entire purpose is to chainload to the iPXE loader).
+
+The `EFI/` directory is similar to <<_efi, EFI/>> above also, but needs fewer configuration files (its only purpose is to bootstrap iPXE).
+
+`EMBED.j2` is the iPXE http://ipxe.org/scripting[embedded script^] (http://ipxe.org/embed[more info^]). This is what chainloads the remote resources (kernel, intird, squashed filesystem images, and so forth).
+
+The `patches/` directory largely control branding of the mini ISO. They are in https://www.gnu.org/software/diffutils/manual/html_node/Unified-Format.html[unified diff^] (or "patch") format.
+
+===== VARS.txt.j2
+This template passes variables specified in the configuration and generated by BDisk to the chroot bash script.
+
+===== VERSION.txt.j2
+This template specifies a VERSION.txt file placed in various locations throughout the builds to help identify which version, build, etc. the ISO is.
+