Derivative Distribution

I’m looking into creating a derivative distribution and I note that the instructions at

Haven’t been updated in a year, and are a bit sparse in places.

More specifically I want to add ZFS root support, and Ubuntu’s snapd.

I have 20+ years of Linux and software development experience. Happy to take on any hurdles I encounter.

4 Likes

Hey @stralytic, there is more documentation here: https://docs.01.org/clearlinux/latest/guides/clear/mixer.html

The tooling is always evolving so let us know how it goes or if you need any help.

1 Like

Feel free to point specific gaps or shortcomings. either in github in the form of an issue, or in here.

Oh, please! Also, snaps and a path to community contributed bundles!

1 Like

@puneetse @ahkok
I’m using autospec and it’s creating a spec file that says I shouldn’t edit it, but I need to add flags to the configure command in that file.

What’s the correct way to do this?

In the autospec directory, create a file named configure with the flags to be added and run make autospec again. If the package built fine, you should see those flags in the newly generated spec file.

More documentation on the autospec control files here: https://github.com/clearlinux/autospec#control-files

Is it necessary for a clearlinux derivative to self host every bundle pulled from upstream or am I missing something?

If i do mixer init without --all-upstream will the upstream packages still be available to my derivative?

That’s correct. One reason for that is a security model of all content, down to the file level, being signed and verified from creation to delivery. If bundles from two sources were to be installed that end-to-end verification mechanism would have to take a compromise.

1 Like

When running:
mixer build update

I’m getting the following error:

Error: Failed to run command in container: failed to execute docker run --runtime=runc -i --network=host --rm --workdir /home/clr/mixer --entrypoint mixer -v /home/clr/mixer:/home/clr/mixer clearlinux/mixer:29 build update --native: exit status 137

Any ideas what I’ve done wrong?

1 Like

You can add --native to bypass the docker situation. Mixer used docker at one point in time to overcome a limitation with format bumps.

Alternatively, update your Clear Linux install. The newest version of mixer-tools just so happens to deprecate that docker functionality altogether: https://github.com/clearlinux/mixer-tools/releases

1 Like

Updated my clearlinux to the latest version and now I’m getting a different error. I’ve tried creating a brand new mixer dir and doing mixer init without changing any config.

Always ends with:
2019/11/14 02:26:48 ERROR: Couldn't build bundles: Error building bundles: rpm2archive cmd failed with exit status 1

Full log below:

clr@clr-a55532d214e54fdc904c66be7514d95b~/mixer $ mixer init
Initializing mix version 10 from upstream version 31630
Fetching upstream bundles from https://github.com/clearlinux/clr-bundles/archive/31630.tar.gz
Adding bundle "os-core" from upstream bundles
Adding bundle "os-core-update" from upstream bundles
Adding bundle "bootloader" from upstream bundles
Adding bundle "kernel-native" from upstream bundles
clr@clr-a55532d214e54fdc904c66be7514d95b~/mixer $ mixer build all
Directory walk started
Directory walk done - 0 packages
Temporary output repo path: ./.repodata/
Preparing sqlite DBs
Pool started (with 5 workers)
Pool finished
Generating certificate for signature validation...
Building bundles...
=> BUILD BUNDLES
Adding bundle "hardware-uefi" from upstream bundles
Bundle "bootloader" already in mix; skipping
Adding bundle "linux-firmware" from upstream bundles
Adding bundle "linux-firmware-extras" from upstream bundles
Adding bundle "qemu-guest-additions" from upstream bundles
Bundle "os-core" already in mix; skipping
Adding bundle "os-core-plus" from upstream bundles
Adding bundle "os-core-webproxy" from upstream bundles
Bundle "os-core" already in mix; skipping
Adding bundle "libglib" from upstream bundles
Adding bundle "p11-kit" from upstream bundles
Adding bundle "syslinux" from upstream bundles
Bundle "os-core" already in mix; skipping
Bundle "libglib" already in mix; skipping
Bundle "bootloader" already in mix; skipping
Bundle "linux-firmware" already in mix; skipping
Bundle "linux-firmware-extras" already in mix; skipping
Adding bundle "libstdcpp" from upstream bundles
Bundle "libglib" already in mix; skipping
Adding bundle "findutils" from upstream bundles
Creating bundles for version 10 based on Clear Linux 31630
Preparing new /home/clr/mixer/update/image/10
  and dnf config: /home/clr/mixer/.yum-mix.conf
Packager command-line: dnf --config=/home/clr/mixer/.yum-mix.conf -y --releasever=31630
Resolving packages using 6 workers
processing hardware-uefi
processing libstdcpp
processing os-core-plus
processing linux-firmware-extras
processing qemu-guest-additions
processing libglib
... done with linux-firmware-extras
processing p11-kit
... done with os-core-plus
processing os-core
... done with libstdcpp
processing syslinux
... done with hardware-uefi
processing kernel-native
... done with qemu-guest-additions
processing linux-firmware
... done with libglib
processing os-core-update
... done with p11-kit
processing os-core-webproxy
... done with syslinux
processing findutils
... done with os-core
processing bootloader
... done with kernel-native
... done with linux-firmware
... done with os-core-update
... done with os-core-webproxy
... done with findutils
... done with bootloader
Resolving files using 6 workers
processing bootloader
Bundle bootloader	5 files
processing qemu-guest-additions
Bundle qemu-guest-additions	5 files
processing os-core-plus
processing libstdcpp
Bundle libstdcpp	5 files
processing os-core
Bundle os-core	5 files
processing libglib
Bundle libglib	5 files
processing linux-firmware-extras
processing hardware-uefi
Bundle os-core-plus	5 files
processing linux-firmware
Bundle linux-firmware	5 files
processing os-core-update
processing p11-kit
Bundle hardware-uefi	5 files
processing os-core-webproxy
Bundle os-core-webproxy	5 files
processing kernel-native
Bundle p11-kit	5 files
Bundle kernel-native	5 files
Bundle linux-firmware-extras	5 files
processing syslinux
Bundle syslinux	5 files
processing findutils
Bundle findutils	5 files
Bundle os-core-update	5 files
Cleaning DNF cache before full install
Installing all bundles to full chroot

TIMINGS
  BUILD BUNDLES 0s
TOTAL: 0s
2019/11/14 02:26:48 ERROR: Couldn't build bundles: Error building bundles: rpm2archive cmd failed with exit status 1

Nope!

No! there is no “if it’s not here, then get it from there”. You’re not “overlaying” the OS with another OS. You are creating a new OS that has whatever you choose from it, plus your local things. If you choose to omit something, your downstream installations can’t get it.

1 Like

@stralytic:

I just tried it and was able to create a mix.

Do you have adequate disk space?

clear@clrtest/run/media/clear/U $ swupd info
Distribution:      Clear Linux OS
Installed version: 31640
Version URL:       https://cdn.download.clearlinux.org/update
Content URL:       https://cdn.download.clearlinux.org/update

clear@clrtest~ $ mkdir mix

clear@clrtest~ $ cd mix

clear@clrtest~/mix $ mixer init
Initializing mix version 10 from upstream version 31640
Fetching upstream bundles from https://github.com/clearlinux/clr-bundles/archive/31640.tar.gz
Adding bundle "os-core" from upstream bundles
Adding bundle "os-core-update" from upstream bundles
Adding bundle "bootloader" from upstream bundles
Adding bundle "kernel-native" from upstream bundles

clear@clrtest~/mix $ sudo mixer build all
Password: 
Directory walk started
Directory walk done - 0 packages
Temporary output repo path: ./.repodata/
Preparing sqlite DBs
Pool started (with 5 workers)
Pool finished
Generating certificate for signature validation...
Building bundles...
=> BUILD BUNDLES
Bundle "os-core" already in mix; skipping
Adding bundle "libglib" from upstream bundles
Adding bundle "p11-kit" from upstream bundles
Adding bundle "libstdcpp" from upstream bundles
Adding bundle "findutils" from upstream bundles
Bundle "libglib" already in mix; skipping
Bundle "os-core" already in mix; skipping
Adding bundle "os-core-plus" from upstream bundles
Adding bundle "os-core-webproxy" from upstream bundles
Adding bundle "syslinux" from upstream bundles
Bundle "os-core" already in mix; skipping
Bundle "libglib" already in mix; skipping
Bundle "bootloader" already in mix; skipping
Adding bundle "linux-firmware" from upstream bundles
Adding bundle "linux-firmware-extras" from upstream bundles
Adding bundle "hardware-uefi" from upstream bundles
Bundle "bootloader" already in mix; skipping
Bundle "linux-firmware" already in mix; skipping
Bundle "linux-firmware-extras" already in mix; skipping
Adding bundle "qemu-guest-additions" from upstream bundles
Creating bundles for version 10 based on Clear Linux 31640
Preparing new /home/clear/mix/update/image/10
  and dnf config: /home/clear/mix/.yum-mix.conf
Packager command-line: dnf --config=/home/clear/mix/.yum-mix.conf -y --releasever=31640
Resolving packages using 4 workers
processing findutils
processing os-core
processing linux-firmware
processing qemu-guest-additions
... done with linux-firmware
processing os-core-update
... done with os-core
processing bootloader
... done with os-core-update
processing hardware-uefi
... done with bootloader
processing kernel-native
... done with hardware-uefi
processing syslinux
... done with kernel-native
processing linux-firmware-extras
... done with syslinux
processing os-core-plus
... done with linux-firmware-extras
processing os-core-webproxy
... done with os-core-plus
processing libglib
... done with os-core-webproxy
processing libstdcpp
... done with libglib
processing p11-kit
... done with libstdcpp
... done with p11-kit
... done with findutils
... done with qemu-guest-additions
Resolving files using 4 workers
processing findutils
processing os-core
processing qemu-guest-additions
processing linux-firmware
Bundle os-core	2283 files
processing os-core-update
Bundle qemu-guest-additions	460 files
processing bootloader
Bundle os-core-update	5099 files
processing hardware-uefi
Bundle bootloader	566 files
processing kernel-native
Bundle hardware-uefi	4794 files
processing os-core-plus
Bundle os-core-plus	2218 files
processing os-core-webproxy
Bundle os-core-webproxy	2760 files
processing libglib
Bundle libglib	411 files
processing libstdcpp
Bundle libstdcpp	299 files
processing p11-kit
Bundle p11-kit	1362 files
processing syslinux
Bundle syslinux	533 files
processing linux-firmware-extras
Bundle linux-firmware-extras	875 files
Bundle kernel-native	8702 files
Bundle findutils	472 files
Bundle linux-firmware	999 files
Cleaning DNF cache before full install
Installing all bundles to full chroot
[1/15] libstdcpp
[2/15] p11-kit
[3/15] syslinux
[4/15] linux-firmware-extras
[5/15] os-core-plus
[6/15] os-core-webproxy
[7/15] libglib
[8/15] findutils
[9/15] linux-firmware
[10/15] qemu-guest-additions
[11/15] hardware-uefi
[12/15] os-core
... building special os-core content
[13/15] os-core-update
... Adding swupd default values to os-core-update bundle
[14/15] bootloader
[15/15] kernel-native

TIMINGS
  BUILD BUNDLES 1m40.609s
TOTAL: 1m40.609s
=> CREATE MANIFESTS
Generating initial manifests...
  kernel-native
  os-core-webproxy
  libstdcpp
  libglib
  p11-kit
  syslinux
  linux-firmware-extras
  os-core-plus
  linux-firmware
  qemu-guest-additions
  findutils
  bootloader
  os-core-update
  hardware-uefi
  os-core
Reading bundle includes...
Adding manifest file records...
Detecting manifest changes...
Writing manifest files...
MoM version 10
- bootloader           10
- findutils            10
- hardware-uefi        10
- kernel-native        10
- libglib              10
- libstdcpp            10
- linux-firmware       10
- linux-firmware-extras 10
- os-core              10
- os-core-plus         10
- os-core-update       10
- os-core-webproxy     10
- p11-kit              10
- qemu-guest-additions 10
- syslinux             10
Signing manifest.
Compressing Manifest.MoM
Compressing bundle manifests
  libstdcpp
  p11-kit
  libglib
  syslinux
  linux-firmware-extras
  os-core-webproxy
  linux-firmware
  qemu-guest-additions
  findutils
  os-core-plus
  bootloader
  os-core
  hardware-uefi
  os-core-update
  kernel-native
  full

=> CREATE FULLFILES (3.774s elapsed since start)
Using 4 workers
- Already created: 0
- Not compressed:  0
- Compressed
  - gzip                 470
  - external-xz          8574
Total fullfiles: 9044

=> CREATE ZERO PACKS (1m13.83s elapsed since start)
Using 4 workers
Creating zero pack for bootloader to version 10
Creating zero pack for kernel-native to version 10
Creating zero pack for hardware-uefi to version 10
Creating zero pack for findutils to version 10
Fullfiles in pack findutils: 94
Deltas in pack findutils: 0
Creating zero pack for libglib to version 10
Fullfiles in pack bootloader: 21
Deltas in pack bootloader: 0
Creating zero pack for libstdcpp to version 10
Fullfiles in pack hardware-uefi: 264
Deltas in pack hardware-uefi: 0
Creating zero pack for linux-firmware to version 10
Fullfiles in pack libstdcpp: 15
Deltas in pack libstdcpp: 0
Creating zero pack for linux-firmware-extras to version 10
Fullfiles in pack linux-firmware-extras: 579
Deltas in pack linux-firmware-extras: 0
Creating zero pack for os-core to version 10
Fullfiles in pack libglib: 57
Deltas in pack libglib: 0
Creating zero pack for os-core-plus to version 10
Fullfiles in pack linux-firmware: 844
Deltas in pack linux-firmware: 0
Creating zero pack for os-core-update to version 10
Fullfiles in pack os-core-plus: 1004
Deltas in pack os-core-plus: 0
Creating zero pack for os-core-webproxy to version 10
Fullfiles in pack os-core: 1778
Deltas in pack os-core: 0
Creating zero pack for p11-kit to version 10
Fullfiles in pack os-core-webproxy: 42
Deltas in pack os-core-webproxy: 0
Creating zero pack for qemu-guest-additions to version 10
Fullfiles in pack qemu-guest-additions: 34
Deltas in pack qemu-guest-additions: 0
Creating zero pack for syslinux to version 10
Fullfiles in pack syslinux: 234
Deltas in pack syslinux: 0
Fullfiles in pack p11-kit: 189
Deltas in pack p11-kit: 0
Fullfiles in pack kernel-native: 3186
Deltas in pack kernel-native: 0
Fullfiles in pack os-core-update: 1202
Deltas in pack os-core-update: 0
Saving the upstream URL: https://cdn.download.clearlinux.org
Saving the upstream version: 31640
Setting latest version to 10
Signing latest_version file.
Signing latest file.

TIMINGS
  CREATE MANIFESTS  3.774s
  CREATE FULLFILES  1m10.056s
  CREATE ZERO PACKS 1m4.991s
TOTAL: 2m18.822s
3 Likes