How to build my own mirror sever

Q&A
1

I would like to use this OS to build my HPCC,but,I need to do it without internet.
So, is there any way to clone the swupd mirror to my computer, and use it ans a personal swupd server?

really need it
thx

2

@464270342, just so I understand what you want to do:

Have a client computer running clear linux that uses an alternate version and update server to keep your system up to date, without internet access of any kind?

you can redirect swupd to use a different url for the version and update server (see the swupd man page on how to do that). You can easily set up your own derivative of Clear using the mixer tool. We have several documents on using mixer on clearlinux.org.

How do you plan to keep your content up to date without internet access?

1 Like
3

It’s possible, but very complex and possibly means you’ll have to mirror a huge amount of content.

Fundamentally, Clear Linux OS is an online OS - you’re expected to have an active network connection. If you don’t, then you should re-evaluate whether you actually want Clear Linux OS.

Can I ask what exactly is the cause of why your system will have NO network access at all? Or do you have intermittent network access?

1 Like
4

Thanks for your reply.

I am going to paticipate in ISC, in this competition,our team will have to build high-performance clusters on site.

But we will be given new CPUs,motherboards……. So, if we carry an SSD with a pre-install OS in it,we might facing really tricky problems.

We are now using CentOS on our HPCC.
In CentOS,we can use “reposync,repomanage,createrepo” command to build a local repo before we go to the compitition.So that we can get our needed softwares even if we don’t have a internet connection.

But we found that we might get a better performance from Clear Linux.So we decide to try it.
I have to admit that I just installed Clear Linux today, and I am not a native English speaker.
The document I found about mix is this:
https://docs.01.org/clearlinux/latest/guides/clear/mixer.html?highlight=mixer#how-it-works

I am a little bit confuse.It seems like a mixer don’t contain all the bundles from upstream,but
need users to decide which bundles to include.Is there any way to clone all the bundles?

And the tutorial is about building an net server.What should I do,if i want to build my swupd
mirror on a local partition?

And I remember RPM is a package of CentOS, why would it show up in the document of mix?

5

In ISC compitition, we have to build our HPCC on site.
But we can’t ensure that we can get internet access.
And carry an SSD with pre-install OS might end up with a lot of issues.(we will get new CPU and motherboards then)
So,our plan is to install OS on site,and use local repo(to insure that we can build our machines even without internet)

6

And I found the connection to swupd server is rather slow in China.
And I don’t found any mirrors in China.
So, I would like to ask Tsinghua University to add a swupd mirror to their server
(https://mirror.tuna.tsinghua.edu.cn/)

But that would need me to provide a way to sync the repo……
（https://github.com/tuna/issues）

7

I see, this is very useful information.

You most likely do not want every bundle to begin with. Since a HPCC cluster likely does not require many end user applications.

So, from that perspective, I suggest you use the mixer method to create a mix and deploy it. This means you need to setup 1 machine as a http service. It can be localhost. You then also need to create your own installer images to do a non-network installation.

All of this is complex and not fully documented because it’s not the typical way that Clear Linux OS is deployed. You’ll need to study the mixer documentation and create a test setup so that you can validate that this all works entirely correctly before you head out to the contest.

8

Clear Linux OS can not be mirrorred (swupd will not accept mirror URLs). We have a CDN and we have a China CDN service that works. If it is slow, I unfortunately can’t fix that but will relay the information to those involved.

9

I believe we’re still waiting on Chinese Government approval to turn on a CDN in China.
If you haven’t realized, we do have a lot of our documentation in Chinese, if that’s easier for you to read. You can toggle the language using the dropdown in the bottom left hand corner of the docs page.
e.g. https://docs.01.org/clearlinux/latest/zh_CN/guides/clear/mixer.html

1 Like
10

It’s not very authentic. Well this is the usual case for technical articles translated to Chinese. It’s hard to find someone to understand those technical stuff while also to be good at expressing them in Chinese.

11

Patches are welcome :slight_smile:

12

I am one of those who can understand technicality but don’t know how to translate it. I basically learnt English by reading documentations, laws and contracts.

13

Thanks a lot for you suggestions

14

Hi, sorry to bring back this thread.
I read that there is lazy mirror method, and some of them use apt-cacher-ng. But i have problem with the pattern, for now i use “PFilePattern: .*” that rewrite the built-in value to allow all kind of files

15

After some testing on regex101

PfilePatternEX: .*\.tar|update.*|latest.*|.*\.sig|.*-.*|Manifest.*

seems ok
If there is more simple way, please let me know

16

https://salsa.debian.org/blade/apt-cacher-ng/-/blob/next/source/acfg_defaults.cc
i place it here, may useful.