Thilo Six wrote:
> Excerpt from Bob Proulx:
> >>> # rsync -av /mnt/backup/etc/ /etc/
> >>> # rsync -n --delete -av /mnt/backup/etc/ /etc/
> >>> # rsync --delete -av /mnt/backup/etc/ /etc/
> >>
> >> I believe this not to be a good idea. When you do this you mess around with
> >> config files under the control of dpkg. With all the mess that creates.
> >
> > Thank you for the review comment! However I respectfully disagree.
>
> I gather we do have different opinions and workflows in this regard which is
> absolutely fine.
> Maybe i am too worried about this rsync method. Probably due to past
> experiences. But it is just too easy to overlook s.th. see below.
At one level the problem is that there are a lot of files in /etc.
Too many to keep completely in human brain memory all at one time.
Plus files will either exist or not exist based upon what packages are
installed on system making the list impossible to enumerate statically.
But I actually think we are closer to the same than different. More
in a moment.
> > And apparently you do as well because you immediately suggested
> > etckeeper which does exactly the same thing of managing files in /etc.
> > It is the same thing but simply using a different tool.
>
> In this regard i do have an other POV then you. I do believe that
> etckeeper does not change any file in /etc without manually
> intervention. (Obviously files in '/etc/.vcs' are excluded from
> this rule.) So i think this is different from the rsync operation
> you suggested.
Well... I would claim that rsync does not touch any files other than
by manual intervention too. One must invoke the command for it to do
anything. The issue in my mind is not the tool and the implication of
how sweeping of changes it can make at one time but rather whether it
is okay to make any changes at all. If it is okay to make changes
(which it is) then whether the modifications are done small scale or
large scale is simply a matter of scale. Rsync is definitely a large
scale autoautomatic sweeping tool. Whereas etckeeper as I unerstand
it is more of a controlled manual targeted tool.
But I acknowledge and concede your point. /etc contains a large and
unknown set of items. Some of those unknown items should not be
restored from backup. Then using a large sweeping tool like rsync to
sweep *all* of the files into place is undoubtedly going to sweep in
something that should not be restored from backup.
* The new system is brand new. The task is to make it look like the
old system. Therefore there is no problem with restoring /etc/apache2
completely from backup for example. And if a completely restoration
is the task then exactly that is needed. Make the new system look
exactly like the old system.
* But the new system is a brand new install, not a clone, and some
files must remain unique. Such as UUIDs in /etc/fstab, /etc/lvm,
/etc/mdadm, /etc/udev/rules and others as you and I have both pointed
out.
> Let me tell you why i suggested etckeeper.
> Here i use it to commit /etc into $VCS on a regularly basis. This helps me to
> keep track of what is going on in /etc.
> Being able to compare my modifications with the original is a feature you surely
> will not miss anymore, after you get used to it.
This is where our strategy of attack on the problem is different. I
started from the idea that there are a lot of files in /etc but only a
few of them need to be excluded from the restoration. Therefore
identify the files that should be excluded. Then restore everything
*except* for the files that should be excluded. Whereas I think you
were started from the idea that there a lot of files in /etc and only
the known okay files to restore should be restored. Identify the
files that should be restored and only restore those files.
Basically one strategy is include all but those excluded and the other
strategy is exclude all but those included.
I think we are in agreement that some files must be restored and some
files must be excluded. We differ on the technique to get there.
> > Most files in /etc are "conffiles". They are within the realm of
> > control of the user. Changes to them are understood and respected by
> > the package manager. There isn't any "mess" caused by interaction
> > with dpkg.
>
> I do have to commit that i did not spent time yet to understand that "conffiles"
> system deeply.
There is also two strategies to attack this problem too. Either
install the desired configuration from backup first and then install
the packages selecting the exising conffiles. Or install the packages
without any existing configuration and after installation restore the
previous files from backup. I had proposed the first with restoring
the /etc files first. But either should result in the same end
system.
> I do have this snipped in my '/etc/apt/apt.conf.d/99local' though:
> // --force-confold:
> // do not modify the current configuration file, the new version is installed
> // with a .dpkg-dist suffix. With this option alone, even configuration files
> // that you have not modified are left untouched. You need to combine it with
> // --force-confdef to let dpkg overwrite configuration files that you have not
> // modified.
>
> // --force-confdef:
> // ask dpkg to decide alone when it can and prompt otherwise. This is the
> // default behavior of dpkg and this option is mainly useful in combination
> // with --force-confold.
>
> // --force-confmiss:
> // ask dpkg to install the configuration file if it’s currently missing (for
> // example because you have removed the file by mistake).
>
> Dpkg::Options {
> "--force-confdef";
> "--force-confold";
> "--force-confmiss";
> }
> credits for this go to: http://raphaelhertzog.com
That controls the behavior of dpkg when it would normally stop and
interact with the user admin who is installing or upgrading a package
about the handling of a conffile. Should you install the new package
file, keep the existing file, or merge them? Or see the differences
between them? The above answers those questions globally so that it
never stops to ask a question.
I fully agree with confmiss. I routinely set that too. But I have
seen heated arguments from people who have argued (apparently
successfully because the change went in) that removing a required /etc
file is a desired configuration and that it must be preserved.
(Regarding the /etc/network/interfaces file.) This adds yet another
logical state to be considered.
I use a system configuration management system similar to cfengine,
puppet, chef that I have programmed to configure the system. I have
directed it to read-modify-write various configuration files to set
them as desired. In that type of environment it is advantageous to me
to use confnew. With confnew I will always get the package version of
the file with the package default configuration. Which will then
immediately be reconfigured into my desired configuration by the
system management tool. (The cfengine, puppet, chef tools could do
the same.) I choose to do this so that my /etc files are always fresh
and as close as possible to a freshly installed system. I have
machines that have been upgraded many, many times. If I were to
preserve the original file then I would have files hanging around that
were from Potato 2.2 days which would be very ancient now.
Certainly for a Stable system only installing security upgrades the
choices for confold should always be fine since there should not be
any differences in the files in Stable in /etc.
But when upgrading across major upgrades I very much prefer to freshen
them with packaged versions and then merge in any desired deviation
from the pristine. Because major changes will bring major behavior
changes and many times for best results I will want to change how the
configuration is done. Often the world improves and I can remove my
customizations entirely which I always consider a good thing.
> > However that isn't true of all files. And in particular there are
> > uuids associated with lvm and md. The /etc/lvm and /etc/mdadm
> > directories and any other UUIDs in /etc/fstab contain UUIDs and other
> > information specific to the new machine and new filesystems. Those
> > would need to be avoided when restoring from backup. Otherwise the
> > new system would have the UUIDs from the old system and that wouldn't
> > work.
> >
> > * Exclude /etc/fstab
> > * Exclude /etc/lvm
> > * Exclude /etc/mdadm
>
> + /etc/initramfs-tools/conf.d/resume
Ha! I always use lvm and so for me the names in the
/etc/initramfs-tools/conf.d/resume file are always logical names and
would be the same across systems. But point taken. If that is a UUID
then it should be excluded from the restore.
Plus there is your note about ethernet addresses. In addition to the
70-persistent-net.rules file that may also affect people using
"guessnet" or other such networking packages.
> ..and maybe some more. This is why i would look into the difference prior to
> rsyncing. After i had become confident i would use rsync, too.
At the center between the strategies of include and exclude is the
need to identify which files must be included in the restoration and
which files must be excluded. Having that list then we would restore
what should be included and avoid those that should be excluded.
I think you and I would both identify these files on-the-fly by seeing
the files, by looking at the file contents, by other dynamic decision
making. But I fear that for many people that would be beyond their
level of skill. Our discussion shows that it isn't trivial. It takes
some experience.
Can we document what we have learned from this discussion? Would
someone be able to follow that documentation and successfully peform a
restoration as described? Is it possible to create a script that
would analyze the backup and the new system and would then identify
the files that should be included and excluded? I think we can do all
of this but am leaving it as an open question for the moment.
Bob
Attachment:
signature.asc
Description: Digital signature