[Slapt-get-devel] slapt-get source priority patches

[Jason Woodward]

Hi Ken,

> Thanks for your reply!  You've given me a few things to think about...

Certainly the same here! =]

> I did consider using suffixes on the SOURCE line, but I came to the 
> conclusion that adding new tokens might be a more backwards- and 
> forwards-compatible way of extending the config file format in general. 
>     Nevertheless, I take your point about gslapt.  I don't have any 
> objections to putting the priority on the source line if that's what 
> you'd prefer.

I believe that is probably the right thing to do.  We can certainly build off of that from there.

> Regarding the named priorities, I agree that they would be more 
> intuitive and less confusing to the average user.  But I wouldn't want 
> to limit the number of possible levels to just 3.  I can think of 
> situations where you would want more levels, (although typically not 
> many more, perhaps an enterprise-level package repository and a local 
> one).  One can envisage complex configuration management schemes where a 
> whole hierarchy of mirrors with different priorities are overlaid on top 
> of each other.  I concede that not many users would find that useful, 
> but I certainly don't want to explicitly block such a powerful feature 
> (and one that I have used myself!).

How about extending at the source level?  If you just defined them in an enumeration then some
header tweaks and maybe some more small mods there and you should be ok.  I don't want to
hinder any major advanced usage at the cost of the casual user, but I like to follow the 80/20
rule.  It should be easy for 80%, and possible for the 20%.
 
> Agreed.  This was my attempt to avoid an invasive patch, and to also 
> avoid any backward-incompatible breakage of the libslapt interface, but 
> I agree that a source structure would be the correct way to do it.

Yeah, lets break away.

> > Same with slapt_cmp_pkg().
> 
> Not so sure about that.  Notice that once a package has its metadata 
> written to the package_data cache, the priority of the source is written 
> along with it, and it is this value that gets used in all priority 
> comparisons, not the current value from the slapt-getrc file.  This was 
> deliberate:  To make a source priority change take effect, it is not 
> sufficient to just modify the config file, you have to do a successful 
> slapt-get --update.

It seems like it will still work if slapt_init_pkg() sets the default priority.  If you have a
use case I can't see lets hash it out.

> Note that the big danger with using source priorities is that if a 
> mirror (with priority > 0) should become unavailable, then you can't 
> safely update any packages at all.  This is because if you remove a 
> mirror, then the other mirrors with lower priorities will start to 
> assert their (previously suppressed) packages and there will be a lot of 
> unwanted reinstalling.  The command-line version of slapt-get functions 
> fine in this respect:  If a mirror is offline, it just aborts and leaves 
> the package_data file unchanged.  But you will notice that I had to add 
> a "safety feature" to gslapt, so that if a source with priority > 0 is 
> unavailable, it aborts the whole show rather than giving the user the 
> chance to continue with the remaining mirrors.  This problem doesn't 
> apply to priority 0 mirrors; they can come and go as before, without any 
> danger of upsetting the installed package base.

Agreed.  This is were the lack of extensibility in the /var/log/packages/* storage is failing
us.  I would rather be able to provide the user with the notice and let them decide than to
fail outright.  Maybe updating the source failure notice in gslapt is adequate. 

> Back to slapt_cmp_pkg(), this should really have been split into two 
> separate functions:  One to compare two packages available on mirrors, 
> and another to compare a mirror package with an installed package.  The 
> ordering rules for these two comparison types are different.  But in 
> order to avoid too much breakage in libslapt, I settled for the trick of 
> using priority -1 to flag an installed package, and implementing 
> slapt_cmp_pkg() very carefully to account for this.  If we don't mind 
> breaking the libslapt interface a bit, then this hack can go away.

A quick test seems to be ok without the -1 hack (and removing the special case).  Can you
provide a test case?

> Hmmm.  I did consider using the "patch priority = mirror priority + 1" 
> rule myself, but it only really works if you have a fixed priority 
> system like you have suggested.  For a situation where you have lots of 
> mirrors and lots of priority levels, it doesn't work so well.  For 
> myself, manually specifying the patch priority seemed the best solution, 
> but I agree it is less than intuitive for users.  I need to think 
> carefully about what could be done about this...

I still think the previous enumeration handles this if we can agree on the fixed priority
definition.

> I'm not so sure now.  I think your suggestions are all valid, in that 
> the configuration needs to be a bit more intuitive for typical users. 
> At the same time, I wouldn't want to reduce the level of control that is 
> available to "power users" who want to play with complex priority 
> configurations.  

Lets see how it works out before we declare the power users as losers in a more fixed
implementation.

> Next up, a patch to override priorities based on regexp 
> matching of package names! ;-)

The inverse of exclude?

> Let me know if you have any further insights; particularly regarding the 
> way you would prefer to change (or not) the libslapt interface.  Or feel 
> free to hack away yourself if you have the time and inclination!  I'm 
> not looking for any "ownership" of these patches but I'll help out if I can!

Certainly, one implementation deserves another!  I am attaching an updated patch that takes
into consideration my points from the earlier email.  I converted the integer priorities to
string tokens.  I simplified the configuration/source interface by embedding the priority
parsing within the _add_source call (and using priority suffixes).  I merged your docs into the
README and FAQ along with some translation updates.  I also removed the installed -1 priority
hack, made slapt_cmp_pkgs() iits own rather than a macro wrapper for slapt_cmp_pkg_versions()
that takes into account the priority if the version strings are not identical before passing
control along to slapt_cmp_pkg_versions().


Let me know what you think.  This is ready, imo, to be committed.  We can certainly
enhance/extend from here.  gslapt can wait until this interface settles down.


Thanks again and take care,
jason
--
Jason Woodward
woodwardj at jaos.org


-------------- next part --------------
A non-text attachment was scrubbed...
Name: priority.patch
Type: text/x-patch
Size: 71124 bytes
Desc: not available
URL: <http://software.jaos.org/pipermail/slapt-get-devel/attachments/20081124/269f038d/attachment-0001.bin>