[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: update of mirrors

• To: Multiple recipients of list <epc-l@iucr.org>
• Subject: Re: update of mirrors
• From: Brian McMahon <bm@iucr.org>
• Date: Fri, 4 Sep 1998 09:26:28 +0100 (BST)

Dear Yves
 
> Just by curiosity I have looked this morning to all mirrors to have an
> idea of the time needed to update the different mirrors.

Here is an update on the position:

Tests have demonstrated that we can mirror using wget but with anonymous ftp
protocol at least an order of magnitude faster, and I shall recommend that
all the mirrors switch to that approach.

But first I need to clean up my WWW directory tree for two reasons:
   (1) to delete or move to another place all the junk and ancillary files
       that have crept into various directories;
   (2) to resolve soft links that point to other files on my local
       filesystem.
(1) is tiresome, but not altogether essential, since it just means the junk
is copied to the mirrors and clutters up their filesystem, but is never seen
by users. (2) is more difficult. They didn't cause a problem until now,
because wget/http followed the links on my system, and downloaded the files
as hard files to the mirrors. wget/ftp will preserve the symbolic links,
rendering in most cases useless locations on the mirrors.

I propose to resolve (2) by changing most of the symlinks (which point to
structure-factor files) into URLs to our ftp site. But this means building a
new ftp hierarchy, and superimposing this on the existing archive file
structure. Since we are also rearranging the archive structure, the details
are quite complex.

It's a project I have high on my list of priorities (the new archive
structure is itself important for the efficient management of our
publications archive into the future), but it needs a suitable long, quiet
period for implementation. I can't yet give a firm schedule for it.

As soon as it's complete and working, I shall vigorously encourage all the
mirrors to adopt the new mirroring scheme and update their sites frequently
(since the overhead will be so much less, a daily update will usually be
feasible).

The Israeli site is not only out of date, it's also incomplete. The South
African site is hampered by very slow communications and complications with
working through a proxy. There is a Japan site, known to the local Japanese
community, which does update regularly but is unwilling to "go public" until
more efficient mirroring is implemented. The other sites are working at
least tolerably well.

I look forward to the day - I hope not far off - when all the sites can be
relied upon to be quite up to date.

Regards
Brian

Reply to: [list | sender only]

• Prev by Date: update of mirrors
• Next by Date: Re: ICSTI-forum
• Prev by thread: update of mirrors
• Next by thread: RSC web site
• Index(es):
  • Date
  • Thread