Re: ncsa rebuttal

[Frederick Gray <fegray@berkeley.edu>]

> Fred could you talk with Valery today ! to see what Merlin
> can offer in the future. We still have to consider whether
> and how to include such info.

Hi, Peter,

Brendan and I had a conversation with Valeri on Friday afternoon.
I think that the basic message is that it would be possible to use merlin for
an analysis of the full dataset, but not by any means convenient:

- The new model of merlin is that heavy users are strongly encouraged to
  contribute funding to purchase dedicated resources in the cluster.

- Merlin3 will include 14 new compute nodes, each with two dual core AMD Opteron
  2.4 GHz CPUs and 8GB RAM.  It is basically intended as a place for
  local users to develop parallelized computational code to eventually move to
  the Horizon supercomputing center.  While any leftover time might be
  made available to "job array" users like us, we are certainly not the
  intended user group.

- The old merlin2 will be kept in operation with about 24 general-use
  machines with dual Athlon 1600MP CPUs, plus 8 reserved for some
  specific users who have contributed funds.  The nominal configuration has
  32 nodes, but usually about 8 have failed at any given moment.
  We can expect that the load will decrease to some extent as parallelized
  users move to merlin3 and Horizon.

- Your NCSA proposal corresponds to about 200K merlin2 CPU hours.
  According to Valeri, that time might be available over a year or two
  of steady running.  However, as this is a shared facility, there are
  no promises; it depends on the needs of other users as well.

- We could expect something like a 5 MB/s sustained bandwidth to the archive,
  which is likely to limit the raw data processing rate: to run through 20 TB
  would take 6 weeks based on the archive transfer alone.

- When I mentioned the referee evaluations of the NCSA proposal, Valeri
  seemed pretty sympathetic to the referees' point of view.  Job array users
  are seen as something of a waste of a nice machine, given the
  investment in interconnect technology in these clusters. 

Thanks,

-- Fred

-- Fred Gray / Visiting Postdoctoral Researcher                 --
-- Department of Physics / University of California, Berkeley   --
-- fegray@berkeley.edu / phone 510-642-2438 / fax 510-643-8497  --