Compute & space resources & environment

A number of sites are graciously allowing us to use some of their compute/space.

First, you should read DirectoryStructure and SyncFilesBetweenSites

JHU

• Compute
  • 12 nodes with 2 x 2.4Ghz processors and 4G of RAM each, running Sun GridEngine version 6. Sungrid job submissions can be done from machines named x13 to x24. Use the option "-l ws06afsr=true" when submitting jobs.

• Space
  • 200G on a central NAS server accessible over NFS

• Testing GMTK
  • to test Karen's parallel scripts,
    • unzip /export/ws06afsr/users/lyung/gmtktest/test_parallel_scripts_v2.tgz to home directory
    • checkout parallel and bin (see Partha's CvsDocumentation)
    • follow Karen's README commands

• Setting up your environment
  • To use the common versions of GMTK, HTK and parallel scripts,
    • add the following to your .bash_profile: export PATH=$PATH:/export/ws06afsr/src_checked_out/bin/`uname`/:/export/ws06afsr/src_checked_out/parallel

Edinburgh

• Compute -- non-exclusive (but low-competition) use of:
  • Townhill GridEngine cluster
    • 34 quad-processor-equivalent Dell PE1425 3.2GHz nodes with 4Gb RAM per node (equivalent to 136 virtual processors)
    • https://wiki.inf.ed.ac.uk/DistributedComputing/Townhill - the headnode is called townhill
    • we can also run on the three other clusters in Edinburgh
      • see https://wiki.inf.ed.ac.uk/DistributedComputing/GridEngine for their headnode names and their memory/speed specs
      • we will have to stop using them as soon as the MT group needs this resource
  • Condor pool
    • Currently about 200 nodes of varying specs, but expect this to vary somewhat over the summer
    • https://wiki.inf.ed.ac.uk/DistributedComputing/Condor

• Space -- 500 GB at /group/cstr/projects/dbns/export/ws06afsr which is NFS/AFS mounted on both the townhill cluster head & compute nodes, and all the condor pool machines.
  • Under Grid Engine: it is not possible to ssh to the townhill nodes. Initially, we will try using files directly from /group/cstr/projects/dbns/ws06afsr but if this causes problems, then we will have to start copying static data, and possibly parameter files, to the node scratch disks
  • Under Condor: always use files directly from /group/cstr/projects/dbns/export/ws06afsr and don't copy to local scratch disk.

• Documentation on
  • LogInToEdinburgh
  • GridEngineAtEdinburgh
  • CondorAtEdinburgh

UIUC

• Compute
  • Cluster with 16 compute nodes we two 2.8Ghz Xeon CPUs and 1Gb RAM each connected by a 10gb ethernet. Jobs can be submitted via LAVA or the sungrid engine.
  • About 6 assorted machines with two or four processors and two or four gigs of RAM.

• Space -- The total is about 3 terabytes on the cluster. The assorted machines also have may be 200gigs of networked available disk space.

• login instructions:
  • ssh to ifp-32.ifp.uiuc.edu. Your login and password will be provided shortly.
  • To keep things organized, please keep your shared files in /cworkspace/ifp-32-1/hasegawa/jhu06/. That directory is on a networked filesystem. A lot more space is available on the individual compute nodes. You can access this space via /cworkspace/c1-XX where XX is 1 through 16. You can also connect to the compute nodes directly from the ifp-32 machine by 'ssh compute-1-XX' where XX is 1 through 16, but there should be no need to do that.
  • You can submit jobs via the sun grid engine or via lava. You can also see various statistics about the server. Make sure you use https and not http if you submit your password to it.
  • to see Karen's parallel test gmtk test in action run ./emtrain_parallel.JHU_ws06 header.emtrain.test in /cworkspace/ifp-32-1/hasegawa/jhu06/akantor/test_emtrain_parallel .

UW

• Compute
  • Music cluster running pmake
    • 36 CPUs, each about 1Gflop, 4 per box, 4GB per box

• First log in
• Using pmake on music

Fine print

• set ALLOWTOPICVIEW = WorkshopGroup

-- KarenLivescu - 12 May 2006