Inhalt
2.1 Preparatory Access
2.1.1 Requirements
2.1.2 Detailed information
2.2 Project Access
2.2.1 Requirements
2.2.2 Detailed information
2.3 Programmatic access
3.1 Technical assessment
3.2 Scientific assesment
3.3 Useful Hints
3.3.1 Technical Aspects
3.3.2 Scientific Aspects
This document is intended as a first overview and quick checklist for PRACE applications. For definitive information, please consult the PRACE web site.
1 Resources
As of 2014, six Tier-0 installations are being operated by 4 countries. Actual availability of these resources varies per call. System types are differentiated by
- Processor type (X86, Power PC)
- Number of cores (16-128) and memory (16-512GB) per node
- Availability and type of accelerators (none, Nvidia, Xeon Phi)
- Type (Infiniband, proprietary) and topology (Fat tree, fat tree with islands, 3D-5D Torus) of network
Overview (table headers link to installation web sites)
Curie | Fermi | Hornet | MareNostrum | SuperMUC | ||||||
---|---|---|---|---|---|---|---|---|---|---|
FN | TN | HN | Fermi | Hornet | Standard | Hybrid | TN | FN (*) | ||
Country | France | France | France | Italy | Germany | Spain | Spain | Germany | Germany | |
System Type | Bullx | Bullx | Bullx | Blue Gene/Q | Cray XC30 | IBM System x iDataPlex | IBM System x iDataPlex | IBM System x iDataPlex | IBM BladeCenter HX5 | |
Compute | Processor type | Intel Nehalem EX 2,27 Ghz | Intel SandyBridg e EP 2,7 Ghz | Intel Westmere EP 2,67 Ghz | IBM PowerPC A2 (1,6 GHz) 16 cores/node | Intel Xeon E5-2680v3 (Haswell) | Intel Sandy Bridge EP | Intel Sandy Bridge EP | Intel Sandy Bridge EP | Intel Westmere EX |
Total number of nodes | 90 | 5040 | 144 | 10240 | 3944 | 3028 | 42 | 9216 | 205 | |
Number of cores per node | 128 | 16 | 8 | 16 | 24 | 16 | 16 | 16 | 40 | |
Total number of cores | 11520 | 80640 | 1152 | 163840 | 94656 | 48448 | 672 | 147456 | 8200 | |
Number of accelerators / node | - | - | 2 | - | - | - | 2 | - | - | |
Type of accelerator | - | - | Nvidia M2090 | - | - | - | Intel Xeon Phi 5110P | - | - | |
Memory | Memory / Node | 512 GB | 64 GB | 24 GB | 16 GB | 128GB | 32 | 64 GB | 32 GB | 256 GB |
Network | Network Type | Infiniband QDR 10 | Infiniband QDR 10 | Infiniband QDR 10 | IBM Custom | Cray Aries | Infiniband FDR10 | Infiniband FDR10 | Infiniband FDR10 | Infiniband QDR |
Connectivity | Fat tree | Fat tree | Fat tree | 5D Torus | Dragonfly | Fat Tree | Fat Tree | Fat tree within island (8192 cores), pruned tree between islands | Fat Tree | |
Home file system | type | NFS | NFS | NFS | GPFS | NFS | GPFS | GPFS | NAS | NAS |
capacity | 8 TB | 8 TB | 8 TB | 100 TB | 60 TB | 59 TB | 59 TB | 1,5 PB | 1,5 PB | |
Work file system | type | Lustre | Lustre | Lustre | GPFS | Lustre | GPFS | GPFS | GPFS | GPFS |
capacity | 600 TB | 600 TB | 600 TB | 2PB | 7 PB | 612 TB | 612 TB | 7 PB | 7 PB | |
Scratch file system | type | Lustre | Lustre | Lustre | GPFS | - | GPFS | GPFS | GPFS | GPFS |
capacity | 3,4 PB | 3,4 PB | 3,4 PB | 1 PB | - | 1,1 PB | 1,1 PB | 3 PB | 3 PB | |
Archive | capacity | Unlimited | Unlimited | Unlimited | On demand | On demand | 2,4 PB | 2,4 PB | 30 PB On demand | 30 PB On demand |
Minimum required job size | Number of cores | 128 | 512 | 32 | 2048 | 2048 | 1024 | 16 | 512 | 512 |
(*) Only for pre-/post-processing purposes. Please verify with Gauss@LRZ for large computing resource requests on FN.
2 Types of Access
PRACE resources can be requested for
- Preparatory Access (code scaling, optimization),
- Project Access (large-scale, computationally intensive projects), and
- Programmatic Access.
2.1 Preparatory Access
Purpose: prepare code for production runs (development, optimization, scaling)
Time window: Rolling call, no closing date, quarterly cut-off dates.
Maximum CPU allocation: typically 50000 - 250000 core hours depending on type of access and system.
- Type A: code scalability tests - max 2 months
- Type B: code development and optimization - max 6 months
- Type C: code development and optimization with support by PRACE experts - max 6 months
2.1.1 Requirements
- Detailed Work plan mandatory
- No Production Runs
- Aim is good scaling of code (min 1024c)
- Avoid job farming; if jobs farming is necessary: provide job bundling
- Statement about I/O (parallel I/O?)
- Contact the ZID before submission
2.1.2 Detailed information
2.2 Project Access
Purpose: large scale scientific production runs using codes that have been tested and demonstrate high efficiency and scalability.
It is strongly recommended to start with preparatory access before applying for project access.
Time window: semi-annual calls for proposals (February, September) open for one month, access starts 6 months later
Time allocation: typically more than 3 million core-hours; total available capacity ranging from 200000 (hybrid systems) or 70 to 200 million (conventional systems) core-hours per installation and call.
- Duration max 1 year; multi-year projects are possible if need for resource allocation is demonstrated.
- Preparatory access recommended but not mandatory if scalability and efficiency has been previously demonstrated.
2.2.1 Requirements
In addition to filling in the request form, you must supply the following information:
- Description of research project (including discussion of scientific scientific questions and goals, novelty, impact and timeliness of proposal.
- Your experience using HPC systems, detailed Work plan (how resources and staff are managed)
- Numerical methods and algorithms
- Need for Tier-0 system
- Suitability and fitness of desired machine
- Justification for necessary number of CPU-hours.
2.2.2 Detailed information
- Call Announcements
- Project Access
- Application Procedure
- Technical Guidelines
- Terms of Reference
- Application Guide
2.3 Programmatic access
Purpose: stable and reliable access for particularly large-scale long-term projects of exceptional scientific quality and broad European importance.
3 Assessment of projects
Each scientific project application undergoes two phases of assessment (preparatory access will only be technically assessed).
3.1 Technical assessment
Applicants are expected to select suitable resources based on the characteristics of their code. For regular access, the following criteria must be met:
- The need to use a PRACE resource
- Software availability, demonstration of efficiency, scaling and suitability (pre-installed software, tests of software developed by applicant)
- Feasibility, suitability of project for selected system.
3.2 Scientific assesment
Proposals must address the following criteria:
- Scientific excellence, impact of proposed research
- Novelty, transformative qualities (development of important and relevant scientific topic, expected advances)
- Relevance to call (if scope is stated in the call)
- Methodology (description and appropriateness of methods to achieve project goals)
- Dissemination (planned channels and resources for dissemination of results, list of recent publications relevant to proposed project)
- Management (demonstration of solid management structure, ensuring successful project completion)
3.3 Useful Hints
3.3.1 Technical Aspects
- Target large scaling (at least 8000 cores)
- I/O Strategy has to be convincing (not millions of files)
- Provide restart or snapshot capabilities
- Max. Runtime is 48 h
- Max #jobs is 10
- Be sure to show that you need max 1.6 GB/c
- LRZ: min 3 million cpuh, max 170 million cpuh
3.3.2 Scientific Aspects
Scientific reviewers will asses your project using the Reviewers peer review form. Keep this in mind when preparing your application.
- Include description of objects under study (e.g. materials, problem size, parameters...) and detailed motivation for planned investigations.
- Include your previous publications on the same topic.
- Describe dissemination of results, in particular if external funding might interfere with free publication.
4 Allocation
Resources will be allocated to successful projects and have to be used within the given time frame. Application for extensions or multi-year projects are possible.