forked from cea-hpc/clustershell
-
Notifications
You must be signed in to change notification settings - Fork 0
Scalable cluster administration Python framework — Manage node sets, node groups and execute commands on cluster nodes in parallel.
hdoreau/clustershell
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
----------------------------------------- ClusterShell 1.6 Python Library and Tools ----------------------------------------- ClusterShell is an event-driven open source Python library, designed to run local or distant commands in parallel on server farms or on large Linux clusters. It will take care of common issues encountered on HPC clusters, such as operating on groups of nodes, running distributed commands using optimized execution algorithms, as well as gathering results and merging identical outputs, or retrieving return codes. ClusterShell takes advantage of existing remote shell facilities already installed on your systems, like SSH. ClusterShell's primary goal is to improve the administration of high- performance clusters by providing a lightweight but scalable Python API for developers. It also provides clush, clubak and nodeset, three convenient command-line tools that allow traditional shell scripts to benefit from some of the library features. ------------------- Requirements (v1.6) ------------------- * GNU/Linux, *BSD, Mac OS X, etc. * OpenSSH (ssh/scp) * Python 2.x (x >= 4) ------- License ------- ClusterShell is distributed under the CeCILL-C license, a French transposition of the GNU LGPL, and is fully LGPL-compatible (see Licence_CeCILL-C_V1-en.txt). ------------ Installation ------------ When possible, please use the RPM/deb package distribution: https://github.com/cea-hpc/clustershell/wiki/GetClusterShell Otherwise in the source directory, use: # python setup.py install # cp -r conf /etc/clustershell For installation on Mac OS X, please see: https://github.com/cea-hpc/clustershell/wiki/ClusterShellOnMacOSX ---------- Test Suite ---------- Regression testing scripts are available in the 'tests' directory: $ cd tests $ nosetests -sv <Test.py> $ nosetests -sv --all-modules You have to allow 'ssh localhost' with no warning for "remote" tests to run. -------------- Documentation ------------- Local API documentation is available, just type: $ pydoc ClusterShell Online API documentation (epydoc) is available here: http://packages.python.org/ClusterShell/ -------------------------- ClusterShell interactively -------------------------- Python 2.7.1 (r271:86832, Apr 12 2011, 16:15:16) [GCC 4.6.0 20110331 (Red Hat 4.6.0-2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from ClusterShell.Task import task_self >>> from ClusterShell.NodeSet import NodeSet >>> task = task_self() >>> task.run("/bin/uname -r", nodes="linux[4-6,32-39]") <ClusterShell.Worker.Ssh.WorkerSsh object at 0x20a5e90> >>> for buf, key in task.iter_buffers(): ... print NodeSet.fromlist(key), buf ... linux[32-39] 2.6.40.6-0.fc15.x86_64 linux[4-6] 2.6.32-71.el6.x86_64 ------------------ ClusterShell Tools ------------------ Powerful tools are provided: clush, nodeset and clubak. * clush is a friendly and full-featured parallel shell (see: man clush). If in doubt, just check if your other parallel tools can do things like: # tar -czf - dir | clush -w node[10-44] tar -C /tmp -xzvf - * nodeset is used to deal with your cluster nodeset, it can be bound to external groups (see: man nodeset and man groups.conf). * clubak is a tool used to format output from clush/pdsh-like output (already included in clush with -b), see man clubak. ----- Links ----- Main web site: http://clustershell.sourceforge.net or http://cea-hpc.github.com/clustershell/ Github source respository: https://github.com/cea-hpc/clustershell Github Wiki: https://github.com/cea-hpc/clustershell/wiki Github Issue tracking system: https://github.com/cea-hpc/clustershell/issues Sourceforge.net project page: http://sourceforge.net/projects/clustershell Python Package Index (PyPI) link: http://pypi.python.org/pypi/ClusterShell ClusterShell was born along with Shine, a scalable Lustre FS admin tool: http://lustre-shine.sourceforge.net ------- Authors ------- Stephane Thiell <[email protected]> Aurelien Degremont <[email protected]> Henri Doreau <[email protected]> CEA/DAM 2010, 2011, 2012 - http://www-hpc.cea.fr
About
Scalable cluster administration Python framework — Manage node sets, node groups and execute commands on cluster nodes in parallel.
Resources
Stars
Watchers
Forks
Packages 0
No packages published
Languages
- Python 99.0%
- Other 1.0%