The Sequencer Open-Source Software
QuickStart
Are you looking for an open-source solution that can:
- ping various devices in parallel with a straightforward
summary output of who is alive and who is not:
$ sequencer ping host[1-100,150-200] rack[1-5]
switch[2,4,5] anydevice[4-5]
- power off a whole set of devices in parallel:
$ sequencer poweroff rack[1-5] switch[2,4,5] anydevice[4-5]
- power on your whole cluster, taking dependencies into
account (e.g: power on a switch before its connected devices):
$ sequencer poweron-dep ALL
- generate an Emergency PowerOff (EPO) script for your
data-center/cluster, that takes into account various dependencies
(power off nodes before switches, clients before server), deals with
failures kindly and still be very efficient (>4000 nodes in less
than 10 minutes):
- Compute dependencies:
$ sequencer depmake epo > epo.dep
- Check dependencies:
$ vi epo.dep
- Generate the epo script:
$ cat epo.dep | sequencer seqmake > epo.seq
- Check the script:
$ vi epo.seq
- Execute the script (when required!):
$ cat epo.seq | sequencer seqexec
If this is your case, you can continue with
an
overview of
the sequencer or jump
directly to
the
tutorial
where you will learn how the "sequencer can ping the world".
The sequencer is an engine used by system administrators for
the computation and execution of sequences of actions over
hardware or software.
As an example, consider using the sequencer for
starting/stopping a whole set of hardware (or software) components
such as a whole group of racks including connecting switches. The
sequencer is:
- Smart: the order in which poweroff commands will
be executed comes from a dependency graph computed from customizable
dependency rules and the given rack names. For example, switches
should be powered off after the devices they are connected
to. Otherwise, those connected devices won't be able to get powered
off because they will become unreachable.
- Predictable: the order in which poweroff commands
will be executed can be verified, modified and recorded before being
run thanks to the incremental mode.
- Easy to use: as shown by the following examples:
- blackbox mode: sequencer poweroff rack[1-4]
- incremental mode: sequencer seqexec -f /root/prod/epo.seq
- Efficient: the sequencer can execute independent
instructions in parallel with a customizable upper limit. It has
powered on/off softly the whole petaflopic
cluster Tera-100 (more than 4000 nodes, ranked #6 in
the TOP500 list of
november 2010) in less than 10 minutes.
- Robust: when a command does not complete
correctly, the sequencer does not execute dependent instructions. It
continues however to execute independent ones. For example, if a
connected component does not poweroff successfull, the connecting
switch should not get powered off whereas other switches may.
Main Usage
The sequencer mostly targets management of clusters and
data-centers for:
- power on/off: computers, disk arrays, switches, talim/pdu,
cold doors, and any other manageable devices;
- power on/off groups of such devices such as racks or logical groups;
- stopping software servers such as NFS or lustre when
specific actions should be executed on clients before
the server can be stopped;
- launching actions in parallel locally or remotely (ssh);
- executing simple workflows (the sequencer computes an
instruction sequence, a workflow, from dependency rules);
- more generally executing an instruction set in a
pre-defined, computable order.
Main Features
The sequencer provides the following main features:
- Efficient execution of local and remote actions (ssh)
in parallel
using ClusterShell;
- Customization of dependency rules through scripts: no
programming knowledge required;
- Two execution modes: blackbox
and incremental for two distinct usage targets: daily and
emergency.
- Robust and flexible failure handling: instructions
depending on a failed one are not executed by default unless told
otherwise.
- Provides various reports type to ease maintenance and
profiling;
- UNIX philosophy: stdout, stderr and returned code are
dealt with the UNIX way.
- Determinist: the order in which dependent instructions
are executed is determinist (independent instructions might be
executed in parallel though).
Download & Installation
Downloading by hand is not normally required. The standard
Python installation process will do everything for you, in
particular, dependencies will be
taken into account. See
the wiki
for details.
Nevertheless, the following format are still provided:
You can also clone the project
with
Git by running:
$ git clone git://github.com/pv-bull/sequencer
Documentation
The official documentation is on
the wiki.
Dependencies/Requirements
The sequencer depends on the following open-source products:
- Python >= 2.6,
- ClusterShell >= 1.5,
- python-graph >= 1.7.0,
- pydot,
- python-lxml >= 2.2.3,
- graphviz >= 2.26,
License
GPL v3
Mailing Lists
Post here for any discussion about the
sequencer: sequencer@freelists.org
Instant Messaging (IRC)
Either connect with your IRC client to freenode IRC
(irc.freenode.net) and join channel #sequencer or use
the web IRC
client.
Professional Support & Services
Professional support and services for the sequencer can be
purchased. Contact:
Pierre Vignéras (pierre.vigneras@bull.net)
Authors
This software has been designed and implemented
by Pierre
Vignéras.