q pipeline manager download

q pipeline manager

q pipeline manager Description
The q utility is a platform for creating and managing data analysis pipelines. It expands the value of your existing job scheduler - either Grid Engine or TORQUE PBS - through numerous functions that help you organize, submit, monitor, manage and share your informatics work.

Data processing pipelines require high-level organization and parallelization of work to optimize resource utilization and decrease the time to results. q (from queue) allows complex job sequences to be efficiently assembled and managed, including dependency tracking, parallelization, and pipeline-level monitoring, error recovery, and data protection. Pipelines are constructed from modular script files, with job definitions and results stored in easily retrieved job files. A web interface facilitates job submission and monitoring, with the complete pipeline exportable for full transparency.
q pipeline manager Features

Organization of work into higher order units from existing code pieces in any language.

Pipelines are submitted and managed as a unit by calls to a single master file.

Robust definition and tracking of job dependencies.

Multiple modes of parallelization maximize resource utilization and minimize time to results.

Admixture of submission-time and execution-time commands supports dynamic job configuration.

Intelligent tracking of queued jobs supports incremental execution of new work.

A few q-specific syntax elements create complex pipelines with minimal code.

Highly modular structure facilities code reuse and sharing.

Various features facilitate cross-platform implementation of pipelines.

Extensive submission-time syntax checking helps prevent fractured pip elines.

Enhanced job reporting and interfaces into job scheduler functions, including error handling.

Write-protection and backup of output files integrated into pipeline definition files.

History tracking of all work performed in a pipeline, with rollback.

Web-based interface into job submission and management.

Fully transparent sharing of work at publication via comprehensive HTML reports.

End-user installation with no required management by system administrators.

Download:Click Here