Web based computational steering system

Научная статья

Информатика, кибернетика и программирование

Secondly person cn shorten the debugging time of prllel progrm becuse person is ble to drive the computtion process djusting the lgorithm prmeters or progrm execution pth ccording to his needs. t the moment we hve developed smll nd cler PI nd dt server for prllel progrm developers tht llows progrm to be visulized online. Computtion nodes re prllel progrm processes with some embedded PI clls which connects them to the steering system. Visuliztion frontends re set of worksttion progrms which visulize the tsk stte nd give the bility to...



52.5 KB

0 чел.

Web based computational steering system

Pavel Vasev

16, S.Kovalevskaja street, 620219, Ekaterinburg GSP-384, Russia


This paper presents the insight of web-based steering system for high performance computing. It is a lightweight solution for scientists and allows them to perform application steering with minimal technical efforts.  


HPC, visualization, collaboration, steering, web.


Visualization is a superior tool for analysis of computational experiment in scientific and engineering research. As computer evolution goes on there are various technical approaches that emerge to support visualization. We will consider two of them: application steering and remote visualization.

Application steering (also called online visualization) is a tool to visualize and manipulate the current state of a running HPC (high performance computing) application. It provides great benefits. First, person is capable to perform an online simulation, without waiting for the whole computation process to finish. Secondly, person can shorten the debugging time of a parallel program, because a person is able to drive the computation process, adjusting the algorithm parameters or program execution path according to his needs.

Remote visualization is a technique that uses remote rendering resources. Remote visualization addresses situations when it is much more effective to render computation data remotely than to transmit that data to person’s computer, even partially.

To make these great things possible in a HPC environment, a computational steering system should exist. There are many steering systems developed worldwide. There are different technical and philosophical approaches exist (Averbukh et al, 2002). It is important that there are two end-user sides in such systems: HPC application interface and graphical interface. First is to provide application’s data to the steering system, and second is to see that data. We develop yet another system that targets simplicity of use for a HPC software developer. At the moment we have developed a small and clear API and a data server for parallel program developers that allows a program to be visualized online. This allows creating steer-enabled computing applications and remote visualization applications.

There is one problem. It is very difficult to develop front-end from scratch for each scientific project. Therefore, a universal front-end framework is required. It should provide the ability to generate front-ends for HPC applications semi-automatically. We are working on it and have found some ideas that form the concept of universal web-based front-end for application steering. We will describe the concept in the following pages. Let’s start by examining the status of steering system being developed.


The idea of the system had evolved from the CUMULVS project (Kohl et al, 2006) developed by Oak Ridge National Laboratory. We implemented their approach with some modifications.

Our steering system consists of three layers: computation nodes, dealer service and visualization front-ends. Computation nodes are parallel program processes with some embedded API calls which connects them to the steering system. The dealer service is the heart of the system, it accepts front-end’s requests and proxies them to computational nodes. Visualization front-ends are a set of workstation programs, which visualize the task state and give the ability to control parallel program to the users.

There are two application interfaces in the steering system. One is for computation nodes, and another for front-ends. API for a parallel program is a C/Fortran interface. To make a parallel program steerable, a developer should insert appropriate API calls and link program with additional library. This API calls describes how data is distributed in application’s memory and provides other functionality.

Front-ends communicate with the dealer using SOAP. As parallel program is launched, front-ends can now connect to the dealer service and ask it for contents of the program’s data or perform other actions. Any SOAP-enabled platform and programming language may be used to create front-ends. We have successfully implemented some front-ends using Borland C++ and Adobe Flex 2.

When all preparations are done, a scientist usually works as follows. He starts a parallel program in a usual way. When it is actually spawned by the cluster scheduler, the scientist starts a front-end. Front-end connects to the parallel program, visualizes its state and allows the scientist to perform control actions.


In practice, we have found some issues:

  1.  Development of the front-end is a challenge for scientists. It is too hard for people to deal with scientific area, mathematics, parallel programming, MPI, Linux command line, GUI programming and the SOAP simultaneously. A special front-end developer often is required.
  2.  The front-end software should be installed and configured. It brings some troubles in the early stages of development, especially while updating all the client seats and reconfiguring them.

To resolve this and other issues, we propose the concept of the universal web-based front-end. Some ideas are based on the other works (for ex, Bajaj and Cutchin, 1999). Other ideas came from practice.

  1.  It is web-based. It allows users to access parallel application, visualize application’s state and control it via a web browser.
  2.  It allows users to collaborate during and after parallel program execution. It includes saving and sharing the snapshots of visualizations, and chat capabilities.
  3.  It is web-configurable. User has the ability to drag’n’drop visualization elements, attach them to program resources, and so on.
  4.  It contains a set of predefined visualization elements of various data representation types. This set is expandable by third-party compounds.
  5.  Visualization elements are SOAP clients themselves, and communicate directly with the dealer service.
  6.  Implementation technology of visualization elements may vary. It can be Flex, Silverlight, ActiveX, Java Applets, depending on the situation.
  7.  In advanced cases it may be useful to implement server-side visualization widgets. These widgets, in turn, may be implemented as another parallel program.

The pilot implementation of this concept has succeeded in several projects (Vasin et al, 2007).

  1.  implementation details

User opens a browser and connects to the web server, which in turn connects to the dealer service and retrieves the list of current parallel programs being executed on a cluster. User specifies the desired program. Web browser loads an HTML page with visualization widgets inside it. Each widget on the page is associated with specified resource of the parallel program. Widgets connect directly to the dealer service via SOAP and request data. User interacts with widgets, which in turn may invoke additional data or control requests to the dealer service.

  1.  benefits

First, person receives the steering capability for his program in a very fast way. After insertion of API calls (to publish program data and controllable variables), that person gets a GUI for his HPC program immediately. Further tuning will allow him to receive a more fine-grained control system.

As stated above, visual representation of the program’s data is configurable. Person may arrange and configure data widgets – for example, to select presentation type. We do not fix visualization techniques. Thus we receive an expandable and flexible system.

Proposed architecture may be considered as a basis for remote rendering framework. Imagine that you created parallel rendering application, for example using the Equalizer framework. While running in conjunction with proposed steering system, your application receives visualization parameters, renders image, publishes it to the dealer service and then image output is delivered to client’s web browser.

  1.  conclusion

We have already tested some features of stated concept. The system was successfully used in several research projects (Vasin et al, 2007). Implementing all the features of stated concept will provide easy configurable, cross-platform steering system, which allows users to start steering their parallel applications in a quick way. As an optimum, to enable steering for parallel application, only two steps are necessary. First, researcher should insert data declaration calls in the source code. Second, he should login to the steering system via web browser and configure visual representation of declared data by mouse.

Web-based architecture allows scientists to steer their applications from any physical location connected to the system. Performing smart and secure connections (for example via VPN) will allow seeing the computational task state from other institutions and cities.

Web environment is handy for collaboration. Implementing collaboration service will allow researchers to communicate during and after program execution, discussing the computational experiment.

Logging and snapshot capabilities allow a researcher to turn back to the intermediate visual states of a computation, even after few weeks. In addition, users may share these visual states with each other in a common repository.

When data traffic becomes huge, we may receive a bottleneck. In this case, the following idea may be used. Visualization widgets should have the ability to refer to remote visualization program, which has fast network connection with computational cluster. This program may be parallel and being executed on the same cluster. Visualization program consumes computational program’s published data and generates new “image” data. Thus, we degrade network traffic between cluster and person’s computer to constant level.

Visualization is only a part of a computation process. A person prepares the data, invokes parallel program start command, and then switches to the browser to interact with the parallel program. We consider that in some cases it is useful to perform all these steps in a single web-based environment. Integration with supercomputer’s scheduling system is a subject to additional research.


Averbukh V. L. et al, 2002. Applying visualization to high-performance computations. In The problems of Nuclear Science and Technology, Vol 2, pp. 40-52.  

Bajaj, C. and Cutchin, S, 1999. Web based collaborative visualization of distributed and parallel simulation. Proceedings   of Parallel Visualization and Graphics Symposium. San Francisco, California, United States, pp. 47 – 54.

Kohl J. A. et al, 2006. Cumulvs: Interacting with High-Performance Scientific Simulations, for Visualization, Steering and Fault Tolerance. In International Journal of High Performance Computing Applications, Vol. 20, No. 2, pp. 255-285.

Vasin V.V. et al, 2007. On parallelization and visualization of ill-conditioned problems using methods of regularization and iteration approximation on computational system MVS-1000. Proceedings of International Scientific Conference “Parallel computational technologies”, Chelyabinsk, Russia, pp. 227-233. 


А также другие работы, которые могут Вас заинтересовать

  Режимы течения представляет собой круглую трубу с встроенной на входе капиллярной трубкой для подачи подкрашенной жидкости. Цель работы Визуальное наблюдение изменения составляющих полного напора потока жидкости в трубе переменного сечения. Общие сведения Работа заключается в экспериментальном построении энергетических графиков пьезометрической и энергетической линий одномерного потока жидкости. Уравнение Даниила Бернулли полученное им в 1738 году представляет собой частный случай всеобщего закона сохранения энергии...
41551. Внутренняя структура инновационных процессов на предприятии 155 KB
  Целевые качественные или количественные изменения в выборе и использовании материалов сырья информации оборудования информации работников и других ресурсов Целевые качественны или количественные изменения в результатах производственно хозяйственной деятельности т. в изделиях услугах технологии ИН структуры предприятия ИН на âвыходеâ предприятия Целевые изменения в производственных обслуживающих и вспомогательных процессах как по качеству так и по количеству а так же по организации и способу ее обеспечения. То есть изменения...
41552. Финансовая система государства 1.87 MB
  Основы теории финансов. Взаимосвязь финансов с другими категориями. Функции финансов. Финансовый механизм. Общее понятие о финансовой системе государства. Сферы и звенья финансовой системы, краткая характеристика финансов субъектов хозяйствования. Место и роль государственных финансов в финансовой системе. Бюджетное устройство, его понятие. Принципы построения бюджетной системы. Основные этапы бюджетного процесса. Роль и функции государственного казначейства
  Оценка потребности в финансировании Исходным пунктом для организации финансирования является расчет потребности в средствах необходимых для реализации проекта. Он производится на основе графика выполнения работ что вопервых обеспечивает привязку ко времени а вовторых позволяет увязать объемы финансирования и направления расходования средств. Итогом работы является таблица в которой потребность в средствах расписывается по месяцам. В этом случае более удобна следующая форма представления расчета: Виды работ Месяцы выполнения...
  Инвестиционный проект это план программа хозяйственного мероприятия или предпринимательского проекта реализация которых требует привлечения инвестиций. Структура затрат на реализацию проекта а также доходов получаемых в результате инвестиций традиционно отлична для каждой из выделенных стадий. Определение целей и класса инвестиций 1. Метод расчета рентабельности инвестиций 2.
  Изменения в организации производства и его материальнотехническом обеспечении. Речь может идти также о качественных или количественных изменениях как с положительными так и с отрицательными социальноэкономическими последствиями. Большинство западных авторов обычно подчеркивают необходимость практической реализации изменения. Это могут быть количественные или качественные изменения которые касаются той или иной сферы деятельности предприятия.
  Среди обширного спектра факторов влияющих на выбор миссии компании есть чрезвычайно важный с точки зрения долгосрочной перспективы безкризисного существования фирмы это фактор адекватности инвестирования выдвинутым критериям. Акцент на исследовании этого вопроса делается по двум основным причинам: определяющей роли которая принадлежит инвестиционному процессу в выборе линии поведения фирмы особенно на долгосрочном горизонте ее роста и инноваций продукта. Принятие решений в этой области...
  Одной из важнейших сфер деятельности любой фирмы являются инвестиционные операции. Оба типа инвестиций имеют большое значение для сохранения жизнеспособности фирмы и ее развития. Их целью является прежде всего создание условий для снижения затрат фирмы за счет замены оборудования обучения персонала или перемещения производственных мощностей в регионы с более выгодными условиями производства; инвестиции в расширение производства. Логика зависимости между типом инвестиций и уровнем их риска очевидна: она определяется степенью...
  Как и практически всякая иная политика она неодинакова в разных странах хотя и подчинена одной и той же цели: стимулированию инновационной активности и развитию научнотехнического потенциала. Место и роль инновационной политики в структуре государственного регулирования экономики определяются особенностями инновационного процесса как объекта управления. В практической направленности инновационной идеи и состоит ее притягательная сила для капиталистических компаний. Так к числу внутренних побудительных мотивов инновационной активности можно...