Web based computational steering system

Научная статья

Информатика, кибернетика и программирование

Secondly person cn shorten the debugging time of prllel progrm becuse person is ble to drive the computtion process djusting the lgorithm prmeters or progrm execution pth ccording to his needs. t the moment we hve developed smll nd cler PI nd dt server for prllel progrm developers tht llows progrm to be visulized online. Computtion nodes re prllel progrm processes with some embedded PI clls which connects them to the steering system. Visuliztion frontends re set of worksttion progrms which visulize the tsk stte nd give the bility to...



52.5 KB

0 чел.

Web based computational steering system

Pavel Vasev

16, S.Kovalevskaja street, 620219, Ekaterinburg GSP-384, Russia


This paper presents the insight of web-based steering system for high performance computing. It is a lightweight solution for scientists and allows them to perform application steering with minimal technical efforts.  


HPC, visualization, collaboration, steering, web.


Visualization is a superior tool for analysis of computational experiment in scientific and engineering research. As computer evolution goes on there are various technical approaches that emerge to support visualization. We will consider two of them: application steering and remote visualization.

Application steering (also called online visualization) is a tool to visualize and manipulate the current state of a running HPC (high performance computing) application. It provides great benefits. First, person is capable to perform an online simulation, without waiting for the whole computation process to finish. Secondly, person can shorten the debugging time of a parallel program, because a person is able to drive the computation process, adjusting the algorithm parameters or program execution path according to his needs.

Remote visualization is a technique that uses remote rendering resources. Remote visualization addresses situations when it is much more effective to render computation data remotely than to transmit that data to person’s computer, even partially.

To make these great things possible in a HPC environment, a computational steering system should exist. There are many steering systems developed worldwide. There are different technical and philosophical approaches exist (Averbukh et al, 2002). It is important that there are two end-user sides in such systems: HPC application interface and graphical interface. First is to provide application’s data to the steering system, and second is to see that data. We develop yet another system that targets simplicity of use for a HPC software developer. At the moment we have developed a small and clear API and a data server for parallel program developers that allows a program to be visualized online. This allows creating steer-enabled computing applications and remote visualization applications.

There is one problem. It is very difficult to develop front-end from scratch for each scientific project. Therefore, a universal front-end framework is required. It should provide the ability to generate front-ends for HPC applications semi-automatically. We are working on it and have found some ideas that form the concept of universal web-based front-end for application steering. We will describe the concept in the following pages. Let’s start by examining the status of steering system being developed.


The idea of the system had evolved from the CUMULVS project (Kohl et al, 2006) developed by Oak Ridge National Laboratory. We implemented their approach with some modifications.

Our steering system consists of three layers: computation nodes, dealer service and visualization front-ends. Computation nodes are parallel program processes with some embedded API calls which connects them to the steering system. The dealer service is the heart of the system, it accepts front-end’s requests and proxies them to computational nodes. Visualization front-ends are a set of workstation programs, which visualize the task state and give the ability to control parallel program to the users.

There are two application interfaces in the steering system. One is for computation nodes, and another for front-ends. API for a parallel program is a C/Fortran interface. To make a parallel program steerable, a developer should insert appropriate API calls and link program with additional library. This API calls describes how data is distributed in application’s memory and provides other functionality.

Front-ends communicate with the dealer using SOAP. As parallel program is launched, front-ends can now connect to the dealer service and ask it for contents of the program’s data or perform other actions. Any SOAP-enabled platform and programming language may be used to create front-ends. We have successfully implemented some front-ends using Borland C++ and Adobe Flex 2.

When all preparations are done, a scientist usually works as follows. He starts a parallel program in a usual way. When it is actually spawned by the cluster scheduler, the scientist starts a front-end. Front-end connects to the parallel program, visualizes its state and allows the scientist to perform control actions.


In practice, we have found some issues:

  1.  Development of the front-end is a challenge for scientists. It is too hard for people to deal with scientific area, mathematics, parallel programming, MPI, Linux command line, GUI programming and the SOAP simultaneously. A special front-end developer often is required.
  2.  The front-end software should be installed and configured. It brings some troubles in the early stages of development, especially while updating all the client seats and reconfiguring them.

To resolve this and other issues, we propose the concept of the universal web-based front-end. Some ideas are based on the other works (for ex, Bajaj and Cutchin, 1999). Other ideas came from practice.

  1.  It is web-based. It allows users to access parallel application, visualize application’s state and control it via a web browser.
  2.  It allows users to collaborate during and after parallel program execution. It includes saving and sharing the snapshots of visualizations, and chat capabilities.
  3.  It is web-configurable. User has the ability to drag’n’drop visualization elements, attach them to program resources, and so on.
  4.  It contains a set of predefined visualization elements of various data representation types. This set is expandable by third-party compounds.
  5.  Visualization elements are SOAP clients themselves, and communicate directly with the dealer service.
  6.  Implementation technology of visualization elements may vary. It can be Flex, Silverlight, ActiveX, Java Applets, depending on the situation.
  7.  In advanced cases it may be useful to implement server-side visualization widgets. These widgets, in turn, may be implemented as another parallel program.

The pilot implementation of this concept has succeeded in several projects (Vasin et al, 2007).

  1.  implementation details

User opens a browser and connects to the web server, which in turn connects to the dealer service and retrieves the list of current parallel programs being executed on a cluster. User specifies the desired program. Web browser loads an HTML page with visualization widgets inside it. Each widget on the page is associated with specified resource of the parallel program. Widgets connect directly to the dealer service via SOAP and request data. User interacts with widgets, which in turn may invoke additional data or control requests to the dealer service.

  1.  benefits

First, person receives the steering capability for his program in a very fast way. After insertion of API calls (to publish program data and controllable variables), that person gets a GUI for his HPC program immediately. Further tuning will allow him to receive a more fine-grained control system.

As stated above, visual representation of the program’s data is configurable. Person may arrange and configure data widgets – for example, to select presentation type. We do not fix visualization techniques. Thus we receive an expandable and flexible system.

Proposed architecture may be considered as a basis for remote rendering framework. Imagine that you created parallel rendering application, for example using the Equalizer framework. While running in conjunction with proposed steering system, your application receives visualization parameters, renders image, publishes it to the dealer service and then image output is delivered to client’s web browser.

  1.  conclusion

We have already tested some features of stated concept. The system was successfully used in several research projects (Vasin et al, 2007). Implementing all the features of stated concept will provide easy configurable, cross-platform steering system, which allows users to start steering their parallel applications in a quick way. As an optimum, to enable steering for parallel application, only two steps are necessary. First, researcher should insert data declaration calls in the source code. Second, he should login to the steering system via web browser and configure visual representation of declared data by mouse.

Web-based architecture allows scientists to steer their applications from any physical location connected to the system. Performing smart and secure connections (for example via VPN) will allow seeing the computational task state from other institutions and cities.

Web environment is handy for collaboration. Implementing collaboration service will allow researchers to communicate during and after program execution, discussing the computational experiment.

Logging and snapshot capabilities allow a researcher to turn back to the intermediate visual states of a computation, even after few weeks. In addition, users may share these visual states with each other in a common repository.

When data traffic becomes huge, we may receive a bottleneck. In this case, the following idea may be used. Visualization widgets should have the ability to refer to remote visualization program, which has fast network connection with computational cluster. This program may be parallel and being executed on the same cluster. Visualization program consumes computational program’s published data and generates new “image” data. Thus, we degrade network traffic between cluster and person’s computer to constant level.

Visualization is only a part of a computation process. A person prepares the data, invokes parallel program start command, and then switches to the browser to interact with the parallel program. We consider that in some cases it is useful to perform all these steps in a single web-based environment. Integration with supercomputer’s scheduling system is a subject to additional research.


Averbukh V. L. et al, 2002. Applying visualization to high-performance computations. In The problems of Nuclear Science and Technology, Vol 2, pp. 40-52.  

Bajaj, C. and Cutchin, S, 1999. Web based collaborative visualization of distributed and parallel simulation. Proceedings   of Parallel Visualization and Graphics Symposium. San Francisco, California, United States, pp. 47 – 54.

Kohl J. A. et al, 2006. Cumulvs: Interacting with High-Performance Scientific Simulations, for Visualization, Steering and Fault Tolerance. In International Journal of High Performance Computing Applications, Vol. 20, No. 2, pp. 255-285.

Vasin V.V. et al, 2007. On parallelization and visualization of ill-conditioned problems using methods of regularization and iteration approximation on computational system MVS-1000. Proceedings of International Scientific Conference “Parallel computational technologies”, Chelyabinsk, Russia, pp. 227-233. 


А также другие работы, которые могут Вас заинтересовать

  ВОСПАЛЕНИЕ Сущность воспаления кардинальные признаки адаптивная роль воспаления виды местные и общие процессы при воспалении причины воспаления механизмы альтерации динамика сосудистой реакции в очаге воспаления механизмы экссудации медиаторы воспаления стадии фагоцитоза значение незавершенного фагоцитоза. ФОРМЫ ВИДЫ ВОСПАЛЕНИЯ Альтеративное В. МЕХАНИЗМЫ ВОСПАЛЕНИЯ: АЛЬТЕРАЦИЯ: пусковой механизм В. Ферменты лизосом ведут к дегрануляции тучных клеток и выходу гистамина важнейший медиатор воспаления ...
  Главное при гипертермии снижение теплоотдачи но и нарушения обмена утилизации энергиитепла катехоламины яды митохондриальные йодсодержащие тиреоидные гормоны. Стадии: Компенсированная развитие стрессреакции активация симпатоадреналовой и гипоталамонадпочечниковой систем усиление теплоотдачи пот гипогидратация и повышение вязкости крови выделение солей; учащение ЧД ЧСС повышение ударного ОК и МОК повышение систАД при снижении диастАД; увеличение утилизации кислорода и увеличение выделения СО2 гипокапния с...
  ФИЛОГЕНЕЗ иммунитет есть на самых ранних стадиях жизни: Все ГКГ АТ всех типов Fcрецепторы клеток CD антигены Тлмф АГрецепторы Тклеток АГрецепторы Вклеток это суперсемейство генов иммуноглобулинов возникших и развивающихся вместе Схема 1 Филогенез функций иммунитета Фагоцитоз пищевойзащитный прообраз иммунитета амеба  Распознавание своечужое ГКГ губки предпочтение своего Отторжение чужого кишечнополостные имеется уже иммунная память и цитотоксичность древние Тлмф и ЕК кораллы ...
24204. Исследование регистров, хранение и преобразование многоразрядных двоичных чисел 90.5 KB
  Наиболее простыми регистрами являются регистры памяти. Схема включения регистра 74173. На отечественных схемах символом регистра служат буквы RG . Работу регистра сдвига рассмотрим на примере регистра 74195 К155ИР12 схема включения которого показана на рис.
  Триггер может служить примером простейшего счетчика. Каждый из триггеров такой цепочки называют разрядом счетчика. Нулевое состояние всех триггеров принимается за нулевое состояние счетчика в целом. Число входных импульсов и состояние счетчика взаимно определены только для первого цикла.
24206. Исследование устройств на операционных усилителях 614.5 KB
  Научиться измерять: входные токи напряжение смещения входное и выходное сопротивления время нарастания выходного напряжения операционных усилителей. ОУ в своём составе имеет входной каскад каскад сдвига уровня напряжения и выходной каскад. Каскад сдвига уровня напряжения выполнен по схеме эмиттерного повторителя и исключает из сигнала уровень постоянной составляющей. Входные токи проходят через внутреннее сопротивление источника входного сигнала и создают на нём падение напряжения.
  По частотным характеристикам различают четыре основных вида фильтров рис. Рис. Частотные характеристики идеальных сплошная кривая и реальных пунктирная фильтров нижних частот а верхних б полосового в и режекторного г Фильтры нижних частот ФНЧ пропускают колебания с частотами от нуля до некоторой верхней частоты в фильтры верхних частот ФВЧ колебания с частотой не ниже некоторой нижней частоты н.
24208. Исследование цифро-аналоговых и аналого-цифровых преобразователей 615 KB
  Опорное напряжение U0n 3 В подключается к резисторам матрицы переключателями D C B и A управляемым одноименными клавишами клавиатуры и имитирующими преобразуемый код. Выходное напряжение U0 измеряется мультиметром.1 то напряжение на входе и выходе ОУ равно 0 В.Тогда на вход ОУ через резистор R1 подается напряжение 3 B.
  Поэтому часто дешифраторы называют дешифраторамидемультиплексорами и наоборот. Схема включения дешифратора 74154. 2 приведена схема включения дешифратора 74154 отечественный аналог К155ИДЗ. В режиме дешифратора с генератора слова на входы Gl G2 подается 0 а на адресные входы код в диапазоне 0000.