Web based computational steering system

Научная статья

Информатика, кибернетика и программирование

Secondly person cn shorten the debugging time of prllel progrm becuse person is ble to drive the computtion process djusting the lgorithm prmeters or progrm execution pth ccording to his needs. t the moment we hve developed smll nd cler PI nd dt server for prllel progrm developers tht llows progrm to be visulized online. Computtion nodes re prllel progrm processes with some embedded PI clls which connects them to the steering system. Visuliztion frontends re set of worksttion progrms which visulize the tsk stte nd give the bility to...



52.5 KB

0 чел.

Web based computational steering system

Pavel Vasev

16, S.Kovalevskaja street, 620219, Ekaterinburg GSP-384, Russia


This paper presents the insight of web-based steering system for high performance computing. It is a lightweight solution for scientists and allows them to perform application steering with minimal technical efforts.  


HPC, visualization, collaboration, steering, web.


Visualization is a superior tool for analysis of computational experiment in scientific and engineering research. As computer evolution goes on there are various technical approaches that emerge to support visualization. We will consider two of them: application steering and remote visualization.

Application steering (also called online visualization) is a tool to visualize and manipulate the current state of a running HPC (high performance computing) application. It provides great benefits. First, person is capable to perform an online simulation, without waiting for the whole computation process to finish. Secondly, person can shorten the debugging time of a parallel program, because a person is able to drive the computation process, adjusting the algorithm parameters or program execution path according to his needs.

Remote visualization is a technique that uses remote rendering resources. Remote visualization addresses situations when it is much more effective to render computation data remotely than to transmit that data to person’s computer, even partially.

To make these great things possible in a HPC environment, a computational steering system should exist. There are many steering systems developed worldwide. There are different technical and philosophical approaches exist (Averbukh et al, 2002). It is important that there are two end-user sides in such systems: HPC application interface and graphical interface. First is to provide application’s data to the steering system, and second is to see that data. We develop yet another system that targets simplicity of use for a HPC software developer. At the moment we have developed a small and clear API and a data server for parallel program developers that allows a program to be visualized online. This allows creating steer-enabled computing applications and remote visualization applications.

There is one problem. It is very difficult to develop front-end from scratch for each scientific project. Therefore, a universal front-end framework is required. It should provide the ability to generate front-ends for HPC applications semi-automatically. We are working on it and have found some ideas that form the concept of universal web-based front-end for application steering. We will describe the concept in the following pages. Let’s start by examining the status of steering system being developed.


The idea of the system had evolved from the CUMULVS project (Kohl et al, 2006) developed by Oak Ridge National Laboratory. We implemented their approach with some modifications.

Our steering system consists of three layers: computation nodes, dealer service and visualization front-ends. Computation nodes are parallel program processes with some embedded API calls which connects them to the steering system. The dealer service is the heart of the system, it accepts front-end’s requests and proxies them to computational nodes. Visualization front-ends are a set of workstation programs, which visualize the task state and give the ability to control parallel program to the users.

There are two application interfaces in the steering system. One is for computation nodes, and another for front-ends. API for a parallel program is a C/Fortran interface. To make a parallel program steerable, a developer should insert appropriate API calls and link program with additional library. This API calls describes how data is distributed in application’s memory and provides other functionality.

Front-ends communicate with the dealer using SOAP. As parallel program is launched, front-ends can now connect to the dealer service and ask it for contents of the program’s data or perform other actions. Any SOAP-enabled platform and programming language may be used to create front-ends. We have successfully implemented some front-ends using Borland C++ and Adobe Flex 2.

When all preparations are done, a scientist usually works as follows. He starts a parallel program in a usual way. When it is actually spawned by the cluster scheduler, the scientist starts a front-end. Front-end connects to the parallel program, visualizes its state and allows the scientist to perform control actions.


In practice, we have found some issues:

  1.  Development of the front-end is a challenge for scientists. It is too hard for people to deal with scientific area, mathematics, parallel programming, MPI, Linux command line, GUI programming and the SOAP simultaneously. A special front-end developer often is required.
  2.  The front-end software should be installed and configured. It brings some troubles in the early stages of development, especially while updating all the client seats and reconfiguring them.

To resolve this and other issues, we propose the concept of the universal web-based front-end. Some ideas are based on the other works (for ex, Bajaj and Cutchin, 1999). Other ideas came from practice.

  1.  It is web-based. It allows users to access parallel application, visualize application’s state and control it via a web browser.
  2.  It allows users to collaborate during and after parallel program execution. It includes saving and sharing the snapshots of visualizations, and chat capabilities.
  3.  It is web-configurable. User has the ability to drag’n’drop visualization elements, attach them to program resources, and so on.
  4.  It contains a set of predefined visualization elements of various data representation types. This set is expandable by third-party compounds.
  5.  Visualization elements are SOAP clients themselves, and communicate directly with the dealer service.
  6.  Implementation technology of visualization elements may vary. It can be Flex, Silverlight, ActiveX, Java Applets, depending on the situation.
  7.  In advanced cases it may be useful to implement server-side visualization widgets. These widgets, in turn, may be implemented as another parallel program.

The pilot implementation of this concept has succeeded in several projects (Vasin et al, 2007).

  1.  implementation details

User opens a browser and connects to the web server, which in turn connects to the dealer service and retrieves the list of current parallel programs being executed on a cluster. User specifies the desired program. Web browser loads an HTML page with visualization widgets inside it. Each widget on the page is associated with specified resource of the parallel program. Widgets connect directly to the dealer service via SOAP and request data. User interacts with widgets, which in turn may invoke additional data or control requests to the dealer service.

  1.  benefits

First, person receives the steering capability for his program in a very fast way. After insertion of API calls (to publish program data and controllable variables), that person gets a GUI for his HPC program immediately. Further tuning will allow him to receive a more fine-grained control system.

As stated above, visual representation of the program’s data is configurable. Person may arrange and configure data widgets – for example, to select presentation type. We do not fix visualization techniques. Thus we receive an expandable and flexible system.

Proposed architecture may be considered as a basis for remote rendering framework. Imagine that you created parallel rendering application, for example using the Equalizer framework. While running in conjunction with proposed steering system, your application receives visualization parameters, renders image, publishes it to the dealer service and then image output is delivered to client’s web browser.

  1.  conclusion

We have already tested some features of stated concept. The system was successfully used in several research projects (Vasin et al, 2007). Implementing all the features of stated concept will provide easy configurable, cross-platform steering system, which allows users to start steering their parallel applications in a quick way. As an optimum, to enable steering for parallel application, only two steps are necessary. First, researcher should insert data declaration calls in the source code. Second, he should login to the steering system via web browser and configure visual representation of declared data by mouse.

Web-based architecture allows scientists to steer their applications from any physical location connected to the system. Performing smart and secure connections (for example via VPN) will allow seeing the computational task state from other institutions and cities.

Web environment is handy for collaboration. Implementing collaboration service will allow researchers to communicate during and after program execution, discussing the computational experiment.

Logging and snapshot capabilities allow a researcher to turn back to the intermediate visual states of a computation, even after few weeks. In addition, users may share these visual states with each other in a common repository.

When data traffic becomes huge, we may receive a bottleneck. In this case, the following idea may be used. Visualization widgets should have the ability to refer to remote visualization program, which has fast network connection with computational cluster. This program may be parallel and being executed on the same cluster. Visualization program consumes computational program’s published data and generates new “image” data. Thus, we degrade network traffic between cluster and person’s computer to constant level.

Visualization is only a part of a computation process. A person prepares the data, invokes parallel program start command, and then switches to the browser to interact with the parallel program. We consider that in some cases it is useful to perform all these steps in a single web-based environment. Integration with supercomputer’s scheduling system is a subject to additional research.


Averbukh V. L. et al, 2002. Applying visualization to high-performance computations. In The problems of Nuclear Science and Technology, Vol 2, pp. 40-52.  

Bajaj, C. and Cutchin, S, 1999. Web based collaborative visualization of distributed and parallel simulation. Proceedings   of Parallel Visualization and Graphics Symposium. San Francisco, California, United States, pp. 47 – 54.

Kohl J. A. et al, 2006. Cumulvs: Interacting with High-Performance Scientific Simulations, for Visualization, Steering and Fault Tolerance. In International Journal of High Performance Computing Applications, Vol. 20, No. 2, pp. 255-285.

Vasin V.V. et al, 2007. On parallelization and visualization of ill-conditioned problems using methods of regularization and iteration approximation on computational system MVS-1000. Proceedings of International Scientific Conference “Parallel computational technologies”, Chelyabinsk, Russia, pp. 227-233. 


А также другие работы, которые могут Вас заинтересовать

37106. Послевоенное устройство мира. Формирование двух мировых систем и их противостояние 18.75 KB
  Особенностью послевоенного периода стали народнодемократические социалистические революции в странах Восточной Европы и ряде стран Азии которые при поддержке СССР приступили к строительству социализма. Образовалась мировая система социализма во главе с СССР. В годы мировой войны сложилась единая антифашистская коалиция союз великих держав СССР США Великобритания и Франция. Основой конфронтации стали отношения между двумя сверхдержавами СССР и США.
37107. СССР в период восстановления разрушенного войной хозяйства 23.22 KB
  В 1943 году для организации восстановительных работ в Совете Народных Комиссаров СССР был образован специальный комитет а в Госплане СССР управление обеспечивающее планирование деятельности всех ведомств причастных к этой работе. Постановление ЦК ВКП б и СНК СССР от 21 августа 1943 г. В Госплане СССР и научных учреждениях были разработаны краткосрочные и долгосрочные планы восстановления народного хозяйства как в целом так и по отдельным отраслям. На сессии Верховного Совета СССР был утвержден пятилетний план восстановления и развития...
37108. Разоблачение культа личности Сталина. XX съезд КПСС 15.81 KB
  Именно Хрущеву было поручено прочитать доклад и лично встретить непредсказуемую реакцию участников съезда. Созванный за восемь месяцев до уставного срока в связи с насущной необходимостью подвести итоги произошедших после смерти Сталина изменений и дискуссии о выборе курса съезд завершился знаменитым секретным докладом Хрущева. в последний день работы XX съезда на закрытом заседании с докладом О культе личности и его последствиях выступил первый секретарь ЦК КПСС Н. В докладе были обнародованы и осуждены факты массовых репрессий...
37109. Политика либерализации советского общества. «Оттепель». Н.С. Хрущев – политик и человек 18.45 KB
  Хрущев политик и человек. Хрущев. После недолгой борьбы к власти пришел Никита Сергеевич Хрущев. Резкий решительный неосторожный в словах и поступках Хрущев прошел все ступени партийной работы возглавлял крупные парторганизации.
37110. Нарастание застойных явлений в жизни СССР в 70-е – начале 80-х гг. и их сущность 27.5 KB
  Сохранилась жёстко централизованная система власти. Костяк этой власти составили аппаратчики. К власти пришло третье поколение советских правителей типичных аппаратчиков исполнителей. В моменты ослабления центрального руководства незаметные исполнители почувствовали мощь власти и те блага которые можно приобрести с её помощью.
37111. Перестройка политической системы СССР 24.95 KB
  В истории СССР начался этап получивший название перестройка . По решению XIX партконференции учреждается новый высший орган законодательной власти Съезд народных депутатов СССР и соответствующие республиканские съезды. Был введен пост Президента СССР первым и последним Президентом СССР стал М.
37112. Россия в СНГ и мире: проблема взаимоотношений. Россия на рубеже XX-XXI вв 16.75 KB
  После распада СССР и провозглашения Содружества Независимых Государств для России сложилась новая внешнеполитическая ситуация. Обустройство этих войск в новых местах дислокации обернулось для России значительными финансовыми затратами. 30 мая 1997 года президентами России и республики Беларусь был подписан Устав нового союза однако противников объединения этих двух славянских государств остается немало как с одной так и с другой стороны. В отношении дальнего зарубежья внешняя политика России ставшей правопреемницей СССР характеризуется...
37113. Социально-экономическое развитие суверенной России: переход к рынку, приватизация 17.41 KB
  Приватизация. Осенью 1992 года началась приватизация. Решению этой задачи была подчинена и объявленная приватизация.
37114. Политические преобразования в РФ в 90-е годы. Октябрьский кризис 1993 г. Принятие Конституции, переход к парламентаризму 16.59 KB
  Принятие Конституции переход к парламентаризму. После октябрьских событий началась ликвидация системы Советов завершившаяся с принятием на всенародном референдуме 12 декабря 1993 новой Конституции Российской Федерации 3в 19901993 гг. Ельциным которая впервые занялась разработкой новой Конституции. Проект подготовленный этой комиссией не был принят ни на первом ни на втором обсуждении на съездах народных депутатов но одобрялась общая концепция Конституции что значительно затягивало переход к новому конституционному строю в...