Web based computational steering system

Научная статья

Информатика, кибернетика и программирование

Secondly person cn shorten the debugging time of prllel progrm becuse person is ble to drive the computtion process djusting the lgorithm prmeters or progrm execution pth ccording to his needs. t the moment we hve developed smll nd cler PI nd dt server for prllel progrm developers tht llows progrm to be visulized online. Computtion nodes re prllel progrm processes with some embedded PI clls which connects them to the steering system. Visuliztion frontends re set of worksttion progrms which visulize the tsk stte nd give the bility to...



52.5 KB

0 чел.

Web based computational steering system

Pavel Vasev

16, S.Kovalevskaja street, 620219, Ekaterinburg GSP-384, Russia


This paper presents the insight of web-based steering system for high performance computing. It is a lightweight solution for scientists and allows them to perform application steering with minimal technical efforts.  


HPC, visualization, collaboration, steering, web.


Visualization is a superior tool for analysis of computational experiment in scientific and engineering research. As computer evolution goes on there are various technical approaches that emerge to support visualization. We will consider two of them: application steering and remote visualization.

Application steering (also called online visualization) is a tool to visualize and manipulate the current state of a running HPC (high performance computing) application. It provides great benefits. First, person is capable to perform an online simulation, without waiting for the whole computation process to finish. Secondly, person can shorten the debugging time of a parallel program, because a person is able to drive the computation process, adjusting the algorithm parameters or program execution path according to his needs.

Remote visualization is a technique that uses remote rendering resources. Remote visualization addresses situations when it is much more effective to render computation data remotely than to transmit that data to person’s computer, even partially.

To make these great things possible in a HPC environment, a computational steering system should exist. There are many steering systems developed worldwide. There are different technical and philosophical approaches exist (Averbukh et al, 2002). It is important that there are two end-user sides in such systems: HPC application interface and graphical interface. First is to provide application’s data to the steering system, and second is to see that data. We develop yet another system that targets simplicity of use for a HPC software developer. At the moment we have developed a small and clear API and a data server for parallel program developers that allows a program to be visualized online. This allows creating steer-enabled computing applications and remote visualization applications.

There is one problem. It is very difficult to develop front-end from scratch for each scientific project. Therefore, a universal front-end framework is required. It should provide the ability to generate front-ends for HPC applications semi-automatically. We are working on it and have found some ideas that form the concept of universal web-based front-end for application steering. We will describe the concept in the following pages. Let’s start by examining the status of steering system being developed.


The idea of the system had evolved from the CUMULVS project (Kohl et al, 2006) developed by Oak Ridge National Laboratory. We implemented their approach with some modifications.

Our steering system consists of three layers: computation nodes, dealer service and visualization front-ends. Computation nodes are parallel program processes with some embedded API calls which connects them to the steering system. The dealer service is the heart of the system, it accepts front-end’s requests and proxies them to computational nodes. Visualization front-ends are a set of workstation programs, which visualize the task state and give the ability to control parallel program to the users.

There are two application interfaces in the steering system. One is for computation nodes, and another for front-ends. API for a parallel program is a C/Fortran interface. To make a parallel program steerable, a developer should insert appropriate API calls and link program with additional library. This API calls describes how data is distributed in application’s memory and provides other functionality.

Front-ends communicate with the dealer using SOAP. As parallel program is launched, front-ends can now connect to the dealer service and ask it for contents of the program’s data or perform other actions. Any SOAP-enabled platform and programming language may be used to create front-ends. We have successfully implemented some front-ends using Borland C++ and Adobe Flex 2.

When all preparations are done, a scientist usually works as follows. He starts a parallel program in a usual way. When it is actually spawned by the cluster scheduler, the scientist starts a front-end. Front-end connects to the parallel program, visualizes its state and allows the scientist to perform control actions.


In practice, we have found some issues:

  1.  Development of the front-end is a challenge for scientists. It is too hard for people to deal with scientific area, mathematics, parallel programming, MPI, Linux command line, GUI programming and the SOAP simultaneously. A special front-end developer often is required.
  2.  The front-end software should be installed and configured. It brings some troubles in the early stages of development, especially while updating all the client seats and reconfiguring them.

To resolve this and other issues, we propose the concept of the universal web-based front-end. Some ideas are based on the other works (for ex, Bajaj and Cutchin, 1999). Other ideas came from practice.

  1.  It is web-based. It allows users to access parallel application, visualize application’s state and control it via a web browser.
  2.  It allows users to collaborate during and after parallel program execution. It includes saving and sharing the snapshots of visualizations, and chat capabilities.
  3.  It is web-configurable. User has the ability to drag’n’drop visualization elements, attach them to program resources, and so on.
  4.  It contains a set of predefined visualization elements of various data representation types. This set is expandable by third-party compounds.
  5.  Visualization elements are SOAP clients themselves, and communicate directly with the dealer service.
  6.  Implementation technology of visualization elements may vary. It can be Flex, Silverlight, ActiveX, Java Applets, depending on the situation.
  7.  In advanced cases it may be useful to implement server-side visualization widgets. These widgets, in turn, may be implemented as another parallel program.

The pilot implementation of this concept has succeeded in several projects (Vasin et al, 2007).

  1.  implementation details

User opens a browser and connects to the web server, which in turn connects to the dealer service and retrieves the list of current parallel programs being executed on a cluster. User specifies the desired program. Web browser loads an HTML page with visualization widgets inside it. Each widget on the page is associated with specified resource of the parallel program. Widgets connect directly to the dealer service via SOAP and request data. User interacts with widgets, which in turn may invoke additional data or control requests to the dealer service.

  1.  benefits

First, person receives the steering capability for his program in a very fast way. After insertion of API calls (to publish program data and controllable variables), that person gets a GUI for his HPC program immediately. Further tuning will allow him to receive a more fine-grained control system.

As stated above, visual representation of the program’s data is configurable. Person may arrange and configure data widgets – for example, to select presentation type. We do not fix visualization techniques. Thus we receive an expandable and flexible system.

Proposed architecture may be considered as a basis for remote rendering framework. Imagine that you created parallel rendering application, for example using the Equalizer framework. While running in conjunction with proposed steering system, your application receives visualization parameters, renders image, publishes it to the dealer service and then image output is delivered to client’s web browser.

  1.  conclusion

We have already tested some features of stated concept. The system was successfully used in several research projects (Vasin et al, 2007). Implementing all the features of stated concept will provide easy configurable, cross-platform steering system, which allows users to start steering their parallel applications in a quick way. As an optimum, to enable steering for parallel application, only two steps are necessary. First, researcher should insert data declaration calls in the source code. Second, he should login to the steering system via web browser and configure visual representation of declared data by mouse.

Web-based architecture allows scientists to steer their applications from any physical location connected to the system. Performing smart and secure connections (for example via VPN) will allow seeing the computational task state from other institutions and cities.

Web environment is handy for collaboration. Implementing collaboration service will allow researchers to communicate during and after program execution, discussing the computational experiment.

Logging and snapshot capabilities allow a researcher to turn back to the intermediate visual states of a computation, even after few weeks. In addition, users may share these visual states with each other in a common repository.

When data traffic becomes huge, we may receive a bottleneck. In this case, the following idea may be used. Visualization widgets should have the ability to refer to remote visualization program, which has fast network connection with computational cluster. This program may be parallel and being executed on the same cluster. Visualization program consumes computational program’s published data and generates new “image” data. Thus, we degrade network traffic between cluster and person’s computer to constant level.

Visualization is only a part of a computation process. A person prepares the data, invokes parallel program start command, and then switches to the browser to interact with the parallel program. We consider that in some cases it is useful to perform all these steps in a single web-based environment. Integration with supercomputer’s scheduling system is a subject to additional research.


Averbukh V. L. et al, 2002. Applying visualization to high-performance computations. In The problems of Nuclear Science and Technology, Vol 2, pp. 40-52.  

Bajaj, C. and Cutchin, S, 1999. Web based collaborative visualization of distributed and parallel simulation. Proceedings   of Parallel Visualization and Graphics Symposium. San Francisco, California, United States, pp. 47 – 54.

Kohl J. A. et al, 2006. Cumulvs: Interacting with High-Performance Scientific Simulations, for Visualization, Steering and Fault Tolerance. In International Journal of High Performance Computing Applications, Vol. 20, No. 2, pp. 255-285.

Vasin V.V. et al, 2007. On parallelization and visualization of ill-conditioned problems using methods of regularization and iteration approximation on computational system MVS-1000. Proceedings of International Scientific Conference “Parallel computational technologies”, Chelyabinsk, Russia, pp. 227-233. 


А также другие работы, которые могут Вас заинтересовать

  267 Бюджетного кодекса РФ осуществляет предварительный и текущий контроль за ведением операций с бюджетными средствами главных распорядителей распорядителей и получателей бюджетных средств кредитных организаций других участников бюджетного процесса по исполняемым бюджетам и бюджетам государственных внебюджетных фондов взаимодействует с другими федеральными органами исполнительной власти в процессе осуществления указанного контроля и координирует их работу. Органы Казначейства осуществляют следующие главные задачи: а организация и...
  Их содержанием являются только те из них посредством которых образуются денежные фонды государства его территориальных подразделений а также предприятий организаций учреждений. По своему материальному выражению финансы представляют собой денежные фонды государства его территориальных подразделений субъектов Федерации муниципальных образований предприятий и учреждений организаций используемые для потребностей общества и развития производства. В результате функционирования финансов создаются распределяются и используются денежные...
  Основное предназначение управления финансами предприятия и финансового планирования состоит в том чтобы: обеспечить рациональную сбалансированность активов средств и пассивов источников финансирования предприятия. обеспечить достаточность платежных средств для исполнения всех обязательств предприятия как по срокам. Вторая из перечисленных выше задач теснейшим образом взаимосвязана с первой С точки зрения текущей деятельности текущих операции в центре внимания управляющего финансами находится именно сбалансированность платежного...
  В литературе высказывались сомнения по поводу самостоятельности финансового права как отрасли в системе права. Утверждалось к примеру что это часть государственного и административного права или даже только административного. В дискуссии по этому вопросу были высказаны убедительные доводы опровергнувшие упомянутые сомнения и главный аргумент наличие у финансового права специфики в предмете и методе правового регулирования что требуется для выделения совокупности юридических норм в отрасль права.
80133. Формы межбюджетных трансфертов, предоставляемых из федерального бюджета 116.5 KB
  Межбюджетные трансферты из федерального бюджета за исключением субвенций предоставляются при условии соблюдения органами государственной власти субъектов Российской Федерации бюджетного законодательства Российской Федерации и законодательства Российской Федерации о налогах и сборах. Субъекты Российской Федерации в бюджетах которых расчетная доля межбюджетных трансфертов из федерального бюджета за исключением субвенций в течение двух из трех последних отчетных финансовых лет превышала 5 процентов объема собственных доходов...
  Следовательно финансовая деятельность государства представляет собой деятельность государства по формированию распределению и использованию централизованных и децентрализованных фондов денежных средств обеспечивающих его бесперебойное функционирование и развитие. Под централизованными финансами понимают экономические отношения связанные с образованием и использованием фондов денежных средств аккумулируемых в государственной бюджетной системе и правительственных внебюджетных фондах. Иными словами к централизованным фондам денежных...
80135. Стабилизационный фонд РФ 202.5 KB
  Формирование и использование средств Фонда Фонд аккумулирует поступления средств вывозной таможенной пошлины на нефть и налога на добычу полезных ископаемых нефть при цене на нефть сорта Юралс превышающей базовую цену. Средства Фонда могут быть использованы на покрытие дефицита федерального бюджета при снижении цены на нефть ниже базовой цены. Если накопленный объем средств Фонда превышает 500 млрд. Объем использования средств Фонда определяется федеральным законом о федеральном бюджете на соответствующий финансовый год.
80136. Банковский контроль. Субъекты, осуществляющие банковский контроль 34 KB
  Банковский контроль осуществляется банками в ходе операций по кредитованию финансированию и расчетам. Банковский контроль необходим для эффективного использования банковских ссуд и направлен также на укрепление платежной дисциплины. Обеспечение возвратности кредита создает основу для банковского контроля в процессе кредитования а контроль в свою очередь служит необходимым условием возвратности кредита.