77285

Web based computational steering system

Научная статья

Информатика, кибернетика и программирование

Secondly person cn shorten the debugging time of prllel progrm becuse person is ble to drive the computtion process djusting the lgorithm prmeters or progrm execution pth ccording to his needs. t the moment we hve developed smll nd cler PI nd dt server for prllel progrm developers tht llows progrm to be visulized online. Computtion nodes re prllel progrm processes with some embedded PI clls which connects them to the steering system. Visuliztion frontends re set of worksttion progrms which visulize the tsk stte nd give the bility to...

Английский

2015-02-02

52.5 KB

0 чел.

Web based computational steering system

Pavel Vasev

IMM UrB RAS
16, S.Kovalevskaja street, 620219, Ekaterinburg GSP-384, Russia

ABSTRACT

This paper presents the insight of web-based steering system for high performance computing. It is a lightweight solution for scientists and allows them to perform application steering with minimal technical efforts.  

KEYWORDS

HPC, visualization, collaboration, steering, web.

  1.  INTRODUCTION

Visualization is a superior tool for analysis of computational experiment in scientific and engineering research. As computer evolution goes on there are various technical approaches that emerge to support visualization. We will consider two of them: application steering and remote visualization.

Application steering (also called online visualization) is a tool to visualize and manipulate the current state of a running HPC (high performance computing) application. It provides great benefits. First, person is capable to perform an online simulation, without waiting for the whole computation process to finish. Secondly, person can shorten the debugging time of a parallel program, because a person is able to drive the computation process, adjusting the algorithm parameters or program execution path according to his needs.

Remote visualization is a technique that uses remote rendering resources. Remote visualization addresses situations when it is much more effective to render computation data remotely than to transmit that data to person’s computer, even partially.

To make these great things possible in a HPC environment, a computational steering system should exist. There are many steering systems developed worldwide. There are different technical and philosophical approaches exist (Averbukh et al, 2002). It is important that there are two end-user sides in such systems: HPC application interface and graphical interface. First is to provide application’s data to the steering system, and second is to see that data. We develop yet another system that targets simplicity of use for a HPC software developer. At the moment we have developed a small and clear API and a data server for parallel program developers that allows a program to be visualized online. This allows creating steer-enabled computing applications and remote visualization applications.

There is one problem. It is very difficult to develop front-end from scratch for each scientific project. Therefore, a universal front-end framework is required. It should provide the ability to generate front-ends for HPC applications semi-automatically. We are working on it and have found some ideas that form the concept of universal web-based front-end for application steering. We will describe the concept in the following pages. Let’s start by examining the status of steering system being developed.

  1.  CURRENT STATE

The idea of the system had evolved from the CUMULVS project (Kohl et al, 2006) developed by Oak Ridge National Laboratory. We implemented their approach with some modifications.

Our steering system consists of three layers: computation nodes, dealer service and visualization front-ends. Computation nodes are parallel program processes with some embedded API calls which connects them to the steering system. The dealer service is the heart of the system, it accepts front-end’s requests and proxies them to computational nodes. Visualization front-ends are a set of workstation programs, which visualize the task state and give the ability to control parallel program to the users.

There are two application interfaces in the steering system. One is for computation nodes, and another for front-ends. API for a parallel program is a C/Fortran interface. To make a parallel program steerable, a developer should insert appropriate API calls and link program with additional library. This API calls describes how data is distributed in application’s memory and provides other functionality.

Front-ends communicate with the dealer using SOAP. As parallel program is launched, front-ends can now connect to the dealer service and ask it for contents of the program’s data or perform other actions. Any SOAP-enabled platform and programming language may be used to create front-ends. We have successfully implemented some front-ends using Borland C++ and Adobe Flex 2.

When all preparations are done, a scientist usually works as follows. He starts a parallel program in a usual way. When it is actually spawned by the cluster scheduler, the scientist starts a front-end. Front-end connects to the parallel program, visualizes its state and allows the scientist to perform control actions.

  1.  WEB-BASED CONCEPT

In practice, we have found some issues:

  1.  Development of the front-end is a challenge for scientists. It is too hard for people to deal with scientific area, mathematics, parallel programming, MPI, Linux command line, GUI programming and the SOAP simultaneously. A special front-end developer often is required.
  2.  The front-end software should be installed and configured. It brings some troubles in the early stages of development, especially while updating all the client seats and reconfiguring them.

To resolve this and other issues, we propose the concept of the universal web-based front-end. Some ideas are based on the other works (for ex, Bajaj and Cutchin, 1999). Other ideas came from practice.

  1.  It is web-based. It allows users to access parallel application, visualize application’s state and control it via a web browser.
  2.  It allows users to collaborate during and after parallel program execution. It includes saving and sharing the snapshots of visualizations, and chat capabilities.
  3.  It is web-configurable. User has the ability to drag’n’drop visualization elements, attach them to program resources, and so on.
  4.  It contains a set of predefined visualization elements of various data representation types. This set is expandable by third-party compounds.
  5.  Visualization elements are SOAP clients themselves, and communicate directly with the dealer service.
  6.  Implementation technology of visualization elements may vary. It can be Flex, Silverlight, ActiveX, Java Applets, depending on the situation.
  7.  In advanced cases it may be useful to implement server-side visualization widgets. These widgets, in turn, may be implemented as another parallel program.

The pilot implementation of this concept has succeeded in several projects (Vasin et al, 2007).

  1.  implementation details

User opens a browser and connects to the web server, which in turn connects to the dealer service and retrieves the list of current parallel programs being executed on a cluster. User specifies the desired program. Web browser loads an HTML page with visualization widgets inside it. Each widget on the page is associated with specified resource of the parallel program. Widgets connect directly to the dealer service via SOAP and request data. User interacts with widgets, which in turn may invoke additional data or control requests to the dealer service.

  1.  benefits

First, person receives the steering capability for his program in a very fast way. After insertion of API calls (to publish program data and controllable variables), that person gets a GUI for his HPC program immediately. Further tuning will allow him to receive a more fine-grained control system.

As stated above, visual representation of the program’s data is configurable. Person may arrange and configure data widgets – for example, to select presentation type. We do not fix visualization techniques. Thus we receive an expandable and flexible system.

Proposed architecture may be considered as a basis for remote rendering framework. Imagine that you created parallel rendering application, for example using the Equalizer framework. While running in conjunction with proposed steering system, your application receives visualization parameters, renders image, publishes it to the dealer service and then image output is delivered to client’s web browser.

  1.  conclusion

We have already tested some features of stated concept. The system was successfully used in several research projects (Vasin et al, 2007). Implementing all the features of stated concept will provide easy configurable, cross-platform steering system, which allows users to start steering their parallel applications in a quick way. As an optimum, to enable steering for parallel application, only two steps are necessary. First, researcher should insert data declaration calls in the source code. Second, he should login to the steering system via web browser and configure visual representation of declared data by mouse.

Web-based architecture allows scientists to steer their applications from any physical location connected to the system. Performing smart and secure connections (for example via VPN) will allow seeing the computational task state from other institutions and cities.

Web environment is handy for collaboration. Implementing collaboration service will allow researchers to communicate during and after program execution, discussing the computational experiment.

Logging and snapshot capabilities allow a researcher to turn back to the intermediate visual states of a computation, even after few weeks. In addition, users may share these visual states with each other in a common repository.

When data traffic becomes huge, we may receive a bottleneck. In this case, the following idea may be used. Visualization widgets should have the ability to refer to remote visualization program, which has fast network connection with computational cluster. This program may be parallel and being executed on the same cluster. Visualization program consumes computational program’s published data and generates new “image” data. Thus, we degrade network traffic between cluster and person’s computer to constant level.

Visualization is only a part of a computation process. A person prepares the data, invokes parallel program start command, and then switches to the browser to interact with the parallel program. We consider that in some cases it is useful to perform all these steps in a single web-based environment. Integration with supercomputer’s scheduling system is a subject to additional research.

REFERENCES

Averbukh V. L. et al, 2002. Applying visualization to high-performance computations. In The problems of Nuclear Science and Technology, Vol 2, pp. 40-52.  

Bajaj, C. and Cutchin, S, 1999. Web based collaborative visualization of distributed and parallel simulation. Proceedings   of Parallel Visualization and Graphics Symposium. San Francisco, California, United States, pp. 47 – 54.

Kohl J. A. et al, 2006. Cumulvs: Interacting with High-Performance Scientific Simulations, for Visualization, Steering and Fault Tolerance. In International Journal of High Performance Computing Applications, Vol. 20, No. 2, pp. 255-285.

Vasin V.V. et al, 2007. On parallelization and visualization of ill-conditioned problems using methods of regularization and iteration approximation on computational system MVS-1000. Proceedings of International Scientific Conference “Parallel computational technologies”, Chelyabinsk, Russia, pp. 227-233. 


 

А также другие работы, которые могут Вас заинтересовать

5234. Історія психології. Лекції. Предмет історії психології 1018.5 KB
  Предмет історії психології, її розвиток і методологічні принципи План: Предмет, методи і основні етапи розвитку історії психології. Головні чинники і принципи розвитку психології. І. Предмет, методи і основні етапи історії психології Іст...
5235. Історія української культури. Конспект лекцій 1.02 MB
  Тема 1. Загальне поняття про культуру, її матеріальніта духовні цінності План Походження та зміст поняття культура. Структура культури. Функції культури. Основні культурні форми. Мистецтво як важлива склад...
5236. Історія України від найдавніших часів до сьогодні. Конспект лекцій 1.18 MB
  Найдавніша історія України. Первісне суспільство і перші державні утворення на території України. Початок формування людської цивілізації на території України. Кам'яний вік. Епоха міді-бронзи. Трипільська культура...
5237. Київська Русь. Початки державотворчого процесу 90 KB
  Київська Русь. План лекції: Особливості державотворчого процесу в Європі. Теорії походження Київської Русі. Основні етапи становлення і розвитку Давньоруської держави. Період становлення Київської Русі. Політичний та соціальний устрій Київ...
5238. Українські землі у складі Литви і Польщі. Початок козаччини 36.5 KB
  Українські землі у складі Литви і Польщі. Початок козаччини. Соціально-економічне становище України. Зміни соціально-економічних порядків. 1. 1447 рік на Православну церкву поширено католицькі порядки, згідно яких король-князь мали право, розд...
5239. Козацький період історії україни 113 KB
  Початоккозаччини. 1.Причини і джерела виникнення українського козацтва. 2.Запорозька Січ.2.1. Виникнення Запорозької Січі. 2.2. Соціально-економічний та військовий устрій Запорожжя. 2.3. Як козаки воювали. 2.4. Освіта на Січі. 2.5. Реєстрове (...
5240. Визвольна війна 1648 року. 127.5 KB
  Визвольна війна 1648 року. 1.Напередодні Великого повстання. 2.Богдан Хмельницький 3.Перші перемоги. 4.Бойові дії 1649-1651рр. (Битва під Берестечком 1651р. 18-30 червня) 5. Умови Зборівського і Білоцерківського миру. 6. Причини блискучих пере...
5241. Економічний та культурний вплив Гетьманщини на Московію 67 KB
  Економічний та культурний вплив Гетьманщини на Московію. Економічний та культурний рівень козацької країни невпинно зростав. Літом 1654 року архидіакон Павло Алеппський, супроводжуючи свого батька, антіохійського патріарха Макарія, мандрував Україно...
5242. Итоги и уроки Великой Отечественной войны 49 KB
  Война для Советской страны и ее Вооруженных Сил была самым тяжелым и вместе с тем героическим временем. Она сформировала облик целой эпохи. При всем переплетении света и теней послевоенных десятилетий неизменным остается главное — бессмертный...