Control of Unmanned Underwater Vehicle as a Member of Vehicles Team Performing a Given Task

e-mail: a.zak@amw.gdynia.pl The article presents the control system of a single underwater vehicle which is a member of the underwater vehicles team. The application of the multi-agent system concept for modelling and controlling a team of homogeneous underwater vehicles is discussed. Issues relating to cooperation actions in the team of underwater vehicles are described and solved using the theory of games and vector quality index. The process of negotiations between the vehicles is modelled as a multiplayer cooperative game. Finally, the water region search tasks performed by a team of cooperating underwater vehicles are presented and discussed taking into account possible vehicle failures happening during task realization. Control of Unmanned Underwater Vehicle as a Member of Vehicles Team Performing a Given Task


INTRODUCTION
Currently, in the underwater robotics concepts are developed in which a team of unmanned underwater vehicles performs specific tasks.Among these tasks, the most common ones are: search, inspection, and recognition, which can be performed by a single robot, whereby a team of vehicles can perform them faster and more efficiently.The team of cooperating underwater vehicles has many advantages, such as high resistance to damage that allows the task to be performed in the event of failure of one of the robots, or the possibility of using vehicles with a much simpler structure due to the decomposition of the task.However, the problem of controlling a team of cooperating underwater vehicles is a more difficult issue than controlling a single vehicle.In this case, it is required to consider for example: communication between vehicles, coordination of activities, or conducting negotiations.
In the computer sciences, multi-agent systems are widely used in solving problems of a distributed or computationally complex nature.The multi-agent system includes communicating and cooperating agents whose task is to achieve common goals.The mechanism of multi-agent systems can be easily used to control a team of cooperating underwater vehicles, treating each of them as a separate agent.
The main goal of the research is to develop methods and algorithms for controlling an autonomous underwater vehicle that performs tasks as a member of a vehicle team.
The article consist of two main parts.The first part describes the methods and algorithms of operation of a single, autonomous underwater vehicle treated as an agent in a multi-agent system.The second part presents selected results of researches carried out in simulation conditions for the task of searching water region by a team of homogenous underwater vehicles.In the summary, the research is briefly discussed.

Unmanned Underwater Vehicle as an Agent
Let S={s 1 ,s 2 ,…} mean possible environmental conditions and A={a 1 , a 2 ,…} possible agent actions.While making a formal description of the agent's operation, it can be noted that the action can be treated as a function (Gnatowski, 2005;Weiss, 1999;Zak, 2013a): where S* -selected sequence of environmental states.
The environment can be defined as a function: The above means that the action taken results from the state of the environment, and the environment under the influence of the agent changes its state.
It is also necessary to define a function which allows to read the input signals by the agent, called perception.It can be written as a function: which maps the state of the environment into perception P. In connection with the above, the action function will take the following form: where P* -sequence of perception.
The agent's interactions with the environment can be treated as a game with Nature, meaning a two-person game with zero payout, which has the following assumptions (Ameljańczyk, 1984): -one player is an agent and the other is Nature as a hypothetical opponent of the agent; -nature is a passive opponent, so it does not depend on winning; -the agent's strategies are defined by ways of acting, and the strategies of Nature are its states; -Nature realizes its states according to a specific random mechanism, about which an agent can have or obtain certain information.
Nature has a set of states that will be numbered with a natural number, n Є {1,2,…,k}, n, k Є N, and the agent will have a set of decisions X.The result of the agent taking a certain decision x Є X and realizing a particular state of Nature n Є N is in some way assessed by the agent.Let this assessment having a fixed interpretation be a real number representing the loss that we will denote by the symbol F n (x).
At the beginning, let us assume that the agent has no information about the probability distribution of the states of Nature, so choosing the decision doesn't know what its assessment will be, because it doesn't know which state Nature will take.Any decision x Є X can be evaluated with k numbers F n (x), n Є N (Ameljańczyk, 1984).The agent will try to choose such a decision x that all of its grades F n (x), i.e. loss values, will be as low as possible.It will ensure that regardless of the state n which will be realized, his losses will be as small as possible.Formally, the above decision-making task can be treated as a multi-criteria type optimization task where R -dominance relation.
In this case, it is interesting to determine the form of the dominance relationship RСF(X) × F(X).Naturally, the suggestion is a relationship ≥ generated by a cone (Ameljańczyk, 1984): In the set Y = F(X) we will get a set of Pareto-optimal results.So, the agent will probably choose one of the decisions leading to the set Y k ≤ .A pessimistic agent can use the following dominance relationship R P (Ameljańczyk, 1984): where y, w Є Y and (y,w) Є R P .
This means that the result y is considered better than the result of w if the maximum conditional loss is lower than or equal to the maximum conditional loss in the case of w result.

action: S* → A
(1) In the case of a decision, this means that the decision x such that F(x) = y is better than the decision z such that F(z) = w if it guarantees a loss equal to or lower than the decision z.The dominant result of z will be such a result that (Ameljańczyk, 1984): for every w Є Y -{y}.
If the player is an optimistic agent, he will value the results in a different way.Let y, w Є Y and (y,w) Є R O when This means that the agent will prefer decision x such that F(x) = y over the decision z such that F(z) = w if the loss is lower than or equal to the corresponding loss in the case of a decision z.
Suppose that the agent knows the Nature state probability distribution.Let p n be the probability that the n-th state of Nature will occur.Of course, the following relationships take place: In this case, it seems advisable to use the method of compromise hierarchical solutions in which the order of criteria can be determined using the p n values.The dominance relationship can be defined using the expected loss value.Let y, w Є Y, then (y,w) Є R P when (Ameljańczyk, 1984) Figure 1.
The set of dominating solutions, with such a dominance relationship, is a set of such decisions x Є X, that (Ameljańczyk, 1984) So far, we have treated the agent as an abstract concept.Giving it the autonomy function consists in defining the action function to take depending on the input signals that make up the perception.The proposed solution adopts a layered, hierarchical architecture (Figure 1) (Zak, 2013b).Its advantage is the lack of the need to define the decision mechanism and the simplicity of operation.This architecture will consist of layers, where the lowest layer is responsible for reflex behavior, the middle one for planning, and the highest for cooperation with other agents (Gnatowski, 2005;Maza & Ollero;Mukkerm 1996).Each of the layers has a corresponding database which represents the state of the environment corresponding to the given layer.The control flow in this architecture takes place in two directions.First, the control signal is sent from the lower to the higher layers, and then from the higher to the lower layers.If the lower layer using its database is unable to take appropriate action, it sends the signal to the upper layer.From this it follows that the higher the layer, the less frequently it is activated (Gnatowski, 2005).

Algorithm of Agent Behavior
It was assumed that the team consists of heterogeneous vehicles, i.e. that their hardware and software layers, including operation algorithms are the same.It was also assumed that the team's goal is to search the set water region.The vehicle's operation algorithm should ensure that the goal set for each  agent is achieved, as well as the cooperation between the agents.According to the adopted structure, the algorithms related to detecting obstacles are implemented in the layer of reflex action as well as algorithms related to maintaining the course and speed of movement (Garus & Zak, 2010;Zak, 2016).The planning layer implements algorithms related to driving the vehicle along a set trajectory, including determining the trajectory of obstacle avoiding (Lisowski, 2016;Zak 2013a).At the level of cooperation, the vehicle exchanges data with other team members and also conducts negotiations in the event of damage to any of the vehicles and the need to undertake a substitute task (Lisowski, 2013).For this purpose, information about the condition of a given vehicle is sent between vehicles, as well as in the case of negotiations also the costs of performing replacement tasks by individual vehicles.Since communication between vehicles is most often carried out in water using hydro-modems, in the situation of a common transmission media token ring mechanism can be used.In the case of winning negotiation by a given agent, its trajectory is automatically extended by a part that was taken over as a substitute task (Zak, 2013a).
Figures 2 and 3 present the algorithm of operation of a single agent in the scope of cooperative activities, which should ensure the performance of the task in cooperation with other agents, i.e. to achieve a common goal -the search a given water region.
Convolution results of discrete functions x(n) and ψ (n) for different N.

RESULTS OF RESEARCH
The correctness of the adopted solutions has been verified on the basis of simulation tests carried out in the Matlab environment.The research assumes that a set of heterogeneous autonomous underwater vehicles performs the task of searching a given water region.A single autonomous underwater vehicle is an agent who, together with the other vehicles in team, has to pursue the main objective.Each of the vehicles has a specific purpose, which is to search the indicated part of the water region.The environment is the water into which the underwater vehicle interacts with the thrusters allowing it to move.The task of perception is primarily fulfilled by navigation devices that are necessary for the vehicle to move on a given trajectory.In addition, technical observation devices are used, for example to detect obstacles.Perception functions can also be met by technical observation devices through which the vehicle can detect the objects sought if the task requires it; otherwise, they are not important from the point of view of the agent's implementation.During the task performance, the vehicle cooperates with other vehicles in the team; in particular, it exchanges data coordinating the work.Each vehicle also has low-level control algorithms that allow it to travel along the trajectories.Before starting the mission, each vehicle acquires the following data from the supervision system (mission planner): -number of vehicles participating in the task; -description of the reference system common to all vehicles; -description of the shape of the water region in the form of a contour defined by a set of points with coordinates (x,y) in the assumed reference system; -the trajectory on which a given vehicle is to move, as defined in the adopted reference system; -trajectories of other vehicles involved in the task.
The developed algorithms were tested during a research in which the task was to search the Szczecin harbor by a team consisting of five autonomous underwater vehicles.In the first test, it was assumed that none of the vehicles failed during the task.Table 1 summarizes the data that numerically characterizes the water region and the performed task.Figure 4  The course of the survey lines: 61°T he total length of the route [m]: 15,503.9 The total time of completing the task by a single vehicle [min]:

129.2
As a result of this test, the vehicle team should complete the search task after 39.4 min., which is a shorter time by about 1/3 compared to the time necessary for a single vehicle to perform this task.
In the second test, it was assumed that the underwater vehicle No. 2 had failed after passing 1,975 m of the route assigned to it.According to the adopted solutions, other vehicles undertook negotiations to determine which of the vehicles,  possibly a vehicle coalition, would complete the task for vehicle No. 2. As a result of the negotiations, the vehicles reported the costs of the substitute task and determined the profits from individual coalitions as well as the basis on which the solution was chosen.The optimal solution should ensure the highest profit and the lowest costs.In the proposed solution, it was assumed that the priority was to minimize the time to perform the task of searching the given water region.Accordingly, the cost will be the route that the vehicle undertaking the substitute task has to travel (the sum of the route to reach the starting point of substitute task and the route of performing the substitute task).The profit will be the difference between the maximum cost of performing the substitute task by a single vehicle with the longest route of access to the starting point of the substitute task and the reported costs by particular coalitions of vehicles.In the search for an optimal solution, some restrictions were also adopted.Firstly, a coalition that performs a substitute task must be as large as possible.Secondly, the profit to be obtained as a result of the formation of a given coalition is to be at least L times greater than the maximum profit of all coalition partners, where L is the number of coalitions.In addition, each of the coalition members must have an assigned trajectory on which he will perform a search of a length greater than or equal to the two lengths of the route to the starting point of the substitute task.Based on this, it was determined that the most cost-effective solution would be taken over the task for vehicle No. 2 by a coalition of agents consisting of vehicles No. 3 and No. 5.As a result of realizing the substitute task, the routes of vehicles 3 and 5 will be extended respectively to 6,848.4 m and 6,470.4m.This meant that the entire task would be completed in the time of 54.3 min., i.e. 14.9 min.longer than in the absence of a vehicle No. 2 breakdown.

SUMMARY
On the basis of the conducted research, it can be concluded that the use of many cooperating vehicles for the task of searching a given water region has two basic advantages.It Trans.marit.sci.2019; 02: 18-25 allows to significantly reduce the time necessary to complete the task, and also ensures that the mission will finish successfully even in the event of failure of any of the vehicles.However, it should be noted that the time required to complete the task does not decrease in proportion to the number of vehicles used.This effect results from the need to take into account the time needed to reach the area of operation by each vehicle in the team.The research carried out indicates that the use of five vehicles allows to shorten the task execution time just over three times.
In the solution proposed, a single vehicle is treated as an agent in a multi-agent system.A single agent was modeled as a player with Nature, which allowed to describe his actions mathematically.Adoption of the hierarchical structure of the agent's operation enables a practical application of control algorithms, including controlling its behavior as a member of team.Communication between the vehicles working as a team in one water region is a serious problem, which is particularly difficult in underwater conditions.A proper data exchange allows the coordination of vehicles' activities and cooperation at the time of damage to one of the team members.The results of the tests indicate that the adopted solutions related to the control of a single vehicle being a member of the vehicle team are correct and give a satisfactory result in the case of searching tasks.It should be noted here that in the case of a mission of a different nature, it may be necessary to modify the adopted solutions and adapt them to the specific requirements.

Figure 2 .
Figure 2. The operation algorithm of a single agent -an underwater vehicle, Part 1, a continuation is shown in Figure 3.

Figure 3 .
Figure 3. Algorithm for the operation of a single agent -an underwater vehicle, Part 2 presents the route of passage of each underwater vehicle in the test No. 1, on the background of the electronic water map.
Routes of passage realized by each of the underwater vehicles forming a team of vehicles performing the task of searching the Szczecin harbor.Assignment of the color line to each vehicle: yellow line -vehicle No. 1, green line -vehicle No. 2, blue line -vehicle No. 3, white line -vehicle No. 4, magenta line -vehicle No. 5.