The ExpROVer is a software that allows its users to control Remotely Operated Vehicles (ROVs), specifically designed for the VideoRay Pro 4 (VRP4). It is open-sourced, free, supports various platforms, and even integrates several intelligent capabilities, such as object recognition, and semi-autonomous maneuvers.
It is comprised of 3 main elements:
The server machine is a computer connected to the VRP4’s umbilical cord, and will first bridge the ROV’s software with the ROS, and secondly process the inputs received by the ROV, with CV technology, to detect objects.
End devices communicate with the server machine through a wired or wireless network, using either the developed web interface or a specialized mobile application. They receive the VRP4’s sensors data, and can also send high-level instructions with which to control the ROV.
As such, the ExpROVer software can offer connectivity to your VRP4 from any device, provided the setup was correct, as well as an elevated movement autonomy - requiring less training on ROV operation from the user - and high confidence video analysis, detecting objects and fishes; or even a great starting point to easily modify in order to suit user needs.
ExpROVer aims to be a software solution that simplifies a ROV’s operation while promoting the development of new software modules which enhance the availability of ROV appliances. The main objectives of ExpROVer solution are the following:
The use of ExpROVer software provides its user with several benefits, such as:
The ExpROVer solution comprises two options: the base system and the complete system.
In the base system the user installs the software in the Linux PC connected to the ROV. This software will enable the reading the ROVs’ sensors values and the sending of the instructions into the ROV.
In this option the user should:
$ ./build.sh
.bashrc
file.Make sure your ROV is underwater before you start operating it using the system.
To start using the system you should execute a script:
$ rosrun videoray deploy.sh
.To stop using the system, simply terminate the script (for example, by pressing Ctrl + C
in the open terminal window).
In the complete system option the user should:
$ ./build.sh
.bashrc
file.For the apps installation proceed as following:
If you want to use the server machine directly to control the ROV please carefully read the previous section instead. If you want to use the available applications, please choose between the web application or the mobile application, depending on whether your device is an Android device or not.
To start the application simply type the IP address of the computer serving as server (to obtain this address you can use online tools, your computer settings window, or simple type $ ifconfig
in a terminal window).
A page is displayed in order to proceed with registration (see Figure 2),
If you already have an account, you can instead log into the system (as depicted in Figure 3),
After a successful authentication, the system is ready to operate the ROV. If you have no ROVs, you must first add your ROV. After that you can list all your ROVs (see Figure 4),
Once you add the ROV you can now establish a connection to the VRP4. By using the keyboard you can now remotely control your ROV.
To shutdown the system, you can simply close the browser window.
After the successful installation of the ExpROVer mobile App, double click on the ExpROVer icon. A sign-in screen will appear. Insert your credentials to access your ROV. In case you do not have yet an account, you should sign-up to obtain your credentials. (See Figure 6)
Note that, like in the web platform, you can also use your google account to sign up and to log into the system.
Once in the ExpROVer App the main screen to will appear as shown on the right in Figure 7. This screen presents all your ROVs, If you don’t have any ROV in the list you should click on the button “Add a new ROV”. By doing that a new screen will appear and you must insert the name and the respective IPv4 address of your ROV. You can also personalize it by uploading a photo of your ROV. After doing that and returning to main screen you should now be able to see your newly-added ROV in the list.
After choosing your ROV and turning it on you can star an expedition. To start an expedition click in the button “start an expedition (1). See Figure 8.
By clicking in 2 a new screen will be displayed with the history associated to that ROV, where you can see the video expeditions already performed (see the right side of Figure 8).
The configure button (3), allows you to reconfigure your ROV, changing its name, the IP address as well as its associated the photo.
Once you start an expedition you will be able to see what the ROV “is seeing” (see Figure 9). In the top you should see the ROVs’ depth and the water temperature at ROVs’ depth. You can also see it’s heading, if activated in the settings (5).
To maneuver the ROV you should use the joystick button (1) and to control your ROVs’ depth use the vertical joystick (2).
You can also easily control the intensity of ROVs’ lights by scrolling the lights button (3).
When you detect something that you consider important you can bookmark it, The system will store the exact time of occurrence and you can see it later. (4)
In (5) you can enable the heading sensor visualization, and activate or deactivate the presence of a reference picture.
Simply close the application.
The intelligent functinalities are available as separate scripts.
Computer Vision isn’t run in real time due to the latency it would cause. It can be run on the video captured from the ROV (during operation, the video feed from the ROV is both streamed and saved to a hard drive, making this possible).
You can fint the computer vision software in the project’s repository, integrated in the backend folder (more particuarly, in the subfolder /backend/YOLOv3/
).
To execute fish detection in a video, run the following command:
python yolo_video.py --input <input_video_path> --output <output_path>
# OR
python yolo_video.py [OPTIONS...] --image, for image detection mode
You can fint the semi-autonomous maneuvers in the project’s repository, integrated in the ROV folder (more particuarly, in the subfolder /backend/ROS/scripts/
).
To run any maneuver, with wither the base or complete system currently actively, simply execute the desired maneuver, using a terminal and the command: $ rosrun videoray <script name>
1. I am using the complete system but I cannot establish the connection with the ROV
Check if all the cables are correctly plugged in. If yes, then check if you are using the correct IP address for your ROV.
2. I am using the complete system with the web app but I cannot control my ROV or the video feed from it has frozen
Check if you have Internet access.
3. I have an iPhone and I cannot find the ExpROVer app
ExpROVer mobile app is available only for Android smartphones. However you can use the ExpROVer web app to control your ROV.
4. I am using the complete system and I have no video, despite being able to send commands
The problem is probably your ROV’s camera. See the ROV’s user manual for more detailed information.
5. The temperature displayed has a strange value
The problem is probably your ROV temperature sensor. Please check the ROV’s user manual for more detailed information.
6. The depth is marking more than 300m
The problem is probably with the ROV’s depth/pressure sensor. Consult the ROV’s user manual for more detailed information.