Deploy OpenVidu Pro on premises



Before deploying OpenVidu Pro you need to create an OpenVidu account to get your license key.
There's a 15 day free trial waiting for you!

Deployment instructions 🔗

OpenVidu Pro is deployed on premises in a cluster of machines with Docker and Docker Compose.

NOTE : Docker basic knowledge is not required, but recommended. If you are completely new to Docker and containers, take a few minutes to read the official Docker Get started documentation.

1) Prerequisites 🔗

You must have at least 2 different instances with a clean installation of any modern Linux distribution.

  • One instance for the Master Node
  • One instance for a Media Node

You can actually have as many instances as you want for Media Nodes. The number of Media Nodes determines the size of your cluster: the more Media Nodes, the more video sessions your cluster will be able to handle. Check out Scalability section for further details.

Once you have your instances ready, be sure to meet the following criteria in them:

  • 2 CPUs and 8GB of RAM at least, as well as a generous network bandwidth.

  • Install Docker

  • Install Docker Compose. NOTE: install docker-compose from the link (official Docker site) as minimum version 1.24 is required.

  • Configure a domain name: OpenVidu Pro is deployed using HTTPS because it is mandatory to use WebRTC. Then, if you do not have a domain name, an autogenerated SSL certificate will be used and an ugly warning will appear to your users when entering to your site. And of course you can suffer a man-in-the-middle attack. So it is recommended that you configure a domain name pointing to Master Node public IP (Media Nodes do not need a domain name, it is enough for them to have a public IP). A valid SSL certificate can be automatically generated using Let's Encrypt in the installation process. If you already have a valid SSL certificate of your own, it also can be configured.

  • Allow access to endpoint https://accounts.openvidu.io: it is mandatory that your Master Node is able to reach this endpoint. It is used to check the license key and register its usage. If you are deploying OpenVidu Enterprise with mediasoup, then it is also mandatory that your Media Nodes have also access to that endpoint.

  • Port configuration in Master Node

    • Open these ports (here you have an UFW sample to configure a firewall)

      • 22 TCP: to connect using SSH to admin OpenVidu.
      • 80 TCP: if you select Let's Encrypt to generate an SSL certificate this port is used by the generation process.
      • 443 TCP: OpenVidu Inspector is served by default in standard https port.
      • 3478 TCP+UDP: used by STUN/TURN server to resolve clients IPs
      • 5044 TCP: Necessary for Media Nodes instances to send metrics to OpenVidu. WARNING!! This port must be closed to the Internet and must only be accessible for your Media Nodes, or anyone could send metrics information to OpenVidu.
      • 9200 TCP Necessary for Media Nodes Instances to send metrics and logs to ElasticSearch. WARNING!! This port must be closed to the Internet and must only be accessible for your Media Nodes, or anyone could do http requests to your ElasticSearch.
      • 40000 - 65535 TCP+UDP: used by TURN server to establish relayed media connections.

    • Close all other ports: this is VERY important to avoid external attacks to OpenVidu internal services. Check Master Node troubleshooting section Close ports to avoid external attacks to learn more about this.

    • Free ports inside the server: Master Node services will need the following ports to be available inside the machine: 80, 443, 3478, 5044, 5442, 5443, 6379, 9200. If some of these ports is used by any process, OpenVidu platform won't work correctly. It is a typical error to have an NGINX process in the system before installing OpenVidu. Please uninstall it.

    • It is also recommended to allow outbound traffic to all ports.

  • Port configuration in Media Nodes

    • Open these ports (here you have an UFW sample to configure a firewall)

      • 22 TCP: to connect using SSH to admin OpenVidu.
      • 443 TCP+UDP: used by STUN/TURN server to resolve clients IPs when OPENVIDU_PRO_COTURN_IN_MEDIA_NODES=true.
      • 40000 - 65535 TCP+UDP: used by Kurento Media Server to establish media connections.
      • 8888 TCP: Kurento Media Server handler listens on port 8888. WARNING!! This port must be closed to the Internet and must only be accessible for your Master Node, or anyone could spy your sessions.
      • 3000 TCP: All Media Nodes offer a REST API endpoint to provision their services. WARNING!! This port must be closed to the Internet and must only be accessible for your Master Node, or anyone could spy your sessions.
      • 4000 TCP (Optional) : This port is only needed if you want to use the Speech to text functionality. WARNING!! This port must be closed to the Internet and must only be accessible for your Master Node, or anyone could spy your sessions.

    • Close all other ports: this is VERY important to avoid external attacks to OpenVidu internal services. Check Media Node troubleshooting section Close ports to avoid external attacks to learn more about this.

    • Free ports inside the server: Media Node services will need the port 8888 and 3000 to be available inside the machine.

    • It is also recommended to allow outbound traffic to all ports.



2) Master Node 🔗

2.1) Deployment 🔗

Connect through SSH to Openvidu Server Pro instance. When you are on a terminal of the instance, change to root user. Root permissions are necessary to deploy OpenVidu.

sudo su

The recommended folder to install OpenVidu Pro is /opt. Every other instruction in the documentation regarding on premises deployment assumes this installation path.

cd /opt

Now execute the following command to download and run the installation script.

curl https://s3-eu-west-1.amazonaws.com/aws.openvidu.io/install_openvidu_pro_latest.sh | bash

This will download all required files into openvidu folder and will show this message with basic instructions:

=======================================
Openvidu PRO successfully installed.
=======================================

1. Go to openvidu folder:
$ cd openvidu

2. Configure OPENVIDU_DOMAIN_OR_PUBLIC_IP, OPENVIDU_PRO_LICENSE,
OPENVIDU_SECRET, and ELASTICSEARCH_PASSWORD in .env file:
$ nano .env

3. Start OpenVidu
$ ./openvidu start

CAUTION: The folder 'openvidu/elasticsearch' use user and group 1000 permissions.
This folder is necessary for store elasticsearch data.
For more information, check:
https://docs.openvidu.io/en/2.26.0/deployment/pro/on-premises/#deployment-instructions

To deploy a fixed version, including previous ones, replace latest with the desired version number.
For example: curl https://s3-eu-west-1.amazonaws.com/aws.openvidu.io/install_openvidu_pro_2.29.0.sh | bash


2.2) Configuration 🔗

OpenVidu Pro configuration is specified in the .env file with environment variables.

  • You must give a value to properties DOMAIN_OR_PUBLIC_IP, OPENVIDU_SECRET and ELASTICSEARCH_PASSWORD. Default empty values will fail.
  • You must also provide a value for OPENVIDU_PRO_LICENSE. You need an OpenVidu account to purchase it. There's a 15 day free trial waiting for you!
  • You can change the CERTIFICATE_TYPE if you have a valid domain name. Setting this property to letsencrypt will automatically generate a valid certificate for you (it is required to set property LETSENCRYPT_EMAIL). Or if for any unknown reason you prefer to use your own certificate, set the property to owncert and place the certificate files as explained.
  • All other configuration properties come with sane defaults. You can go through them and change whatever you want. Visit OpenVidu CE configuration and OpenVidu Pro configuration for further information.

The .env file is pretty self-explanatory. It looks like this:

# OpenVidu configuration
# ----------------------
# Documentation: https://docs.openvidu.io/en/stable/reference-docs/openvidu-config/

# NOTE: This file doesn't need to quote assignment values, like most shells do.
# All values are stored as-is, even if they contain spaces, so don't quote them.

# Domain name. If you do not have one, the public IP of the machine.
# For example: 198.51.100.1, or openvidu.example.com
DOMAIN_OR_PUBLIC_IP=

# OpenVidu PRO License
OPENVIDU_PRO_LICENSE=

# OpenVidu SECRET used for apps to connect to OpenVidu server and users to access to OpenVidu Dashboard
OPENVIDU_SECRET=

# Certificate type:
# - selfsigned:  Self signed certificate. Not recommended for production use.
#                Users will see an ERROR when connected to web page.
# - owncert:     Valid certificate purchased in a Internet services company.
#                Please put the certificates files inside folder ./owncert
#                with names certificate.key and certificate.cert
# - letsencrypt: Generate a new certificate using letsencrypt. Please set the
#                required contact email for Let's Encrypt in LETSENCRYPT_EMAIL
#                variable.
CERTIFICATE_TYPE=selfsigned

# If CERTIFICATE_TYPE=letsencrypt, you need to configure a valid email for notifications
LETSENCRYPT_EMAIL=user@example.com

...

If you have questions about how to configure your Domain and SSL certificates, you can check these examples:

Elasticsearch Configuration 🔗

By default, the deployment is configured to use an Elasticsearch and a Kibana service running next to OpenVidu Server Pro. This can be convenient sometimes but it have it downsides because Elasticsearch, Kibana and OpenVidu Server Pro will be running in the same machine. These downsides are:

  • You will need to monitor disk space: OpenVidu generates events and all logs and metrics are sent to Elasticsearch. You will need to take special care of the OPENVIDU_PRO_ELASTICSEARCH_MAX_DAYS_DELETE parameter in the /opt/openvidu/.env file of your deployment so you don't run out of disk space.
  • Resources used By OpenVidu Server Pro are shared with Elasticsearch and Kibana: It is well known that Elasticsearch and Kibana can consume a lot of resources. If you want to keep OpenVidu Server Pro free of this resource consumption, deploy Elasticsearch and Kibana externally.

We recommend to configure an External Elasticsearch and Kibana. Take a look into how to configure it here

If you don't want to use Elasticsearch and Kibana, you can disable it by setting the OPENVIDU_PRO_ELASTICSEARCH parameter to false in the /opt/openvidu/.env file of your deployment. This will disable the Elasticsearch and Kibana services so OpenVidu Server Pro will not send any history sessions, logs or metrics to Elasticsearch.


Coturn Configuration 🔗

Coturn is an implementation of the TURN/STUN protocol needed for WebRTC ICE candidates discovery, and it is an essential part of WebRTC. By default it is configured in OpenVidu, but you can choose where to deploy this service.

It is configured with the parameter OPENVIDU_PRO_COTURN_IN_MEDIA_NODES which by default is set to false, so Coturn will be deployed at the master node. If you want to deploy Coturn across all media nodes, you need to set OPENVIDU_PRO_COTURN_IN_MEDIA_NODES=true, and OpenVidu will deploy that services in all your media nodes.

You can check for more parameters that can be used for Coturn configuration in the Advanced parameters for OpenVidu Pro

Note: If you are using OPENVIDU_PRO_COTURN_IN_MEDIA_NODES=false, you need port 3478 TCP/UDP open in the master node. Otherwise, with OPENVIDU_PRO_COTURN_IN_MEDIA_NODES=true, you need to open port 443 TCP/UDP in media nodes.

Videoconference application 🔗

By default, the OpenVidu Call application is deployed alongside OpenVidu Platform. It is accessible in the URL:

https://DOMAIN_OR_PUBLIC_IP:HTTPS_PORT/

For more information on how to disable it and deploy your own application (if needed) visit the section Deploy OpenVidu based Applications.


2.3) Execution 🔗

To start OpenVidu Platform (and the application if enabled) execute this command:

./openvidu start

All docker images for services will be downloaded (only the first time) and executed.

The first part of the log shows how docker-compose command executes all services:

Creating openvidu_kibana_1          ... done
Creating openvidu_app_1             ... done
Creating openvidu_elasticsearch_1   ... done
Creating openvidu_coturn_1          ... done
Creating openvidu_redis_1           ... done
Creating openvidu_openvidu-server_1 ... done
Creating openvidu_nginx_1           ... done

Then, openvidu-server service logs are shown. When OpenVidu Platform is ready you will see this message:

----------------------------------------------------

   OpenVidu Platform is ready!
   ---------------------------

   * OpenVidu Server: https://DOMAIN_OR_PUBLIC_IP/

   * OpenVidu Dashboard: https://DOMAIN_OR_PUBLIC_IP/dashboard/

----------------------------------------------------

You can press Ctrl+C to come back to the shell and OpenVidu Pro will be executed in the background.


Available services 🔗
  • Consume OpenVidu REST API through https://DOMAIN_OR_PUBLIC_IP/
  • If the application is enabled, it will also be available at https://DOMAIN_OR_PUBLIC_IP/
  • You can access OpenVidu Inspector at https://DOMAIN_OR_PUBLIC_IP/inspector/
  • You can access Kibana at https://DOMAIN_OR_PUBLIC_IP/kibana/

2.4) Administration 🔗

Run the following commands to manage OpenVidu Pro service:

  • Start OpenVidu Pro

    ./openvidu start
    
  • Stop OpenVidu Pro

    ./openvidu stop
    
  • Restart OpenVidu Pro

    ./openvidu restart
    
  • Show logs of OpenVidu Pro

    ./openvidu logs
    
  • Show actual installed version of OpenVidu Server Pro and basic information about the deployment.

    ./openvidu version
    
  • Generate a report with useful information of the OpenVidu Server Pro deployment. This report includes: system information, containers running, logs and configuration files. More information about this command is available at troubleshooting section: Generate a report with all deployment information

    ./openvidu report
    
  • List commands

    ./openvidu help
    

To change current configuration, you just need to update .env configuration file with the new desired values and run ./openvidu restart command.



3) Media Nodes 🔗

Remember that Media Nodes must be deployed in their own separate instance. They cannot coexist with the Master Node or other Media Nodes in the same instance.

3.1) Deployment 🔗

Follow these steps to add one Media Nodes to the cluster. You can add as many Media Nodes as you want by repeating these instructions in different instances.

Connect through SSH to the Media Node instance. When you are on a terminal of the instance, change to root user. Root permissions are necessary to deploy OpenVidu.

sudo su

The recommended folder to install the Media Node is /opt. Every other instruction in the documentation regarding on premises deployment assumes this installation path.

cd /opt

Now execute the following command to download and run the installation script.

curl https://s3-eu-west-1.amazonaws.com/aws.openvidu.io/install_media_node_latest.sh | bash

This will download all required files into kms folder and will show this message with basic instructions:

=======================================
Media Node successfully installed.
=======================================

1. Go to kms folder:
$ cd kms

2. Start Media Node Controller
$ ./media_node start

3. This will run a service at port 3000 which OpenVidu will use to deploy necessary containers.
Add the private ip of this media node in "KMS_URIS=[]" in OpenVidu Pro machine
in file located at "/opt/openvidu/.env" with this format:
        ...
        KMS_URIS=["ws://<MEDIA_NODE_PRIVATE_IP>:8888/kurento"]
        ...
You can also add this node from inspector

4. Start or restart OpenVidu Pro and all containers will be provisioned
automatically to all the media nodes configured in "KMS_URIS"

To deploy a fixed version, including previous ones, replace latest with the desired version number.
For example: curl https://s3-eu-west-1.amazonaws.com/aws.openvidu.io/install_media_node_2.29.0.sh | bash



3.2) Configuration 🔗

There is actually no configuration file in Media Nodes. Everything is configured using the configuration located in /opt/openvidu/.env of Master Node. Media nodes run a service called Media Node Controller which has an http endpoint at port 3000. This http endpoint is used by OpenVidu Pro Server to deploy all the necessary docker containers in a way that all monitoring containers and Kurento Media Server are configured correctly.

By default Kurento Media Server is launched with some default sane configuration:

  • Min Port: 40000
  • Max Port: 65535

However, Kurento Media Server is fully configurable through the configuration file located at /opt/openvidu/.env of Master Node. You can customize how Kurento Media Server is launched in your Media Nodes by adding the environment variables defined in their official documentation in the file located at /opt/openvidu/.env of the Master Node. For example:

KMS_DOCKER_ENV_GST_DEBUG=3,Kurento*:5,kms*:4,sdp*:4,webrtc*:4,*rtpendpoint:4,rtp*handler:4,rtpsynchronizer:4,agnosticbin*:5,kmssdpsession:5

If you add this environment variable you will configure Kurento Media Server in all your media nodes to use GST_DEBUG with this value. Notice that all environment variables located in Master Node related with Kurento Media Server needs to be prefixed with KMS_DOCKER_ENV_.

Example of Kurento Media Server parameters configuration 🔗

Let's supose that we want to run Kurento Media Server with a different range than the default, for example, 50000-60000. To do it you just need to:

1) Ssh into your Master Node.

2) Edit the file located in /opt/openvidu/.env and add these environment variables:

KMS_DOCKER_ENV_KMS_MIN_PORT=50000
KMS_DOCKER_ENV_KMS_MAX_PORT=60000

3) Restart OpenVidu with:

./openvidu restart

OpenVidu will do one of these things in all your Media Nodes configured in KMS_URIS:

  • Deploy (If containers are not running).
  • Restart (If configuration of Kurento Media Server has changed).



3.3) Execution 🔗

To start the Media Node execute this command:

./media_node start

All docker images for services will be downloaded (only the first time) and executed.

The first part of the log shows how docker-compose command executes the Media Node controller service:

Creating kms_media-node-controller_1 ... done

WARNING 1: after the Media Node service is up and running, you must manually add the Media Node to the cluster before you can start using it. Follow instructions in section Change the number of Media Nodes on the fly to do so.



3.4) Administration 🔗

Run the following commands to manage Media Node Controller service:

  • Start Media Node Controller

    ./media_node start
    
  • Stop Media Node Controller and all containers running

    ./media_node stop
    
  • Restart Media Node Controller and stop all running containers

    ./media_node restart
    
  • Show logs of Media Node Controller

    ./media_node logs
    
  • Show logs of Kurento Media Server

    ./media_node kms-logs
    
  • Show actual installed version of OpenVidu Server Pro and basic information about the deployment.

    ./media_node version
    
  • Generate a report with useful information of the OpenVidu Server Pro deployment. This report includes: system information, containers running, logs and configuration files. More information about this command is available at troubleshooting section: Generate a report with all deployment information

    ./media_node report
    
  • List commands

    ./media_node help
    

WARNING 1: after the Media Node service is up and running, you must manually add the Media Node to the cluster before you can start using it. Follow instructions in section Change the number of Media Nodes on the fly to do so.

WARNING 2: a reboot of Master Node will make necessary to manually add all the running Media Nodes to the cluster before you can start using them (unless you have properly configured in Master Node property KMS_URIS). Follow instructions in section Change the number of Media Nodes on the fly to add running Media Nodes.



Domain and SSL Configuration Examples 🔗

For these examples, all steps from Deploying OpenVidu Pro on premises should be followed. These examples are focusing in the Domain and SSL Configuration. The necessary parameters to use are explained in the Configuration Section. This examples scenarios have the purpose to clarify any doubt on how to configure the Domain and SSL configuration of OpenVidu Pro.

As OpenVidu Pro is deployed with default sane configuration, your domain and SSL certificate configuration are the most important parameters to deploy your stack correctly.

Let's see all scenarios:

1) Self-signed certificate 🔗

This scenerario should be used only for development environments. Don't use it in production.

This scenario is meant for you if you want to:

  • Deploy OpenVidu Pro quickly for testing or developing purposes.
  • Deploy OpenVidu Pro without a Fully Qualified Domain Name (FQDN).

Let's suppose that OpenVidu Pro is installed following the instructions from here, and we have, for example, this public IP in the Master Node: 123.12.123.12

For this specific scenario you will need to:

1.1) Edit /opt/openvidu/.env 🔗

To make our deployment works with a self-signed certificate, you just need to edit the file in /opt/openvidu/.env and fill the next variables:

DOMAIN_OR_PUBLIC_IP=123.12.123.12

CERTIFICATE_TYPE=selfsigned

OPENVIDU_SECRET=<YOUR_SECRET>

Only these variables are needed to make it works. The rest of the variables in the /opt/openvidu/.env are referred to OpenVidu CE Configuration and OpenVidu Pro Configuration.

The variable LETSENCRYPT_EMAIL should be empty for this kind of certificate.

Also, if you have a Fully Qualified Domain Name pointing to your IP address, you can use it in the DOMAIN_OR_PUBLIC_IP. For example:

DOMAIN_OR_PUBLIC_IP=example.openvidu.io

2) Let's Encrypt certificate 🔗

This scenario is meant for you if you want to:

  • Deploy OpenVidu Pro for production or even developing purposes.
  • Deploy OpenVidu Pro with a Fully Qualified Domain Name (FQDN).
  • Use a valid SSL certificate.

For this specific scenario you will need to:

2.1) Register a FQDN pointing to your public IP 🔗

This depends of the DNS register you want to use. You need to create a DNS register of type A pointing to the Elastic IP created before. Let's suppose that our domain is: example.openvidu.io.

2.2) Edit /opt/openvidu/.env 🔗

To make our deployment works with a Let's encrypt certificate, you just need to edit the file in /opt/openvidu/.env and fill the next variables:

DOMAIN_OR_PUBLIC_IP=example.openvidu.io

CERTIFICATE_TYPE=letsencrypt

OPENVIDU_SECRET=<YOUR_SECRET>

LETSENCRYPT_EMAIL=youremail@youremail.com

Only these variables are needed to make it works. The rest of the variables in the /opt/openvidu/.env are referred to OpenVidu CE Configuration and OpenVidu Pro Configuration.

You must define the LETSENCRYPT_EMAIL where you want to register the domain and receive Let's Encrypt notifications.

3) Custom Certificate (Commercial CA) 🔗

This scenario is meant for you if you want to:

  • Deploy OpenVidu Pro for production.
  • Deploy OpenVidu Pro with a Fully Qualified Domain Name (FQDN).
  • Use a valid SSL certificate from a Commercial CA.

To use this kind of certificate, you need to generate two files, certificate.cert (public keys of the certificate) and certificate.key (private key) and copy them into the /opt/openvidu/owncert directory.

3.1) Register a FQDN pointing to your public IP 🔗

This depends of the DNS register you want to use. You need to create a DNS register of type A pointing to the Elastic IP created before. Let's suppose that our domain is: example.openvidu.io

3.2) Generate certificates files 🔗

1) Create a CSR and a private key. This can be easily done by executing:

openssl req -newkey rsa:2048 -nodes -keyout certificate.key -out certificate.csr

While executing this command, you will be asked to enter some information to generate the files certificate.key and certificate.csr. Ensure that all these information are correctly inserted (Common Name, Organization Name, etc...). The most important parameter is the Common Name field which should match the name that you want to use for your certificate.

For example, let's suppose that your domain is example.openvidu.io. The parameter Common Name could be: example.openvidu.io or www.example.openvidu.io. Otherwise, if you're using a WildCart certificate, the Common Name parameter would be ** .openvidu.io*.

2) The previous command generated the certificate.key and certificate.csr files. The certificate.csr is the one that you need to provide to your CA. Depending of your CA this step can differ. Check your CA documentation about this topic.

3) Usually the files to generate the certificate.cert can be downloaded or are sent via email from the CA. These files are:

  • The intermediate certificate. (It usually have more than one key with ---BEGIN CERTIFICATE--- This file will be called as intermediate.cert in following steps.
  • Your ssl certificate. An unique certificate key with ---BEGIN CERTIFICATE---. This file will be called as public.cert in following steps.

4) You need to concat these two files in an unique certificate.cert file in this way:

cat public.cert intermediate.crt > certificate.cert

5) Now you have the certificate.key generated in step 1) and the certificate.cert generated in step 4).

If you're still having doubts about how to generate the certificates files, you can follow this guide for a further understanding.



3.3) Copy certificate.key and certificate.cert 🔗

Now that you have both certificate files, you need to copy the certificate.key and the certificate.cert into /opt/openvidu/owncert

The directory should look like this:

/opt/openvidu/owncert
├── certificate.cert
└── certificate.key

3.4) Edit /opt/openvidu/.env 🔗

To make our deployment works with a Custom Certificate (Commercial CA) certificate, you just need to edit the file in /opt/openvidu/.env and fill the next variables:

DOMAIN_OR_PUBLIC_IP=example.openvidu.io

OPENVIDU_SECRET=<YOUR_SECRET>

CERTIFICATE_TYPE=owncert

Only these variables are needed to make it works. The rest of the variables in the /opt/openvidu/.env are referred to OpenVidu CE Configuration and OpenVidu Pro Configuration.

The variable LETSENCRYPT_EMAIL should be empty for this kind of certificate.

Common problems 🔗


Scalability 🔗

Set the number of Media Nodes on startup 🔗

To deploy your OpenVidu Pro cluster with a specific initial number of Media Nodes you just need to install the Media Node service in as many machines as Media Nodes you want. For example, if you want your cluster to be able to grow up to 3 Media Nodes, then you will need 4 machines in your infrastructure: 1 for the Master Node and 3 for each Media Node.

Check out the Prerequisites section and make sure that all of your machines intended to run a Media Node service fulfill the requirements for doing so. Then install and run the Media Node service in all of them.

If you know for sure that your Media Node IPs are not going to change over time, then you can fix them in a configuration property of OpenVidu Server Pro. Use property KMS_URIS to set the IPs of your Media Nodes if you want your Master Node to automatically connect to them on start up. Check out OpenVidu Pro configuration to learn more.

Change the number of Media Nodes on the fly 🔗

You can change the number of Media Nodes on the fly in two different ways.

Be careful: these Media Nodes won't be saved as permanent Media Nodes in Master Node. If you reboot OpenVidu Server Pro, it won't automatically re-connect to them. If you know for sure that the IPs of your Media Nodes are fixed and won't change, you can use configuration property KMS_URIS to make OpenVidu connect to them automatically on startup.

From OpenVidu Inspector 🔗

In Cluster page you can add and remove Media Nodes from your cluster just by pressing buttons. To add a Media Node, you need to have it already up and running (follow steps in Media Nodes section to install and run one) and define its URI like this: ws://NEW_MEDIA_NODE_IP:8888/kurento

WARNING: adding/removing Media Nodes from OpenVidu Inspector in On Premises deployments will not automatically launch/terminate your physical machines:

  • To add a new Media Node you need to have the new Media Node already up and running (follow steps in Media Nodes section to install and run one) and define its URI like stated in the image above.

  • To drop an existing Media Node you will have to terminate the physical machine yourself after successfully calling DELETE /openvidu/api/media-nodes, if that's what you want. Usually you will want to wait until the last of the sessions hosted in this Media Node is closed before you remove it. Then, you can listen to mediaNodeStatusChanged event through OpenVidu Webhook to know when you can safely terminate your instances (listen to terminated status).

With OpenVidu Pro REST API 🔗

You can programmatically add and remove Media Nodes from your cluster by consuming OpenVidu Pro REST API.

WARNING: there are some important aspects to keep in mind when launching and dropping Media Nodes through REST API in on premises OpenVidu Pro clusters:

  • Trying to drop a Media Node which is currently hosting an OpenVidu Session will fail by default. You can manage the drop policy when calling DELETE /openvidu/api/media-nodes through parameter deletion-strategy.

  • Launching/Dropping Media Nodes in on premises deployments will not automatically start/terminate your physical machines:
    • To launch a new Media Node you are required to have the Media Node already running (follow steps in Media Nodes section to install and run one). Then you must provide the Media Node's URI when calling POST /openvidu/api/media-nodes using uri query parameter.
    • To drop an existing Media Node you will have to terminate the physical machine yourself after successfully calling DELETE /openvidu/api/media-nodes, if that's what you want. Usually you will want to wait until the last of the sessions hosted in this Media Node is closed before you remove it. You can achieve this by setting the Media Node status to waiting-idle-to-terminate. Then, you can listen to mediaNodeStatusChanged event through OpenVidu Webhook to know when you can safely terminate your instance. The moment will come when terminated status is achieved: at that point it is safe to shut down the machine hosting the Media Node.



Updating OpenVidu Pro configuration 🔗

You may want to change the current configuration of an existing OpenVidu Pro cluster. This configuration includes all of the parameters listed in these pages:

Once the cluster is running, there are different ways you can update the value of the configuration parameters. Take into account that all of them require restarting your OpenVidu Server Pro process, so any active OpenVidu Session will be terminated.

1) With OpenVidu Inspector 🔗

OpenVidu Inspector allows you to restart the OpenVidu Server Pro process from Config page just by filling a formulary.
More information here.

NOTE 1: take into account that not all configuration properties are able to be updated this way
NOTE 2: new values will be stored and remembered, so they will be used when OpenVidu Server Pro is restarted in the future

2) With OpenVidu Pro REST API 🔗

You can consume REST API method POST /openvidu/api/restart to programmatically restart the OpenVidu Server Pro process and update its configuration values.

NOTE 1: take into account that not all configuration properties are able to be updated this way
NOTE 2: new values will be stored and remembered, so they will be used when OpenVidu Server Pro is restarted in the future

3) Manually connecting through SSH 🔗

The ultimate and most definitive way of updating the configuration parameters of an OpenVidu Pro cluster is connecting to the Master Node through SSH and changing the desired values:

  1. SSH to the OpenVidu Server Pro machine using your private rsa key
  2. Using root user with sudo su command, go to OpenVidu Pro installation folder (default and recommended is /opt/openvidu)
  3. Update file .env with the new configuration values
  4. Restart OpenVidu Server Pro with ./openvidu restart

Keep an eye on the OpenVidu logs that will automatically display after restarting the service to check that everything went well.



Troubleshooting 🔗

Troubleshooting Master Node 🔗

Configuration errors 🔗

If there's any error with the configuration, a report detailing which configuration property has issues and a step-by-step guide to fix it will be immediately shown by OpenVidu. The report will be similar to this:

Configuration errors
--------------------

* Property OPENVIDU_SECRET is not set. Cannot be empty.
* Property DOMAIN_OR_PUBLIC_IP is not set. Cannot be empty


Fix config errors
---------------

1) Return to shell pressing Ctrl+C
2) Set correct values in '.env' configuration file
3) Restart OpenVidu with:

    $ ./openvidu restart

Docker compose 🔗

To solve any other issue apart from configuration ones, it is important to understand how is OpenVidu service executed.

OpenVidu is executed as a docker-compose file. The commands executed by the script are the standard docker-compose commands, so internally they just do:

  • start
    • $ docker-compose up -d
    • $ docker-compose logs -f openvidu-server
  • stop
    • $ docker-compose down
  • restart
    • $ docker-compose down
    • $ docker-compose up -d
    • $ docker-compose logs -f openvidu-server
  • logs
    • $ docker-compose logs -f openvidu-server


As you can see, logs of openvidu-server service are shown when platform is started or restarted. This log contains the most important information for the OpenVidu execution.

Show service logs 🔗

Take a look to service logs to see what happened.

./openvidu logs

You can also see all service logs together:

docker-compose logs -f

Or you can inspect one by one the other services:

docker-compose logs -f nginx
docker-compose logs -f coturn
docker-compose logs -f redis
docker-compose logs -f app

Review the configuration 🔗

Sometimes we may have a typo when writing a property name. For this reason, openvidu-server prints in the log all the configuration properties you have set in .env file and the default values for all other configuration properties. In that way, you can double check what openvidu-server actually sees.

Configuration properties
---------------------
* CERTIFICATE_TYPE=selfsigned
* OPENVIDU_CDR=false
* OPENVIDU_CDR_PATH=/opt/openvidu/cdr
* DOMAIN_OR_PUBLIC_IP=my.domain.com
* OPENVIDU_RECORDING=false
* OPENVIDU_RECORDING_AUTOSTOP_TIMEOUT=120
* OPENVIDU_RECORDING_COMPOSED_URL=

...

Change log level of the services 🔗

To change the level of openvidu-server logs change the property OV_CE_DEBUG_LEVEL in configuration file .env.

Close ports to avoid external attacks 🔗

Closing all non-necessary ports in your Master Node machine is very important to avoid external attacks. Some administrators using OpenVidu have reported attacks because their ports weren't properly closed. Of course, all of the opened ports stated in Prerequisites section must be accessible from the exterior, but the rest must be closed to grant proper protection.

Typically, cloud providers initiate their machines with all ports closed by default, so usually it is only necessary to open the required ones. For example, you can configure public/private ports in in OpenStack with OpenStack security groups, in AWS with AWS security groups, etc...

If your only choice is to manually configure a firewall, you can for example install in any GNU/LInux system the great UFW (Uncomplicated Firewall) (sudo apt install ufw) with the following configuration to only allow the required ports necessary in your Master Node:

ufw allow ssh
ufw allow 80/tcp
ufw allow 443/tcp
ufw allow 3478/tcp
ufw allow 3478/udp
ufw allow 5044/tcp
ufw allow 9200/tcp
ufw allow 40000:65535/tcp
ufw allow 40000:65535/udp
ufw enable

Generate a report with all deployment information 🔗

If you're having problems deploying Master Node, you can generate a report to take a look to a lot of useful information about it:

./openvidu report

This command will generate a file with this format: openvidu-report-dd-mm-yyyy-hh-mm.txt. You can send this report to our Discourse forum if you find some bug or you need help to deploy your Master Node.

If you have ssh access to your Master Node, you can download this report by executing:

scp USER@OPENVIDU_SERVER_PRO_IP:/opt/kms/openvidu-report-dd-mm-yyyy-hh-mm.txt report.txt

Replace USER with a user of your Master Node with root and use the name of your report file. Replace OPENVIDU_SERVER_PRO_IP with your Master Node ip.

The report contains information about:

  • OpenVidu Installation type
  • OpenVidu version
  • System information (Docker and docker compose version, Linux distribution...)
  • Docker containers running
  • Docker containers logs
  • Configuration files

Troubleshooting Media Nodes 🔗

Show service logs 🔗

Take a look to the Media Node service logs to see what happened.

Media Node Controller logs 🔗

To show all Media Node Controller logs, execute:

./media_node logs

Also you can execute this to follow Media Node Controller logs:

./media_node logs -f
Kurento Media Server logs 🔗

If you want to check Kurento Media Server, you can't do docker-compose logs kms. Kurento Media Server logs are not managed by Docker. They're stored in /opt/openvidu/kurento-logs. If you want to check Kurento logs you can check this directory or execute:

./media_node kms-logs

To follow Kurento Media Server logs in real time you can execute:

./media_node kms-logs -f
Other services logs 🔗

One filebeat container and two metricbeat containers are running in Media Nodes after OpenVidu Pro started all services on it. To access these logs you can just execute:

docker logs <BEAT_CONTAINER_NAME>

Where BEAT_CONTAINER_NAME can be:

  • filebeat-elasticsearch: Service sending kurento logs to Elasticsearch.
  • metricbeat-elasticsearch: Service sending metrics to ElasticSearch.
  • metricbeat-autoscaling: Service sending metrics to OpenVidu for Auto-scaling.

Change log level of the services 🔗

1) In Master Node, update /opt/openvidu/.env file with this environment variable:

KMS_DOCKER_ENV_GST_DEBUG=3,Kurento*:5,kms*:4,sdp*:4,webrtc*:4,*rtpendpoint:4,rtp*handler:4,rtpsynchronizer:4,agnosticbin*:5,kmssdpsession:5

2) Restart OpenVidu:

./openvidu restart

OpenVidu will detect configuration changes and restart Kurento Media Server with this configuration in all your Media Nodes.

For more information about possible values visit Kurento Debug Logging.

Change Kurento Media Server docker image 🔗

OpenVidu and Kurento Media Server evolve at a different pace. Sometimes, it is possible that a new KMS is released but OpenVidu is not still updated. In that case, if you hit a bug that might be solved in the latest KMS version, you can test if just updating KMS fixes your issue.

1) In Master Node, update /opt/openvidu/.env file with this environment variable:

KMS_IMAGE=<DOCKER_IMAGE_NAME>

Where DOCKER_IMAGE_NAME is the docker image you want to use. Location of Kurento Media Server images:

./openvidu restart

OpenVidu will detect configuration changes and restart Kurento Media Server with this configuration in all your Media Nodes.

Close ports to avoid external attacks 🔗

Closing all non-necessary ports in your Media Node machines is very important to avoid external attacks. Some administrators using OpenVidu have reported attacks because their ports weren't properly closed. Of course, all of the opened ports stated in Prerequisites section must be accessible from the exterior, but the rest must be closed to grant proper protection.

Typically, cloud providers initiate their machines with all ports closed by default, so usually it is only necessary to open the required ones. For example, you can configure public/private ports in in OpenStack with OpenStack security groups, in AWS with AWS security groups, etc...

If your only choice is to manually configure a firewall, you can for example install in any GNU/LInux system the great UFW (Uncomplicated Firewall) (sudo apt install ufw) with the following configuration to only allow the required ports necessary in your Media Nodes:

ufw allow ssh
ufw allow 40000:65535/tcp
ufw allow 40000:65535/udp
ufw allow 3000/tcp from <OPENVIDU_SERVER_PRO_IP>
ufw allow 4000/tcp from <OPENVIDU_SERVER_PRO_IP>
ufw allow 8888/tcp from <OPENVIDU_SERVER_PRO_IP>
ufw allow 80/tcp from <OPENVIDU_SERVER_PRO_IP>
ufw enable

Kurento Media Server crash 🔗

Sometimes Kurento Media Server (the service in charge of streaming media inside of Media Nodes) may crash. If this happens on a regular basis, or better, you have isolated a specific use case where KMS always crashes, then perform the following steps to collect a crash report that will help us fix the issue.

You must perform these steps in all of the Media Nodes experiencing KMS crashes. Take into account that all of these instructions must be executed with root permissions:

sudo su


1) Enable KMS crash reporting 🔗
echo "/opt/openvidu/kms-crashes/core_%e_%p_%u_%t" | tee /proc/sys/kernel/core_pattern > /dev/null

Let's prove that crash reporting is properly working. Execute the followings commands:

docker container kill --signal=SIGSEGV $(docker ps --format "{{.Names}}" | grep kms | head -1)
ls -l /opt/openvidu/kms-crashes

If you see something like this ...

-rw------- 1 root root 227282944 abr 17 19:06 core_kurento-media-s_1_0_1587143183
-rw------- 1 root root   3911680 abr 17 19:06 core_kurento-media-s_65_0_1587143183
-rw------- 1 root root 227233792 abr 17 19:06 core_kurento-media-s_66_0_1587143183

... then everything is working as expected. Delete this test report and restart the Media Node services:

rm -rf /opt/openvidu/kms-crashes/*
cd /opt/kms
./media_node restart


2) Replicate the error 🔗

Now is the moment to replicate the KMS error. Use your OpenVidu application until the error appears, and then we'll be ready to collect the crash report.


3) Collect the KMS crash report 🔗

Once the error has occurred, compress the crash report in a core_dumps.tar.gz file and download it. You can do so from your computer with the following SSH commands:

ssh USER@MEDIA_NODE_IP "sudo tar zcvfP ~/core_dumps.tar.gz /opt/openvidu/kms-crashes/*"
scp USER@MEDIA_NODE_IP:~/core_dumps.tar.gz core_dumps.tar.gz

Replace USER with a user of your Media Node instance with root permissions and MEDIA_NODE_IP with the instance IP address. This only applies to a single Media Node. If you have more Media Nodes experiencing KMS crashes, perform these same steps in all of them. Send us the resulting zipped report files.


4) Clean the KMS crash report 🔗

So as not to consume too much hard drive, remember to delete the crash reports once you have sent it to us. IMPORTANT: obviously, do NOT do this before zipping and sending the report.

ssh USER@MEDIA_NODE_IP "sudo rm /opt/openvidu/kms-crashes* && sudo rm ~/core_dumps.tar.gz"

Replace USER with a user of your Media Node instance with root permissions and MEDIA_NODE_IP with the instance IP address. This only applies to a single Media Node and must be performed for each Media Node from which you downloaded a crash report.

Generate a report with all deployment information 🔗

If you're having problems with your Media Node deployment, you can generate a report to take a look to a lot of useful information about it:

./media_node report

This command will generate a file with this format: media-node-report-dd-mm-yyyy-hh-mm.txt. You can send this report to our Discourse forum if you find some bug or you need help to deploy your Media Node.

If you have ssh access to your Media Node, you can download this report by executing:

scp USER@MEDIA_NODE_IP:/opt/kms/media-node-report-dd-mm-yyyy-hh-mm.txt report.txt

Replace USER with a user of your Media Node with root and use the name of your report file. Replace MEDIA_NODE_IP with your Media Node ip.

The report contains information about:

  • OpenVidu Installation type
  • OpenVidu version
  • System information (Docker and docker compose version, Linux distribution...)
  • Docker containers running
  • Docker containers logs
  • Configuration files