A problem that accompanied me for more than half a year, I spent evenings with error analysis and troubleshooting. After some update my Wemos D1 Mini (ESP8266) could not establish a TLS encrypted connection to the Mosquitto MQTT broker.

Error message in PlatformIO’s Serial Monitor:

Connection to MQTT broker failed with state failed with state -2 

Error message in the log /var/log/mosquitto.log of Mosquitto MQTT Broker:

New connection from 87.152.151.230 on port 8883. 
Socket error on client <unknown>, disconnecting.

First I searched in the reference of the PubSubClient, tried several previous versions here, because I suspected the error here first. https://pubsubclient.knolleary.net/api.html

-2 : MQTT_CONNECT_FAILED - the network connection failed 

Solutions on Mosquitto MQTT Broker

https://mosquitto.org/man/mosquitto-conf-5.html
The first is require_certificate, which may be set to true or false. If false, the SSL/TLS component of the client will verify the server but there is no requirement for the client to provide anything for the server: authentication is limited to the MQTT built in username/password.

/etc/mosquitto/mosquitto.conf

listener 8883 
certfile /etc/mosquitto/certs/mqtt_server.crt 
cafile /etc/mosquitto/certs/ca.crt 
keyfile /etc/mosquitto/certs/mqtt_server.key 
require_certificate false

Solutions on Arduino

Adding the right libraries

#include <ESP8266WiFi.h> 
#include <WiFiClientSecure.h> 

Increasing the MQTT max packet size

Sets the largest packet size, in bytes, the client will handle. Any packet received that exceeds this size will be ignored. Default: 128 bytes

#define MQTT_MAX_PACKET_SIZE 256 

Changing the Wifi mode

Add the following statement before the actual WiFi connect takes place.

WiFi.mode(WIFI_STA); 

Using the WifiClientSecure instead of the WifiClient

WiFiClientSecure

Make sure ESP8266 is set to 160MHz

platformio.ini

[env:d1_mini] 
board_build.f_cpu = 160000000L 

Reading out the TLS Buffer

To find out what happens during the TLS handshake you should read the buffer

Serial.print("Connection to MQTT broker failed with state: ");
Serial.println(client.state()); 
char puffer[100];
espClient.getLastSSLError(puffer,sizeof(puffer));
Serial.print("TLS connection failed with state: ");
Serial.println(puffer); 

Here I finally got the following error message, which led me to the actual solution of the problem.

Chain could not be linked to a trust anchor 

Add espClient.setInsecure(); to the setup section

If you are not afraid of DNS attacks and want to disable CA validation, this statement will ultimately lead to success.

void setup() {
Serial.begin(115200);
espClient.setInsecure();
reconnect(); 

For the sake of completeness, the PubSubClient also offers the possibility of integrating the certificate of the Certification Authority (CA) by which a higher security level can be achieved.