WebSocket API v1 - unexpected disconnections from market data feeds

Our WebSocket API v1 is intended to provide a persistent source of our market data (tickers, order books, time and sales, etc.), but sometimes WebSocket connections are terminated unexpectedly by Cloudflare.

Cloudflare initiated disconnections usually result in networking errors, such as the following 1006 error:

Error reading from websocket: close 1006 (abnormal closure): unexpected EOF

or the following where Python throws an exception when trying to read a new market data message from an already closed connection:

websocket._exceptions.WebSocketConnectionClosedException: socket is already closed

While there is no solution that prevents these type of unexpected disconnections, there are some very effective work arounds. In the event that you experience frequent WebSocket disconnections, we recommend implementing one or more of the following:

Alternative IP address

Cloudflare's security precautions are often applied to specific IP addresses, groups of similar IP addresses (such as a class C block) or certain geographical locations, hence changing your IP address is sometimes a simple but effective solution.

For example, hosting your API software in a different AWS region or using a VPN in a different geographical location, would both significantly change the local IP address and potentially resolve the unexpected disconnection issue.

Reconnection logic

Using a single WebSocket connection that is monitored for any unintended disconnections and reconnected and resubscribed as necessary is another simple but effective solution. The following is basic example Python code showing how to reconnect and resubscribe a WebSocket connection whenever a WebSocketConnectionClosed exception is thrown:

#!/usr/bin/env python3 import sys from websocket import create_connection ws = create_connection('wss://ws.kraken.com/') print(ws.recv()) ws.send('{"event":"subscribe", "subscription":{"name":"ticker"}, "pair":["XBT/USD"]}') print(ws.recv()) for count in range(100): if count == 30 or count == 60: ws.close() try: print(ws.recv()) except: ws = create_connection('wss://ws.kraken.com/') print(ws.recv()) ws.send('{"event":"subscribe", "subscription":{"name":"ticker"}, "pair":["XBT/USD"]}') print(ws.recv()) ws.close() sys.exit(0)

As shown in the following sample output, the WebSocket connection and subscription are renewed whenever the connection is unexpectedly terminated (at 30 and 60 iterations of the loop):

Redundant connections

Using multiple (two or more) redundant WebSocket connections is usually the most effective solution to unexpected WebSocket disconnections, because it allows the market data feeds to continue uninterrupted regardless of how frequently the underlying connections are terminated.

The following basic example Python code shows how to implement multiple WebSocket connections with automatic failover:

#!/usr/bin/env python3 import sys from websocket import create_connection ws = [ None, None ] data = [ "", "" ] ws[0] = create_connection("wss://ws.kraken.com/") print("WebSocket (Primary): %s" % ws[0].recv()) ws[1] = create_connection("wss://ws.kraken.com/") print("WebSocket (Backup): %s" % ws[1].recv()) ws[0].send('{"event":"subscribe", "subscription":{"name":"spread"}, "pair":["XBT/USD"]}') print("WebSocket (Primary): %s" % ws[0].recv()) ws[1].send('{"event":"subscribe", "subscription":{"name":"spread"}, "pair":["XBT/USD"]}') print("WebSocket (Backup): %s" % ws[1].recv()) source = 0 for count in range(100): try: try: data[0] = ws[0].recv() except Exception: source = 1 try: data[1] = ws[1].recv() except Exception: source = 0 except KeyboardInterrupt: ws[0].close() print("WebSocket (%(source)s): %(data)s" % {"source":"Primary" if source == 0 else "Backup", "data":data[source]}) ws[source].close() sys.exit(1)

The above code creates two WebSocket connections and subscribes both connections to the same market data feed. The primary connection is used to output the market data, until a keyboard interrupt is generated (via Ctrl+C, for example), which closes the primary connection. Subsequent reads from the primary connection throw an exception, which is caught and used to change the market data source to the backup connection, which is then used to output the market data.

The following sample output shows how the market data source changes from the primary connection to the backup connection without interrupting the market data feed:

WebSocket (Primary): {"connectionID":6894434610526943167,"event":"systemStatus","status":"online","version":"1.0.1"} WebSocket (Backup): {"connectionID":11520162761468018366,"event":"systemStatus","status":"online","version":"1.0.1"} WebSocket (Primary): {"channelID":275,"channelName":"spread","event":"subscriptionStatus","pair":"XBT/USD","status":"subscribed","subscription":{"name":"spread"}} WebSocket (Backup): {"channelID":275,"channelName":"spread","event":"subscriptionStatus","pair":"XBT/USD","status":"subscribed","subscription":{"name":"spread"}} WebSocket (Primary): [275,["9674.90000","9675.00000","1591440118.120752","10.88476562","3.47935600"],"spread","XBT/USD"] WebSocket (Primary): {"event":"heartbeat"} WebSocket (Primary): {"event":"heartbeat"} WebSocket (Primary): [275,["9674.90000","9675.00000","1591440127.108830","10.98812533","3.47935600"],"spread","XBT/USD"] WebSocket (Primary): {"event":"heartbeat"} WebSocket (Primary): [275,["9674.90000","9675.00000","1591440129.406073","1.87412533","7.47935600"],"spread","XBT/USD"] WebSocket (Primary): [275,["9674.90000","9675.00000","1591440129.505372","0.77412533","7.47935600"],"spread","XBT/USD"] WebSocket (Primary): [275,["9674.90000","9675.00000","1591440129.572658","0.10335971","7.47935600"],"spread","XBT/USD"] WebSocket (Primary): [275,["9674.90000","9675.00000","1591440129.572658","0.10335971","7.47935600"],"spread","XBT/USD"] ^C WebSocket (Backup): [275,["9674.90000","9675.00000","1591440130.156840","0.10335971","8.27935600"],"spread","XBT/USD"] WebSocket (Backup): [275,["9674.90000","9675.00000","1591440130.195899","0.10335971","8.47435600"],"spread","XBT/USD"] WebSocket (Backup): [275,["9673.80000","9675.00000","1591440130.459388","0.10335971","8.47435600"],"spread","XBT/USD"] WebSocket (Backup): [275,["9672.60000","9675.00000","1591440130.195899","0.00516924","8.47435600"],"spread","XBT/USD"] WebSocket (Backup): [275,["9673.80000","9675.00000","1591440130.764856","0.00400000","8.47435600"],"spread","XBT/USD"] WebSocket (Backup): [275,["9673.80000","9674.90000","1591440130.780514","0.00400000","0.25000000"],"spread","XBT/USD"] WebSocket (Backup): {"event":"heartbeat"} WebSocket (Backup): [275,["9673.80000","9674.90000","1591440132.433940","4.00400000","0.25000000"],"spread","XBT/USD"] WebSocket (Backup): [275,["9673.80000","9674.90000","1591440132.519509","4.00000000","0.25000000"],"spread","XBT/USD"] WebSocket (Backup): [275,["9673.90000","9674.90000","1591440132.527887","0.10336612","0.25000000"],"spread","XBT/USD"] WebSocket (Backup): [275,["9673.90000","9675.00000","1591440132.527887","0.10336612","4.27935600"],"spread","XBT/USD"]

Note that redundant WebSocket connections is the most complex of the three solutions, but it is the optimal solution with regard to reliability.