Load Crypto Data for 1000s of Symbols in Parallel — with CCXT, Pandas, and Crypto-Pandas
If you’ve ever tried to collect OHLCV data for hundreds of symbols from a crypto exchange, you know how slow and error-prone it can be.
Crypto-Pandas changes that.
Built on top of CCXT and Pandas, Crypto-Pandas is designed to give algorithmic traders a seamless interface to load, process, and analyze market data — all in one line, and all returned immediately as clean Pandas DataFrames.
In this tutorial (our third in the series), we demonstrate how to fetch 1-minute OHLCV data for 1000 trading pairs on Binance — asynchronously, in parallel, and with results ready for use right out of the box.
Why Async + Pandas Matters
Traditional approaches to loading OHLCV data often look like this:
- Loop through each symbol one-by-one.
- Sleep between requests to avoid rate limits.
- Handle retries and timeouts manually.
- Merge results manually afterward.
With asyncio, cctx and crpto-pandas
- All requests are launched concurrently using asyncio.gather.
- Errors and retries are managed under the hood.
- And the result is returned as a single, unified DataFrame — timestamped, labeled, and sorted.
The Code (1000 Symbols in Parallel)
Here’s the exact code from the video:
import asyncio
import time
import pandas as pd
import ccxt.pro as ccxt_pro
from crypto_pandas import AsyncCCXTPandasExchange
# Initialize Binance and wrap it with Crypto-Pandas
exchange = ccxt_pro.binance()
pandas_exchange = AsyncCCXTPandasExchange(exchange=exchange)
async def main():
# Load tradable symbols and limit to 1000
symbols = await pandas_exchange.load_markets()
symbols = symbols.head(1000)
print(symbols[[“symbol”, “base”, “quote”]])
# Prepare tasks to fetch OHLCV data
tasks = [
pandas_exchange.fetch_ohlcv(symbol=symbol, timeframe=”1m”, limit=1000)
for symbol in symbols[“symbol”]
]
# Execute all tasks concurrently
start_time = time.time()
results = await asyncio.gather(*tasks, return_exceptions=True)
await exchange.close()
end_time = time.time()
print(f”Execution time: {end_time – start_time:.2f} seconds”)
# Combine all results into a single DataFrame
results = pd.concat(results)
print(results)
if __name__ == “__main__”:
asyncio.run(main())
Output Format
After ~40 seconds, we get a single dataframe containing the OHLCV for our 1000 symbols:
Execution time: 39.88 seconds
timestamp open high low close volume symbol
0 2025-07-02 17:38:00+00:00 0.02356 0.02356 0.02350 0.02350 181.7848 ETH/BTC
1 2025-07-02 17:39:00+00:00 0.02349 0.02349 0.02348 0.02349 66.2383 ETH/BTC
2 2025-07-02 17:40:00+00:00 0.02349 0.02351 0.02349 0.02349 21.6071 ETH/BTC
3 2025-07-02 17:41:00+00:00 0.02348 0.02349 0.02347 0.02349 45.1281 ETH/BTC
4 2025-07-02 17:42:00+00:00 0.02350 0.02352 0.02350 0.02350 11.4382 ETH/BTC
.. … … … … … … …
995 2025-07-03 10:13:00+00:00 0.02139 0.02139 0.02139 0.02139 0.0000 EGLD/BNB
996 2025-07-03 10:14:00+00:00 0.02139 0.02139 0.02139 0.02139 0.0000 EGLD/BNB
997 2025-07-03 10:15:00+00:00 0.02139 0.02139 0.02139 0.02139 1.0000 EGLD/BNB
998 2025-07-03 10:16:00+00:00 0.02139 0.02139 0.02139 0.02139 0.0000 EGLD/BNB
999 2025-07-03 10:17:00+00:00 0.02139 0.02139 0.02139 0.02139 0.0000 EGLD/BNB
[1000000 rows x 7 columns]
No post-processing required. The data is aligned, typed labeled automatically by Crypto-Pandas.
What You Can Do With This
This async OHLCV pipeline is perfect for:
- Backtesting multi-asset strategies.
- Scanning thousands of pairs for entry signals.
- Initializing large-scale datasets for machine learning models.
Watch the Tutorial
We walk through this exact example step-by-step in our third YouTube video:
▶ Load OHLCV for 1000 Symbols in Parallel (YouTube)
Quick Links
- Try it in Binder otebook : Launch on Binder
- GitHub Repository: github.com/sigma-quantiphi/crypto-pandas
- Official Site: sigmaquantiphi.com
What’s Next?
This is just one feature of Crypto-Pandas. In future videos and articles, we’ll show how to:
- Subscribe to live trades and orderbooks via WebSocket.
- Track and manage positions across multiple exchanges and accounts.
If you’re serious about building scalable, multi-symbol crypto trading systems, Crypto-Pandas is here to help.
