Code for Downloading Historical m1 Candles into a Pandas DataFrame

For those who do not use fxcmpy (or those needing to access more than 10,000 candles at once), here’s a quick way to download historical m1 (or H1 or D1) candles into a Pandas DataFrame:

import datetime
import pandas as pd

url = 'https://candledata.fxcorporate.com/'##This is the base url
periodicity='m1' ##periodicity, can be m1, H1, D1
url_suffix = '.csv.gz' ##Extension of the file name
symbol = 'EURUSD'  #select your symbol

Available Currencies: AUDCAD,AUDCHF,AUDJPY, AUDNZD,CADCHF,EURAUD,EURCHF,EURGBP, EURJPY,EURUSD,GBPCHF,GBPJPY,GBPNZD,GBPUSD,GBPCHF,GBPJPY, GBPNZD,NZDCAD,NZDCHF.NZDJPY,NZDUSD,USDCAD,USDCHF,USDJPY

The candle files are stored in compressed csv. The storage structure comes as {periodicity}/{symbol}/{year}/{week_of_year}.csv.gz
The first week of the year will be 1.csv.gz where the last week might be 52 or 53. That will depend on the year. Once we have the week of the year we will be able to pull the correct file with the data that is needed.

start_dt =  datetime.date(2018,1,2)##select start date
end_dt = datetime.date(2018,2,2)##select end date

start_wk = start_dt.isocalendar()[1]##find the week of the year for the start  
end_wk = end_dt.isocalendar()[1] ##find the week of the year for the end 
year = str(start_dt.isocalendar()[0]) ##pull out the year of the start

The URL is a combination of the currency, periodicity, year, and week of the year.
Example URL: https://candledata.fxcorporate.com/m1/EURUSD/2017/29.csv.gz
The example URL should be the first URL of this example. First create the empty dataframe which we will store the candles in:

data=pd.DataFrame()

This will loop through the weeks needed, create the correct URL and print out the length of the file.

for i in range(start_wk, end_wk ):
        url_data = url + periodicity + '/' + symbol + '/' + year + '/' + str(i) + url_suffix
        print(url_data)
        tempdata = pd.read_csv(url_data, compression='gzip', index_col='DateTime', parse_dates=True)
        data=pd.concat([data, tempdata])
    print(data)

For more information, see our Github page or leave a reply.


Full code:

import datetime
import pandas as pd

url = 'https://candledata.fxcorporate.com/'##This is the base url
periodicity='m1' ##periodicity, can be m1, H1, D1
url_suffix = '.csv.gz' ##Extension of the file name
symbol = 'EURUSD'  #select your symbol
start_dt =  datetime.date(2018,1,2)##select start date
end_dt = datetime.date(2018,2,2)##select end date

start_wk = start_dt.isocalendar()[1]##find the week of the year for the start  
end_wk = end_dt.isocalendar()[1] ##find the week of the year for the end 
year = str(start_dt.isocalendar()[0]) ##pull out the year of the start

data=pd.DataFrame()

for i in range(start_wk, end_wk ):
            url_data = url + periodicity + '/' + symbol + '/' + year + '/' + str(i) + url_suffix
            print(url_data)
            tempdata = pd.read_csv(url_data, compression='gzip', index_col='DateTime', parse_dates=True)
            data=pd.concat([data, tempdata])
print(data)
2 Likes

hello Liza
I don`t get to work in periodicity D1 when in m1 and 1H work perfectly.

What could be the problem
thank you in advance

Hi @franlm1, can you send the code you’re using as well as the error message you’re getting?

the code is the same which you wrote in Quant Community

I´ve attached Screenshots where you can see the error