submitted by jtimperio to DataHoarder [link] [comments]
Intro:In my exploration of the world of big data and I became curious about tick data. Unfortunately, market data is almost always behind a paywall or de-sampled to the point of uselessness. After discovering the Dukascopy API, I knew I wanted to make this data available for all in a more accessible format. Over the course of a few months, I downloaded, cleaned, parsed, and compressed over a decade of Forex tick data on 37 currency pairs and commodities. Today I am happy to finally release the final result of my work to the DataHoarder community!
Download Links:Warning: I have rented a seedbox for the next 3 months from seedbox.io but I have been having some issues. If you have any issues with the torrent please leave a comment. Also, PLEASE SEED when you are done. This is quite a large data set and I can only push so much data on my own.
Torrent File: https://drive.google.com/file/d/18ymZWeFLJK7FggK_iiWZ-TxgWIVdJVvv/view?usp=sharingCompanion Blog Post: https://www.driftinginrecursion.com/post/dukascopy_opensource_data/
About the Data:The data was collected from https://www.dukascopy.com/ via a public API that allows for the download of tick data on the hour level. These files come in the form of a .bi5 file. The data starts as early as 2004 all the way to 2019.
These files were decompressed, then merged into yearly CSV’s named in the following convention. “AUDCHF_tick_UTC+0_00_2011.csv” or ‘Pair_Resolution_Timezone_Year.csv’
These CSV’s are split into 3 categories “Majors”, “Crosses”, “Commodities”.
Majors, Crosses, and Commodities have had their timestamps modified so that they are in the official UTC ISO standard. This was originally done for a Postgresql database that quickly became obsolesced. Any files that have been modified are appended with a “-Parse”. These timestamps have been modified in the following format.
Millisecond timestamps to UTC +00:00 time [2017.01.01 22:37:08.014] -- [2017-01-01T22:37:08.014+00:00]
User Resources:For those looking to use this data in a live context or update it frequently, I have included a number of tools for both Windows and Linux that will be useful.
WindowsThe ~/dukascopy/resources/windows folder contains a third party tool written in java that can download and convert Dukascopy’s .bi5 files. I have also included the latest zstd binaries from Zstandard Github page.
LinuxLinux is my daily driver in 99% of cases, so I have developed all my scraping tools using Linux only tools. In the ~/dukascopy/resources/linux folder you will find a number of shell script and pyhton3 files that I used to collect this data. There are quite a few files in this directory but I will cover the core ones below.
download-day.py:This file is used to download a single symbol for a single day and then convert and merge all 24 .bi5 files into a single CSV.
download-year.pyThis file is used to download a single symbol for a full year and then convert and merge all .bi5 files into a single CSV.
dukascopy.pyThis file contains all the core logic for downloading and converting data from dukascopy.
utc-timestamp-convert.pyThis tad slow but works well enough. It requires the pandas project and parses timestamps into the UTC ISO standard. This is useful for those looking to maintain the format of new files with the those in this repo, or those looking to use this in a SQL database.
週四 (27 日) 全球股災氾濫，華爾街延續拋售，恐慌指數 VIX 大漲近 35%，原油價格暴跌至每桶 47 美元以下，十年期美債再創歷史新低。submitted by eagleocean to saraba1st [link] [comments]
美股道瓊、那指、標普陷入 10% 的修正，道瓊歷經史上最大的單日跌幅，超過週一跌幅 1031 點，標普 500 自去年 10 月以來首次收於 3,000 點下方，那指迎來 2011 年 8 月以來最糟的單日表現。
儘管美國總統川普週三 (26 日) 召開武漢肺炎記者會，試圖緩解大眾的擔憂，但美國疾病控制與預防中心 (CDC) 證實，加州首度出現未知感染源案例，警告美國恐怕已出現社區傳播，華爾街恐慌持續蔓延。
中國境外伊朗、義大利和韓國疫情嚴峻，疫情強襲伊朗政壇，伊朗副總統艾伯特卡 (Masoumeh Ebtekar) 證實罹患武漢肺炎。世界衛生組織 (WHO) 表示，武漢肺炎已達決定性時刻，並且有爆發全球大流行的可能。
週四 (27 日) 美股四大指數表現：
道瓊 30 檔成分股虧損不堪：埃克森美孚 (XOM-US) 下跌 6.02%；波音 (BA-US) 下跌 5.83%；P& G (PG-US) 下跌 5.54%；思科 (CSCO-US) 下跌 5.03%；美國運通 (AXP-US) 下跌 4.80%；IBM (IBM-US) 下跌 4.75%；高盛 (GS-US) 下跌 4.68%。
費半成份股遭受重創：AMD (AMD-US) 下跌 7.33%；英特爾 (INTC-US) 下跌 6.40%；Nvidia (NVDA-US) 下跌 5.57%；應用材料 (AMAT-US) 下跌 4.70%；高通 (QCOM-US) 下跌 4.67%。
武漢肺炎 (COVID-19) 疫情持續延燒，微軟週三盤後發布財測警告，稱「更多個人電腦」部門的營收財測將無法達陣，Windows OEM 和 Surface 受到的負面影響比之前預期的要大，拖累微軟週四股價暴跌 7.05%。
週四 Facebook 因擔憂武漢病毒疫情，宣布取消年度盛事「F8 開發者大會」，其股價下跌 3.78%。
疫情爆發導致全球各地口罩貨源緊張，知名口罩生產商 3M (MMM-US) 股價逆勢上漲 0.82%。
美國銀行表示，世界經濟正處於自金融危機 (2009 年) 以來最疲軟的一年，預測今年全球經濟增長將下滑至 2.8%，而非此前預期的 3.1%，經濟將呈 U 形復甦，而非出現 V 型反彈。
高盛首席美國股票策略師 David Kostin 週四給客戶的一份報告表示，預測 2020 年美國企業盈利增長將為零，這是基於更新的營收模型並考慮病毒廣泛傳播的可能性。
Ascent Private Capital Management 全球投資策略師 Tom Hainlin 表示：「短期內我們非常謹慎，似乎沒有人真的是冠狀病毒專家。在我們的投資生涯中，從未見過這樣的情況。」
MUFG Union Bank 首席金融分析師 Chris Rupkey 稱：「股市和債市表明，我們在劫難逃。任何對未來有更好想法的人請告訴我們，因為現在經濟朝向直線下滑。」
The Foreign Exchange (Forex) Market Explained: Dynamics, Participants and Transaction Methods 1. A graphical translation of “Foreign Exchange Market Structure, Players and Evolution”, by Michael R. King, Carol Osler and Dagfinn Rime, last revised on August 14, 2011 NoBrainerTrades.com 2. The last quarter of 2011 was as quiet as the second one — the most volatility during 2011 was experienced in Q1 (Japan’s Tsunami) and Q3 (Euro Zone’s instability). In Japan especially, volumes were relatively low as the Yen traded in a narrow band, becoming unattractive to day traders. FXCM further increased its position as the world’s largest forex broker, … Overview of MACD; Relative Strength Index; Japanese Candlesticks; Moving Averages; Fibonacci Ratios; Two-Bar Pattern; Market Myths Exposed; Contact Us; Month: November 2011 Daily Forex Analysis – November 30, 2011 by ForexCycle.com November 30, 2011 01:49. USDCHF Analysis. USDCHF had formed a cycle top at 0.9329 on 4-hour chart. Range trading between 0.9085 and 0.9329 would likely be seen ... Overview of the Forex market. Make profits on the foreign exchange market. Access the most rewarding market with JustForex! trade on Forex online today! FOREX.com offers forex & metals trading with award winning trading platforms, tight spreads, quality executions, powerful trading tools & 24-hour live support Forex Market; Forex Trading; Technical Analysis; Chart Pattern; Trading Strategy; Forex Brokers. Avatrade; Broker News; Broker Review; Elliott Wave Courses. Markets Reverse; 5 Ways Using Trendlines; Overview of MACD; Relative Strength Index; Japanese Candlesticks; Moving Averages ; Fibonacci Ratios; Two-Bar Pattern; Market Myths Exposed; Contact Us; Daily Forex Analysis – February 25, 2011 ... UUP Remains Bullish In Cycles from May 2011. Firstly the dollar tracking ETF fund UUP inception date was 2/20/2007. Interestingly the dollar index has a low in March 2008. The UUP ETF fund shows a low in May 2011. The dollar index did make a pullback cycle low in May 2011 however it was well above the March 2008 lows. The bullish cycle up from ... Forex trading involves risk. Losses can exceed deposits. Losses can exceed deposits. We recommend that you seek independent advice and ensure you fully understand the risks involved before trading. Stock Market and Forex Relationships: How a Stock Move Translates to Currency Trades 2011-10-17 11:38:00 Jeremy Wagner, Lead Trading Instructor , Forex Market Overview Long Term Trends Today's Price Surprises Highs & Lows Forex Market Map Performance Leaders Currency Converter . Crypto Market Pulse . Crypto Market Overview Performance Leaders Market Capitalizations Bitcoin Futures. Trading Signals. New Recommendations. Commitment of Traders. Commitment of Traders Legacy Report Financial TFF Report. Currency Groupings. Popular Cross ...
[index]          
Market Overview Part 2 UUP, FXE, TNA. 10 Best Side Hustle Ideas: How I Made $600 in One Day - Duration: 16:07. Let's Talk Money! with Joseph Hogue, CFA Recommended for you For the Love of Physics - Walter Lewin - May 16, 2011 - Duration: 1:01:26. ... 95% Winning Forex Trading Formula - Beat The Market Maker 📈 - Duration: 37:53. TRADE ATS 902,827 views. 37:53 ... Forex Trendy - Best Trend Scanner (LIVE) How To Identify Price Patterns for Trade Entry in Forex? 4. How To Identify Stock Market Direction (Trends) Part 1 Forex Trading, Triangle Pattern, in Real ... This video is unavailable. Watch Queue Queue. Watch Queue Queue FXStreet is the leading independent portal dedicated to the Foreign Exchange (Forex) market. On this channel you'll find extracts of webinars, demos and inte... Published on Nov 29, 2011. Advice from planning expert Tim Berry: Out of the whole world, which smaller groups are you targeting as your potential customers? You want to paint a picture of the ... IFS+ is a comprehensive software solution which allows to manage brokerage activities on the Forex market. IFS+ system is packed with full set of brokerage t...