Merge pull request #2012 from DerekMelchin/bug-restructure-wriiting-a…

…lgo-history-requests

Add draft for new historical data structure

08 Drafts/05 Historical Data/01 Getting Started/01 History Requests.html

@@ -0,0 +1,18 @@
+<p>Follow these steps to get some historical data:</p>
+
+<ol>
+    <li>Create a new project.</li>
+    <p>The process to creating a new project depends on if you use the <a href='/docs/v2/cloud-platform/projects/getting-started#03-Create-Projects'>Cloud Platform</a>, <a href='/docs/v2/local-platform/projects/getting-started#03-Create-Projects'>Local Platform</a>, or <a href='/docs/v2/lean-cli/projects/project-management#02-Create-Projects'>CLI</a>.</p>
+
+    <li>In the <code class='python'>initialize</code><code class='csharp'>Initialize</code> method, <a href='/docs/v2/writing-algorithms/securities/requesting-data'>add an asset</a>.</li>
+    <div class="section-example-container">
+        <pre class="csharp">var spy = AddEquity("SPY");</pre>
+        <pre class="python">spy = self.add_equity('SPY')</pre>
+    </div>
+
+    <li>Call the <code class='python'>history</code><code class='csharp'>History</code> method with the asset's <a href='/docs/v2/writing-algorithms/key-concepts/security-identifiers'>Symbol</a> and a lookback period.</li>
+    <div class="section-example-container">
+        <pre class="csharp">var history = History(spy.Symbol, 10, Resolution.Daily);</pre>
+        <pre class="python">history = self.history(spy.symbol, 10, Resolution.DAILY)</pre>
+    </div>
+</ol>

08 Drafts/05 Historical Data/01 Getting Started/02 Warm Up Periods.html

@@ -0,0 +1,12 @@
+<p>Follow these steps to add a warm-up period to the start of your algorithm:</p>
+
+<ol>
+    <li>Create a new project.</li>
+    <p>The process to creating a new project depends on if you use the <a href='/docs/v2/cloud-platform/projects/getting-started#03-Create-Projects'>Cloud Platform</a>, <a href='/docs/v2/local-platform/projects/getting-started#03-Create-Projects'>Local Platform</a>, or <a href='/docs/v2/lean-cli/projects/project-management#02-Create-Projects'>CLI</a>.</p>
+
+    <li>In the <code class='python'>initialize</code><code class='csharp'>Initialize</code> method, call the <code class='python'>set_warm_up</code><code class='csharp'>SetWarmUp</code> method with the warm-up duration.</li>
+    <div class="section-example-container">
+        <pre class="csharp">SetWarmUp(10, Resolution.Daily);</pre>
+        <pre class="python">self.set_warm_up(10, Resolution.DAILY)</pre>
+    </div>
+</ol>

08 Drafts/05 Historical Data/01 Getting Started/metadata.json

@@ -0,0 +1,12 @@
+{
+    "type": "metadata",
+    "values": {
+        "description": "",
+        "keywords": "",
+        "og:description": "",
+        "og:title": "Getting Started - Documentation QuantConnect.com",
+        "og:type": "website",
+        "og:site_name": "Getting Started - QuantConnect.com",
+        "og:image": "https://cdn.quantconnect.com/docs/i/writing-algorithms/historical-data/___________.png"
+    }
+}

08 Drafts/05 Historical Data/02 History Requests/metadata.json

@@ -0,0 +1,12 @@
+{
+    "type": "metadata",
+    "values": {
+        "description": "You can use a direct historical data request at any time throughout your algorithm. It returns all of the data you request as a single object.",
+        "keywords": "historical data, trade data, quote data, custom data, live trading",
+        "og:description": "You can use a direct historical data request at any time throughout your algorithm. It returns all of the data you request as a single object.",
+        "og:title": "History Requests - Documentation QuantConnect.com",
+        "og:type": "website",
+        "og:site_name": "History Requests - QuantConnect.com",
+        "og:image": "https://cdn.quantconnect.com/docs/i/writing-algorithms/historical-data/history-requests.png"
+    }
+}

08 Drafts/05 Historical Data/03 History Responses/metadata.json

@@ -0,0 +1,12 @@
+{
+    "type": "metadata",
+    "values": {
+        "description": "",
+        "keywords": "",
+        "og:description": "",
+        "og:title": "Getting Started - Documentation QuantConnect.com",
+        "og:type": "website",
+        "og:site_name": "Getting Started - QuantConnect.com",
+        "og:image": "https://cdn.quantconnect.com/docs/i/writing-algorithms/historical-data/___________.png"
+    }
+}

08 Drafts/05 Historical Data/04 Asset Classes/01 US Equities/metadata.json

@@ -0,0 +1,12 @@
+{
+    "type": "metadata",
+    "values": {
+        "description": "",
+        "keywords": "",
+        "og:description": "",
+        "og:title": "Getting Started - Documentation QuantConnect.com",
+        "og:type": "website",
+        "og:site_name": "Getting Started - QuantConnect.com",
+        "og:image": "https://cdn.quantconnect.com/docs/i/writing-algorithms/historical-data/___________.png"
+    }
+}

08 Drafts/05 Historical Data/05 Alternative Data/metadata.json

@@ -0,0 +1,12 @@
+{
+    "type": "metadata",
+    "values": {
+        "description": "",
+        "keywords": "",
+        "og:description": "",
+        "og:title": "Getting Started - Documentation QuantConnect.com",
+        "og:type": "website",
+        "og:site_name": "Getting Started - QuantConnect.com",
+        "og:image": "https://cdn.quantconnect.com/docs/i/writing-algorithms/historical-data/___________.png"
+    }
+}

08 Drafts/05 Historical Data/06 Custom Data/metadata.json

@@ -0,0 +1,12 @@
+{
+    "type": "metadata",
+    "values": {
+        "description": "",
+        "keywords": "",
+        "og:description": "",
+        "og:title": "Getting Started - Documentation QuantConnect.com",
+        "og:type": "website",
+        "og:site_name": "Getting Started - QuantConnect.com",
+        "og:image": "https://cdn.quantconnect.com/docs/i/writing-algorithms/historical-data/___________.png"
+    }
+}

08 Drafts/05 Historical Data/07 Warm Up Periods/01 Introduction.php

@@ -0,0 +1 @@
+<?php echo file_get_contents(DOCS_RESOURCES."/datasets/warm-up-introduction.html"); ?>

08 Drafts/05 Historical Data/07 Warm Up Periods/02 Set Warm Up Periods.html

@@ -0,0 +1,70 @@
+<p>
+You can set a warm-up period based on a period of time or a number of bars. To set a warm-up, in your algorithm's <code class="csharp">Initialize</code><code class="python">initialize</code> method, call the <code class="csharp">SetWarmUp</code><code class="python">set_warm_up</code> method.</p>
+
+<div class="section-example-container">
+<pre class="csharp">// Wind time back 7 days from start
+SetWarmUp(TimeSpan.FromDays(7));
+
+// Feed in 100 bars before start date
+SetWarmUp(100);
+</pre>
+<pre class="python"># Wind time back 7 days from start
+self.set_warm_up(timedelta(7))
+
+# Feed in 100 bars before start date
+self.set_warm_up(100)
+</pre>
+</div>
+
+<p>If you pass a <code class="python">timedelta</code><code class="csharp">TimeSpan</code> argument, the warm-up period consists of that <code class="python">timedelta</code><code class="csharp">TimeSpan</code> before the start date and pipes in data that matches the resolution of your data subscriptions.</p>
+
+<p>If you pass just an integer argument, LEAN calculates the start of the warm-up period for each of your security subscriptions by using the number of bars you specify, the resolution of each security, and the trading calendar of each security. After it calculates the start time of the warm-up period for each security, it sets the earliest start time as the start of the algorithm warm-up period. For instance, in the following example, the warm-up period consists of 100 minute resolution bars for SPY and as many second resolution bars for BTCUSD that occur from the start of the SPY warm-up period to the algorithm <code class="csharp">StartDate</code><code class="python">start_date</code>.</p>
+
+<div class="section-example-container">
+<pre class="csharp">AddEquity("SPY", Resolution.Minute, fillForward: false);
+AddCrypto("BTCUSD", Resolution.Second, fillForward: false);
+SetWarmUp(100);
+</pre>
+<pre class="python">self.add_equity("SPY", Resolution.MINUTE, fill_forward=False)
+self.add_crypto("BTCUSD", Resolution.SECOND, fill_forward=False)
+self.set_warm_up(100)
+</pre>
+</div>
+
+<p>To use a specific resolution of data for the warm-up period, pass a <code>resolution</code>&nbsp;argument to the <code class="csharp">SetWarmUp</code><code class="python">set_warm_up</code> method.</p>
+
+<div class="section-example-container">
+<pre class="csharp">// Feed in 100 days of daily data before the start date
+SetWarmUp(TimeSpan.FromDays(100), Resolution.Daily);
+
+// Feed in data for 100 trading days of daily data before the start date
+SetWarmUp(100, Resolution.Daily);</pre>
+<pre class="python"># Feed in 100 days of daily data before the start date
+self.set_warm_up(timedelta(days=100), Resolution.DAILY)
+
+# Feed in data for 100 trading days before the start date
+self.set_warm_up(100, Resolution.DAILY)</pre>
+</div>
+
+<p>If you pass a <code class="python">timedelta</code><code class="csharp">TimeSpan</code> and a resolution, the warm-up period consists of the time period and resolution you set, regardless of the resolution of your data subscriptions.</p>
+
+<p>If you pass an integer and a resolution, the warm-up period consists of the number of bars and resolution you set, regardless of the resolution of your data subscriptions. In this case,&nbsp;<span style="font-size: 15px;">LEAN calculates the start of the warm-up period for each of your security subscriptions just like it does when you only pass an integer argument. After it calculates the start time of the warm-up period for each security, it sets the earliest start time as the start of the algorithm warm-up period.</span></p><p>If you pass a resolution argument and you registered indicators or consolidators for automatic updates, the warm-up period resolution must be less than or equal to the lowest resolution of your automatic indicators and consolidators. For instance, in the following example, the indicators use minute resolution data, so you can set the warm-up period to use tick, second, or minute resolution data.</p>
+
+<div class="section-example-container">
+<pre class="csharp">_spy = AddEquity("SPY", Resolution.Minute, fillForward: false).Symbol;
+_btc = AddCrypto("BTCUSD", Resolution.Second, fillForward: false).Symbol;
+
+_spySma = SMA(_spy, 10);
+_btcSMA = SMA(_btc, 10, Resolution.Minute);
+
+SetWarmUp(100, Resolution.Minute);
+</pre>
+<pre class="python">self._spy = self.add_equity("SPY", Resolution.MINUTE, fill_forward=False).symbol
+self._btc = self.add_crypto("BTCUSD", Resolution.SECOND, fill_forward=False).symbol
+
+self._spy_sma = self.sma(self._spy, 10)
+self._btc_sma = self.sma(self._btc, 10, Resolution.MINUTE)
+
+self.set_warm_up(100, Resolution.MINUTE)
+</pre>
+</div>

08 Drafts/05 Historical Data/07 Warm Up Periods/03 Warm Up Vs Reality.html

@@ -0,0 +1,19 @@
+<p>You may need to distinguish warm-up data from real data. To check if the algorithm is currently in a warm up period, use the <code class="csharp">IsWarmingUp</code><code class="python">is_warming_up</code> property.</p>
+<div class="section-example-container">
+<pre class="csharp">// In Initialize
+var emaFast = EMA("IBM", 50);
+var emaSlow = EMA("IBM", 100);
+SetWarmup(100);
+
+// In OnData: Don't run if the indicators aren't ready
+if (IsWarmingUp) return;
+</pre>
+<pre class="python"># In Initialize
+self._ema_fast = self.ema("IBM", 50);
+self._ema_slow = self.ema("IBM", 100);
+self.set_warmup(100);
+
+// In OnData: Don't run if the indicators aren't ready
+if self.is_warming_up: return
+</pre>
+</div>

08 Drafts/05 Historical Data/07 Warm Up Periods/04 Event Handler.html

@@ -0,0 +1,11 @@
+
+<p>The <code class="csharp">OnWarmupFinished</code><code class="python">on_warmup_finished</code> event handler executes when the warm up period ends, even if you don't set a warm up period.<br></p>
+<div class="section-example-container">
+<pre class="csharp">public override void OnWarmupFinished()
+{
+    // Done warming up
+}
+</pre>
+<pre class="python">def on_warmup_finished(self) -&gt; None:
+    pass # Done warming up</pre>
+</div>

08 Drafts/05 Historical Data/07 Warm Up Periods/05 Limitations.html

@@ -0,0 +1 @@
+<p>You can't place trades during the warm-up period because they would be operating on fictional fast-forwarded data.</p>

08 Drafts/05 Historical Data/07 Warm Up Periods/99 Examples.html

@@ -0,0 +1,11 @@
+  <div class="example-fieldset"> 
+    <div class="example-legend">Demonstration Algorithms</div>
+
+    <a class="python example-algorithm-link" href="https://github.com/QuantConnect/Lean/blob/master/Algorithm.Python/WarmupAlgorithm.py" target="_BLANK">
+        WarmupAlgorithm.py <span class="badge-python pull-right">Python</span>
+    </a>
+
+    <a class="csharp example-algorithm-link" href="https://github.com/QuantConnect/Lean/blob/master/Algorithm.CSharp/WarmupAlgorithm.cs" target="_BLANK">
+        WarmupAlgorithm.cs <span class="badge-csharp pull-right">C#</span>
+    </a>
+  </div>

08 Drafts/05 Historical Data/07 Warm Up Periods/metadata.json

@@ -0,0 +1,12 @@
+{
+    "type": "metadata",
+    "values": {
+        "description": "LEAN supports an automated fast-forward system called \"Warm Up\". It simulates winding back the clock from the time you deploy the algorithm.",
+        "keywords": "historical data, warm up, algorithmic trading",
+        "og:description": "LEAN supports an automated fast-forward system called \"Warm Up\". It simulates winding back the clock from the time you deploy the algorithm.",
+        "og:title": "Warm Up Periods - Documentation QuantConnect.com",
+        "og:type": "website",
+        "og:site_name": "Warm Up Periods - QuantConnect.com",
+        "og:image": "https://cdn.quantconnect.com/docs/i/writing-algorithms/historical-data/warm-up-periods.png"
+    }
+}

08 Drafts/05 Historical Data/08 Rolling Window/01 Introduction.php

@@ -0,0 +1 @@
+<? include(DOCS_RESOURCES."/rolling-window/introduction.html"); ?>

08 Drafts/05 Historical Data/08 Rolling Window/02 Supported Types.php

@@ -0,0 +1 @@
+<? include(DOCS_RESOURCES."/rolling-window/supported-types.html"); ?>

08 Drafts/05 Historical Data/08 Rolling Window/03 Add Data.php

@@ -0,0 +1 @@
+<? include(DOCS_RESOURCES."/rolling-window/add-data.html"); ?>

08 Drafts/05 Historical Data/08 Rolling Window/04 Warm Up.php

@@ -0,0 +1 @@
+<? include(DOCS_RESOURCES."/rolling-window/warm-up.html"); ?>

08 Drafts/05 Historical Data/08 Rolling Window/05 Check Readiness.php

@@ -0,0 +1 @@
+<? include(DOCS_RESOURCES."/rolling-window/check-readiness.html"); ?>

08 Drafts/05 Historical Data/08 Rolling Window/06 Adjust Size.php

@@ -0,0 +1 @@
+<? include(DOCS_RESOURCES."/rolling-window/adjust-size.html"); ?>

08 Drafts/05 Historical Data/08 Rolling Window/07 Access Data.php

@@ -0,0 +1 @@
+<? include(DOCS_RESOURCES."/rolling-window/access-data.html"); ?>

08 Drafts/05 Historical Data/08 Rolling Window/08 Combine with Indicators.php

@@ -0,0 +1 @@
+<? include(DOCS_RESOURCES."/indicators/historical-values.html"); ?>

08 Drafts/05 Historical Data/08 Rolling Window/09 Combine with Consolidators.php

@@ -0,0 +1 @@
+<? include(DOCS_RESOURCES."/rolling-window/combine-with-consolidators.html"); ?>

08 Drafts/05 Historical Data/08 Rolling Window/10 Cast to Other Types.php

@@ -0,0 +1 @@
+<? include(DOCS_RESOURCES."/rolling-window/cast-to-other-types.html"); ?>

08 Drafts/05 Historical Data/08 Rolling Window/11 Delete Data.php

@@ -0,0 +1 @@
+<? include(DOCS_RESOURCES."/rolling-window/delete-data.html"); ?>

08 Drafts/05 Historical Data/08 Rolling Window/12 Examples.php

@@ -0,0 +1 @@
+<? include(DOCS_RESOURCES."/rolling-window/examples.html"); ?>

08 Drafts/05 Historical Data/08 Rolling Window/metadata.json

@@ -0,0 +1,12 @@
+{
+    "type": "metadata",
+    "values": {
+        "description": "A RollingWindow is an array of a fixed-size that holds trailing data. It's more efficient to store data than to make many historical data requests.",
+        "keywords": "historical data, rolling window, trailing data, indicators, consolidators",
+        "og:description": "A RollingWindow is an array of a fixed-size that holds trailing data. It's more efficient to store data than to make many historical data requests.",
+        "og:title": "Rolling Window - Documentation QuantConnect.com",
+        "og:type": "website",
+        "og:site_name": "Rolling Window - QuantConnect.com",
+        "og:image": "https://cdn.quantconnect.com/docs/i/writing-algorithms/historical-data/rolling-window.png"
+    }
+}

08 Drafts/05 Historical Data/09 Common Errors/01 Introduction.html

@@ -0,0 +1,7 @@
+add Live Trading
+Other data providers
+Trading disabled in warm-up
+ Limitations of quantconnect provider
+Limitations of brokerages
+E.g. IB is slow and leads to timeout if we request or warm up too many symbols (e.g. options)	
+Indicator readiness.