From f31d441dce0eb220b1a5f71bad4c5efd995e8bdc Mon Sep 17 00:00:00 2001 From: Raju Jha Date: Fri, 30 Mar 2018 11:05:06 +0530 Subject: [PATCH 01/11] written guide for elasticsearch local --- docs/guides/run-elasticsearch-local.rst | 99 +++++++++++++++++++++++++ 1 file changed, 99 insertions(+) create mode 100644 docs/guides/run-elasticsearch-local.rst diff --git a/docs/guides/run-elasticsearch-local.rst b/docs/guides/run-elasticsearch-local.rst new file mode 100644 index 00000000000..73b498879a7 --- /dev/null +++ b/docs/guides/run-elasticsearch-local.rst @@ -0,0 +1,99 @@ +========================================== +Enabling Elasticsearch on the local server +========================================== + +Read the Docs has been using Elasticsearch which is a platform for distributed search and analysis of data in real time. To enable the search feature on your local installation, you need to install the elasticsearch locally and run the Elastic server. + +Installation has been mainly divided into following steps. + +1. Installing Java +------------------ + +Elasticsearch requires JAVA 8 or later. Use `Oracle official documentation `_. +or opensource distribution like `OpenJDK `_. + +After installing java, verify the installation by,:: + + $ java -version + +The result should be something like this:: + + openjdk version "1.8.0_151" + OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-0ubuntu0.16.04.2-b12) + OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode) + + +2. Downloading and installing Elasticsearch +------------------------------------------- + +Elasticsearch can be downloaded directly from elastic.co in zip, tar.gz, deb, or rpm packages. For Ubuntu, it's best to use the deb (Debian) package which will install everything you need to run Elasticsearch. + +RTD currently uses elasticsearch 1.x which can be easily downloaded and installed from the `Official Website +`_. + + 1. First download the deb package version 1.3.8 from this `Link `_. + + 2. Install the downloaded package by following command + ``$ sudo apt install .{path-to-downloaded-file}/elasticsearch-1.3.8.deb`` + +3. Running Elasticsearch from command line +------------------------------------------ + +Elasticsearch is not started automatically after installation. How to start and stop Elasticsearch depends on whether your system uses SysV init or systemd (used by newer distributions). You can tell which is being used by running this command:: + + $ ps -p 1 + +**Running Elasticsearch with SysV init** + +Use the ``update-rc.d command`` to configure Elasticsearch to start automatically when the system boots up:: + + $ sudo update-rc.d elasticsearch defaults 95 10 + +Elasticsearch can be started and stopped using the service command:: + + $ sudo -i service elasticsearch start + $ sudo -i service elasticsearch stop + +If Elasticsearch fails to start for any reason, it will print the reason for failure to STDOUT. Log files can be found in /var/log/elasticsearch/. + +**Running Elasticsearch with systemd** + +To configure Elasticsearch to start automatically when the system boots up, run the following commands:: + + $ sudo /bin/systemctl daemon-reload + $ sudo /bin/systemctl enable elasticsearch.service + +Elasticsearch can be started and stopped as follows:: + + $ sudo systemctl start elasticsearch.service + $ sudo systemctl stop elasticsearch.service + +To verify run:: + + $ curl http://localhost:9200 + +You should get something like: + +.. code-block:: json + + { + status: 200, + name: "Amina Synge", + version: { + number: "1.3.8", + build_hash: "475733ee0837fba18c00c3ee76cd49a08755550c", + build_timestamp: "2015-02-11T14:45:42Z", + build_snapshot: false, + lucene_version: "4.9" + }, + tagline: "You Know, for Search" + } + +4. Index the data available at RTD database +------------------------------------------- + +In order to search through the RTD database, you need to index it into the elasticsearch index.:: + + $ python manage.py reindex_elasticsearch + +You are ready to go! From b448a885793d500ed21a111dbfde29a04c2114a5 Mon Sep 17 00:00:00 2001 From: Raju Jha Date: Mon, 16 Apr 2018 18:35:02 +0530 Subject: [PATCH 02/11] done requested changes and resolved styling issues --- docs/guides/run-elasticsearch-local.rst | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/docs/guides/run-elasticsearch-local.rst b/docs/guides/run-elasticsearch-local.rst index 73b498879a7..be43357080f 100644 --- a/docs/guides/run-elasticsearch-local.rst +++ b/docs/guides/run-elasticsearch-local.rst @@ -2,14 +2,14 @@ Enabling Elasticsearch on the local server ========================================== -Read the Docs has been using Elasticsearch which is a platform for distributed search and analysis of data in real time. To enable the search feature on your local installation, you need to install the elasticsearch locally and run the Elastic server. +Read the Docs has been using Elasticsearch for indexing and searching. To enable this on your local installation, you need to install elasticsearch and run the Elastic server locally. Installation has been mainly divided into following steps. 1. Installing Java ------------------ -Elasticsearch requires JAVA 8 or later. Use `Oracle official documentation `_. +Elasticsearch requires Java 8 or later. Use `Oracle official documentation `_. or opensource distribution like `OpenJDK `_. After installing java, verify the installation by,:: @@ -26,15 +26,14 @@ The result should be something like this:: 2. Downloading and installing Elasticsearch ------------------------------------------- -Elasticsearch can be downloaded directly from elastic.co in zip, tar.gz, deb, or rpm packages. For Ubuntu, it's best to use the deb (Debian) package which will install everything you need to run Elasticsearch. +Elasticsearch can be downloaded directly from elastic.co. For Ubuntu, it's best to use the deb (Debian) package which will install everything you need to run Elasticsearch. -RTD currently uses elasticsearch 1.x which can be easily downloaded and installed from the `Official Website -`_. +RTD currently uses elasticsearch 1.x which can be easily downloaded and installed from the `elastic.co +`_. - 1. First download the deb package version 1.3.8 from this `Link `_. +Install the downloaded package by following command:: + $ sudo apt install .{path-to-downloaded-file}/elasticsearch-1.3.8.deb - 2. Install the downloaded package by following command - ``$ sudo apt install .{path-to-downloaded-file}/elasticsearch-1.3.8.deb`` 3. Running Elasticsearch from command line ------------------------------------------ @@ -92,7 +91,7 @@ You should get something like: 4. Index the data available at RTD database ------------------------------------------- -In order to search through the RTD database, you need to index it into the elasticsearch index.:: +In order to search through the RTD database, you need to index it into the elasticsearch index:: $ python manage.py reindex_elasticsearch From 9363600883763b3739c159fcd456323be68d80b4 Mon Sep 17 00:00:00 2001 From: Raju Jha Date: Mon, 16 Apr 2018 18:39:03 +0530 Subject: [PATCH 03/11] typo correction --- docs/guides/run-elasticsearch-local.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/guides/run-elasticsearch-local.rst b/docs/guides/run-elasticsearch-local.rst index be43357080f..fb04fe17c92 100644 --- a/docs/guides/run-elasticsearch-local.rst +++ b/docs/guides/run-elasticsearch-local.rst @@ -32,6 +32,7 @@ RTD currently uses elasticsearch 1.x which can be easily downloaded and installe `_. Install the downloaded package by following command:: + $ sudo apt install .{path-to-downloaded-file}/elasticsearch-1.3.8.deb From 4adc8a65d7613b40663b4df5d1c7755fb8970b60 Mon Sep 17 00:00:00 2001 From: Raju Jha Date: Mon, 16 Apr 2018 18:41:51 +0530 Subject: [PATCH 04/11] grammar correction --- docs/guides/run-elasticsearch-local.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/run-elasticsearch-local.rst b/docs/guides/run-elasticsearch-local.rst index fb04fe17c92..4203f562606 100644 --- a/docs/guides/run-elasticsearch-local.rst +++ b/docs/guides/run-elasticsearch-local.rst @@ -28,7 +28,7 @@ The result should be something like this:: Elasticsearch can be downloaded directly from elastic.co. For Ubuntu, it's best to use the deb (Debian) package which will install everything you need to run Elasticsearch. -RTD currently uses elasticsearch 1.x which can be easily downloaded and installed from the `elastic.co +RTD currently uses elasticsearch 1.x which can be easily downloaded and installed from `elastic.co `_. Install the downloaded package by following command:: From 3c73c7c299df3c4ba496a034074a68163e0ef550 Mon Sep 17 00:00:00 2001 From: Raju Jha Date: Mon, 16 Apr 2018 18:45:47 +0530 Subject: [PATCH 05/11] converted indentation to spaces --- docs/guides/run-elasticsearch-local.rst | 54 ++++++++++++------------- 1 file changed, 27 insertions(+), 27 deletions(-) diff --git a/docs/guides/run-elasticsearch-local.rst b/docs/guides/run-elasticsearch-local.rst index 4203f562606..315501a471d 100644 --- a/docs/guides/run-elasticsearch-local.rst +++ b/docs/guides/run-elasticsearch-local.rst @@ -14,13 +14,13 @@ or opensource distribution like `OpenJDK `_. After installing java, verify the installation by,:: - $ java -version + $ java -version The result should be something like this:: - openjdk version "1.8.0_151" - OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-0ubuntu0.16.04.2-b12) - OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode) + openjdk version "1.8.0_151" + OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-0ubuntu0.16.04.2-b12) + OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode) 2. Downloading and installing Elasticsearch @@ -33,7 +33,7 @@ RTD currently uses elasticsearch 1.x which can be easily downloaded and installe Install the downloaded package by following command:: - $ sudo apt install .{path-to-downloaded-file}/elasticsearch-1.3.8.deb + $ sudo apt install .{path-to-downloaded-file}/elasticsearch-1.3.8.deb 3. Running Elasticsearch from command line @@ -41,18 +41,18 @@ Install the downloaded package by following command:: Elasticsearch is not started automatically after installation. How to start and stop Elasticsearch depends on whether your system uses SysV init or systemd (used by newer distributions). You can tell which is being used by running this command:: - $ ps -p 1 + $ ps -p 1 **Running Elasticsearch with SysV init** Use the ``update-rc.d command`` to configure Elasticsearch to start automatically when the system boots up:: - $ sudo update-rc.d elasticsearch defaults 95 10 + $ sudo update-rc.d elasticsearch defaults 95 10 Elasticsearch can be started and stopped using the service command:: - $ sudo -i service elasticsearch start - $ sudo -i service elasticsearch stop + $ sudo -i service elasticsearch start + $ sudo -i service elasticsearch stop If Elasticsearch fails to start for any reason, it will print the reason for failure to STDOUT. Log files can be found in /var/log/elasticsearch/. @@ -60,40 +60,40 @@ If Elasticsearch fails to start for any reason, it will print the reason for fai To configure Elasticsearch to start automatically when the system boots up, run the following commands:: - $ sudo /bin/systemctl daemon-reload - $ sudo /bin/systemctl enable elasticsearch.service + $ sudo /bin/systemctl daemon-reload + $ sudo /bin/systemctl enable elasticsearch.service Elasticsearch can be started and stopped as follows:: - $ sudo systemctl start elasticsearch.service - $ sudo systemctl stop elasticsearch.service + $ sudo systemctl start elasticsearch.service + $ sudo systemctl stop elasticsearch.service To verify run:: - $ curl http://localhost:9200 + $ curl http://localhost:9200 You should get something like: .. code-block:: json - { - status: 200, - name: "Amina Synge", - version: { - number: "1.3.8", - build_hash: "475733ee0837fba18c00c3ee76cd49a08755550c", - build_timestamp: "2015-02-11T14:45:42Z", - build_snapshot: false, - lucene_version: "4.9" - }, - tagline: "You Know, for Search" - } + { + status: 200, + name: "Amina Synge", + version: { + number: "1.3.8", + build_hash: "475733ee0837fba18c00c3ee76cd49a08755550c", + build_timestamp: "2015-02-11T14:45:42Z", + build_snapshot: false, + lucene_version: "4.9" + }, + tagline: "You Know, for Search" + } 4. Index the data available at RTD database ------------------------------------------- In order to search through the RTD database, you need to index it into the elasticsearch index:: - $ python manage.py reindex_elasticsearch + $ python manage.py reindex_elasticsearch You are ready to go! From ec37b68a1144060bb3c966d224a6c96f0c218646 Mon Sep 17 00:00:00 2001 From: Raju Jha Date: Mon, 23 Apr 2018 21:44:28 +0530 Subject: [PATCH 06/11] added link from install.rst to run-elasticsearch-local.rst --- docs/guides/run-elasticsearch-local.rst | 2 ++ docs/install.rst | 4 +--- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/guides/run-elasticsearch-local.rst b/docs/guides/run-elasticsearch-local.rst index 315501a471d..c265b80fd5d 100644 --- a/docs/guides/run-elasticsearch-local.rst +++ b/docs/guides/run-elasticsearch-local.rst @@ -1,3 +1,5 @@ +.. _running-elasticsearch-locally: + ========================================== Enabling Elasticsearch on the local server ========================================== diff --git a/docs/install.rst b/docs/install.rst index ad644d1c946..370ed5b5139 100644 --- a/docs/install.rst +++ b/docs/install.rst @@ -65,9 +65,7 @@ need to install Python 3 with virtualenv in your system as well. If you want full support for searching inside your Read the Docs site you will need to install Elasticsearch_. - Ubuntu users could install this package as following:: - - sudo apt-get install elasticsearch + Ubuntu users could install this package by following :doc:`guides/run-elasticsearch-local`. .. note:: From a5c86fef43928d268fb64f8605dd3add9a6fa182 Mon Sep 17 00:00:00 2001 From: Raju Jha Date: Mon, 23 Apr 2018 21:50:37 +0530 Subject: [PATCH 07/11] removed unwanted line --- docs/guides/run-elasticsearch-local.rst | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/guides/run-elasticsearch-local.rst b/docs/guides/run-elasticsearch-local.rst index c265b80fd5d..315501a471d 100644 --- a/docs/guides/run-elasticsearch-local.rst +++ b/docs/guides/run-elasticsearch-local.rst @@ -1,5 +1,3 @@ -.. _running-elasticsearch-locally: - ========================================== Enabling Elasticsearch on the local server ========================================== From 6cd82f7e6bf5a3d55f20025df668950ad0485cb5 Mon Sep 17 00:00:00 2001 From: Eric Holscher Date: Wed, 6 Jun 2018 16:38:15 -0400 Subject: [PATCH 08/11] Move elastic search guide to custom installs & link to it properly --- docs/{guides => custom_installs}/run-elasticsearch-local.rst | 0 docs/install.rst | 2 +- 2 files changed, 1 insertion(+), 1 deletion(-) rename docs/{guides => custom_installs}/run-elasticsearch-local.rst (100%) diff --git a/docs/guides/run-elasticsearch-local.rst b/docs/custom_installs/run-elasticsearch-local.rst similarity index 100% rename from docs/guides/run-elasticsearch-local.rst rename to docs/custom_installs/run-elasticsearch-local.rst diff --git a/docs/install.rst b/docs/install.rst index f96b1104a6f..1303c5eec04 100644 --- a/docs/install.rst +++ b/docs/install.rst @@ -57,7 +57,7 @@ need to install Python 2.7 with virtualenv in your system as well. If you want full support for searching inside your Read the Docs site you will need to install Elasticsearch_. - Ubuntu users could install this package by following :doc:`guides/run-elasticsearch-local`. + Ubuntu users could install this package by following :doc:`/custom_installs/run-elasticsearch-local`. .. note:: From 09beef8340af04e5a8dde3c1ef35c2ea4a8420dc Mon Sep 17 00:00:00 2001 From: Eric Holscher Date: Wed, 6 Jun 2018 16:41:52 -0400 Subject: [PATCH 09/11] Mention ICU plugin --- .../{run-elasticsearch-local.rst => elasticsearch.rst} | 4 ++++ docs/install.rst | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-) rename docs/custom_installs/{run-elasticsearch-local.rst => elasticsearch.rst} (96%) diff --git a/docs/custom_installs/run-elasticsearch-local.rst b/docs/custom_installs/elasticsearch.rst similarity index 96% rename from docs/custom_installs/run-elasticsearch-local.rst rename to docs/custom_installs/elasticsearch.rst index 315501a471d..c36b0028156 100644 --- a/docs/custom_installs/run-elasticsearch-local.rst +++ b/docs/custom_installs/elasticsearch.rst @@ -72,6 +72,10 @@ To verify run:: $ curl http://localhost:9200 +You need the icu plugin:: + + $ elasticsearch/bin/plugin -install elasticsearch/elasticsearch-analysis-icu/2.3.0 + You should get something like: .. code-block:: json diff --git a/docs/install.rst b/docs/install.rst index 1303c5eec04..83ca8782d62 100644 --- a/docs/install.rst +++ b/docs/install.rst @@ -57,7 +57,7 @@ need to install Python 2.7 with virtualenv in your system as well. If you want full support for searching inside your Read the Docs site you will need to install Elasticsearch_. - Ubuntu users could install this package by following :doc:`/custom_installs/run-elasticsearch-local`. + Ubuntu users could install this package by following :doc:`/custom_installs/elasticsearch`. .. note:: From 288ac2b39f41cf1415cd5402b472b684dc3cb8b0 Mon Sep 17 00:00:00 2001 From: Eric Holscher Date: Thu, 7 Jun 2018 10:19:37 -0400 Subject: [PATCH 10/11] More details --- docs/custom_installs/elasticsearch.rst | 41 +++++++++++++++++++------- 1 file changed, 30 insertions(+), 11 deletions(-) diff --git a/docs/custom_installs/elasticsearch.rst b/docs/custom_installs/elasticsearch.rst index c36b0028156..57e9ba0d9d1 100644 --- a/docs/custom_installs/elasticsearch.rst +++ b/docs/custom_installs/elasticsearch.rst @@ -6,8 +6,8 @@ Read the Docs has been using Elasticsearch for indexing and searching. To enable Installation has been mainly divided into following steps. -1. Installing Java ------------------- +Installing Java +--------------- Elasticsearch requires Java 8 or later. Use `Oracle official documentation `_. or opensource distribution like `OpenJDK `_. @@ -23,8 +23,8 @@ The result should be something like this:: OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode) -2. Downloading and installing Elasticsearch -------------------------------------------- +Downloading and installing Elasticsearch +---------------------------------------- Elasticsearch can be downloaded directly from elastic.co. For Ubuntu, it's best to use the deb (Debian) package which will install everything you need to run Elasticsearch. @@ -35,9 +35,15 @@ Install the downloaded package by following command:: $ sudo apt install .{path-to-downloaded-file}/elasticsearch-1.3.8.deb +Custom setup +------------ -3. Running Elasticsearch from command line ------------------------------------------- +You need the icu plugin:: + + $ elasticsearch/bin/plugin -install elasticsearch/elasticsearch-analysis-icu/2.3.0 + +Running Elasticsearch from command line +--------------------------------------- Elasticsearch is not started automatically after installation. How to start and stop Elasticsearch depends on whether your system uses SysV init or systemd (used by newer distributions). You can tell which is being used by running this command:: @@ -72,9 +78,6 @@ To verify run:: $ curl http://localhost:9200 -You need the icu plugin:: - - $ elasticsearch/bin/plugin -install elasticsearch/elasticsearch-analysis-icu/2.3.0 You should get something like: @@ -93,8 +96,24 @@ You should get something like: tagline: "You Know, for Search" } -4. Index the data available at RTD database -------------------------------------------- +Index the data available at RTD database +---------------------------------------- + +You need to create the indexes:: + + from readthedocs.search.indexes import Index, ProjectIndex, PageIndex, SectionIndex + + index = Index() + index_name = index.timestamped_index() + index.create_index(index_name) + index.update_aliases(index_name) + # Update mapping + proj = ProjectIndex() + proj.put_mapping() + page = PageIndex() + page.put_mapping() + sec = SectionIndex() + sec.put_mapping() In order to search through the RTD database, you need to index it into the elasticsearch index:: From f42fd654541fc5ede72781a57b8b59512f9fb262 Mon Sep 17 00:00:00 2001 From: Eric Holscher Date: Thu, 7 Jun 2018 10:38:29 -0400 Subject: [PATCH 11/11] No json --- docs/custom_installs/elasticsearch.rst | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/custom_installs/elasticsearch.rst b/docs/custom_installs/elasticsearch.rst index 57e9ba0d9d1..5c2cc68496f 100644 --- a/docs/custom_installs/elasticsearch.rst +++ b/docs/custom_installs/elasticsearch.rst @@ -79,9 +79,7 @@ To verify run:: $ curl http://localhost:9200 -You should get something like: - -.. code-block:: json +You should get something like:: { status: 200,