Skip to content

Commit

Permalink
HBASE-23697 Document new RegionProcedureStore operation and migration
Browse files Browse the repository at this point in the history
  • Loading branch information
saintstack committed Apr 15, 2020
1 parent 3421423 commit 88c02c1
Show file tree
Hide file tree
Showing 2 changed files with 25 additions and 2 deletions.
7 changes: 5 additions & 2 deletions src/main/asciidoc/_chapters/architecture.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -693,9 +693,12 @@ See <<arch.catalog.meta>> for more information on the meta table.
[[master.wal]]
=== MasterProcWAL

_MasterProcWAL is replaced in hbase-2.3.0 by an alternate Procedure Store implementation; see
<<in-master-procedure-store-region>>. This section pertains to hbase-2.0.0 through hbase-2.2.x_

HMaster records administrative operations and their running states, such as the handling of a crashed server,
table creation, and other DDLs, into its own WAL file. The WALs are stored under the MasterProcWALs
directory. The Master WALs are not like RegionServer WALs. Keeping up the Master WAL allows
table creation, and other DDLs, into a Procedure Store. The Procedure Store WALs are stored under the
MasterProcWALs directory. The Master WALs are not like RegionServer WALs. Keeping up the Master WAL allows
us run a state machine that is resilient across Master failures. For example, if a HMaster was in the
middle of creating a table encounters an issue and fails, the next active HMaster can take up where
the previous left off and carry the operation to completion. Since hbase-2.0.0, a
Expand Down
20 changes: 20 additions & 0 deletions src/main/asciidoc/_chapters/upgrading.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -319,6 +319,26 @@ Quitting...
The RegionServer Grouping feature has been reimplemented. See section
<<migrating.rsgroup>> in <<ops_mgt>> for more details.

[[upgrade2.3]]
=== Upgrade from 2.0.x-2.2.x to 2.3+
There is no special consideration upgrading to hbase-2.3.x from earlier versions. From 2.2.x, it should be
rolling upgradeable. From 2.1.x or 2.0.x, you will need to clear the <<upgrade2.2>> hurdle first.

[[in-master-procedure-store-region]]
==== New In-Master Procedure Store

Of note, HBase 2.3.0 changes the in-Master Procedure Store implementation. It was a dedicated custom store
(see <<master.wal>>) to instead use a standard HBase Region (link:https://issues.apache.org/jira/browse/HBASE-23326).
The migration from the old to new format is automatic run by the new 2.3.0 Master on startup. The old _MasterProcWALs_
dir which hosted the old custom implementation files in _${hbase.rootdir}_ is deleted on successful
migration. A new _MasterProc_ sub-directory replaces it to host the Store files and WALs for the new
Procedure Store in-Master Region. The in-Master Region is unusual in that it writes to an
alternate location at _${hbase.rootdir}/MasterProc_ rather than under _${hbase.rootdir}/data_ in the
filesystem and the special Procedure Store in-Master Region is hidden from all clients other than the active
Master itself. Otherwise, it is like any other with the Master process running flushes and compactions,
archiving WALs when over-flushed, and so on. Its files are readable by standard Region and Store file
tooling for triage and analysis as long as they are pointed to the appropriate location in the filesystem.

[[upgrade2.2]]
=== Upgrade from 2.0 or 2.1 to 2.2+

Expand Down

0 comments on commit 88c02c1

Please sign in to comment.